ICML 2024 Papers — Page 26
International Conference on Machine Learning · 2610 papers
UniCorn: A Unified Contrastive Learning Approach for Multi-view Molecular Representation Learning
Shikun Feng (Institute for AI Industry Research), Yanyan Lan (Institute for AI Industry Research)
Representation LearningDrug DiscoveryGraph Neural NetworkContrastive LearningGraphBiomedical DataPhysics Related
🎯 What it does: A unified contrastive learning framework called UniCorn is proposed and implemented, integrating fragment masking, torsion augmented denoising, and 2D-3D cross-modal contrastive distillation to achieve multi-view molecular representation learning, enhancing the prediction performance of quantum, physical chemistry, and biological properties.
Unified Generation, Reconstruction, and Representation: Generalized Diffusion with Adaptive Latent Encoding-Decoding
Guangyi Liu (MBZUAI), Zhiting Hu (Carnegie Mellon University)
GenerationData SynthesisRepresentation LearningProtein Structure PredictionTransformerDiffusion modelImageTextBiomedical Data
🎯 What it does: A general diffusion model EDDPM is designed and implemented, unifying data generation, reconstruction, and latent representation, applicable to various data types such as text, images, and protein sequences.
Unified Training of Universal Time Series Forecasting Transformers
Gerald Woo (Salesforce AI Research), Doyen Sahoo (Salesforce AI Research)
TransformerTime Series
🎯 What it does: A unified training time series prediction Transformer (MOIRAI) is proposed, achieving zero-shot prediction across frequencies, multiple variables, and flexible distributions.
Uniform Memory Retrieval with Larger Capacity for Modern Hopfield Models
Dennis Wu (University of Northwestern), Han Liu (University of Northwestern)
ClassificationRetrievalImageTime Series
🎯 What it does: A two-stage modern Hopfield model retrieval framework U Hop- is proposed, which enhances memory capacity and eliminates memory confusion by learning separable kernel feature mappings Φ.
Uniformly Stable Algorithms for Adversarial Training and Beyond
Jiancong Xiao (University of Pennsylvania), Asuman E. Ozdaglar
OptimizationAdversarial AttackDiffusion modelImage
🎯 What it does: A Moreau Envelope Algorithm (MEA) is designed to achieve uniform stability in adversarial training by transforming the adversarial training problem into a min-min form and alternately solving the inner and outer problems, thereby alleviating robust overfitting.
Unifying Bayesian Flow Networks and Diffusion Models through Stochastic Differential Equations
Kaiwen Xue (Renmin University of China), Chongxuan Li (Renmin University of China)
GenerationData SynthesisComputational EfficiencyDiffusion modelFlow-based ModelImageTextStochastic Differential Equation
🎯 What it does: This paper unifies Bayesian Flow Networks (BFN) and Diffusion Models (DM) by constructing a linear Stochastic Differential Equation (SDE) and proves that the regression loss of BFN is equivalent to Denoising Score Matching (DSM). Consequently, it transfers high-order fast samplers from DM to BFN, proposing BFT-Solvers, which significantly improve sampling quality and speed with a small number of function evaluations (NFE).
Unifying Image Processing as Visual Prompting Question Answering
Yihao Liu (Shanghai Artificial Intelligence Laboratory), Chao Dong (Shenzhen Institute of Advanced Technology)
RestorationSuper ResolutionTransformerPrompt EngineeringImage
🎯 What it does: The PromptGIP model is proposed, unifying low-level image processing (image restoration, enhancement, edge detection, etc.) into a visual prompt question-answering framework, enabling a single model to support multiple tasks.
Universal Consistency of Wide and Deep ReLU Neural Networks and Minimax Optimal Convergence Rates for Kolmogorov-Donoho Optimal Function Classes
Hyunouk Ko (Georgia Institute of Technology), Xiaoming Huo (Georgia Institute of Technology)
ClassificationOptimization
🎯 What it does: This study investigates the universal consistency of wide and deep ReLU neural networks in binary classification and provides the convergence rate at which the neural network classifier achieves the limit of minimized risk for function classes that satisfy the Kolmogorov–Donoho optimal index.
Universal Gradient Methods for Stochastic Convex Optimization
Anton Rodomanov (CISPA Helmholtz Center for Information Security), Volkan Cevher (Laboratory for Information and Inference Systems)
OptimizationTabularBiomedical Data
🎯 What it does: This paper proposes a universal gradient method for Stochastic Convex Optimization (SCO), which can automatically adapt to the noise of the oracle and the Hölder smoothness of the objective function without prior knowledge of the specific settings.
Universality of Linear Recurrences Followed by Non-linear Projections: Finite-Width Guarantees and Benefits of Complex Eigenvalues
Antonio Orvieto (Max Planck Institute for Intelligent Systems), Samuel L Smith
Recurrent Neural NetworkTime SeriesSequential
🎯 What it does: Proves that linear recursion combined with position-aware MLP can approximate any regular causal sequence-to-sequence mapping, and provides sufficient conditions for the hidden state to have no information loss.
Unleashing the Power of Meta-tuning for Few-shot Generalization Through Sparse Interpolated Experts
Shengzhuang Chen (City University of Hong Kong), Ying Wei (Nanyang Technological University)
Domain AdaptationKnowledge DistillationMeta LearningMixture of ExpertsImage
🎯 What it does: A Sparse Meta-Tuning (SMAT) framework has been developed, which integrates pre-trained models with sparse expert interpolation to achieve efficient generalization for few-shot tasks.
Unlock the Cognitive Generalization of Deep Reinforcement Learning via Granular Ball Representation
Jiashun Liu (Tianjin University), Shuyin Xia (CQUPT)
Reinforcement LearningAuto EncoderImage
🎯 What it does: A GB-RL framework based on VAE and Granular Ball is proposed, utilizing reward-agnostic environmental dynamics prediction to achieve cognitive generalization from simple to complex scenarios.
Unlocking the Power of Spatial and Temporal Information in Medical Multimodal Pre-training
Jinxia Yang (Renmin University of China), Ji-Rong Wen (Renmin University of China)
ClassificationTransformerMixture of ExpertsContrastive LearningMultimodalityBiomedical Data
🎯 What it does: This study proposes the Med-ST framework, which utilizes multi-view spatial information and historical time series information from medical multimodal data for visual-text pre-training.
Unmasking Vulnerabilities: Cardinality Sketches under Adaptive Inputs
Sara Ahmadian (Google Research), Edith Cohen (Google Research)
Adversarial Attack
🎯 What it does: This paper studies the vulnerabilities of cardinality sketches (such as HLL++) in adaptive query scenarios and designs attack methods against standard estimators and arbitrary estimators.
Unraveling the Impact of Heterophilic Structures on Graph Positive-Unlabeled Learning
Yuhao Wu (University of Sydney), Tongliang Liu (University of Sydney)
ClassificationOptimizationGraph Neural NetworkGraph
🎯 What it does: This paper proposes a positive and negative sample incomplete learning method for graph-structured data (Graph PU Learning with Label Propagation Loss, GPL), which enhances the performance of positive and negative sample classification and class prior estimation by reducing the impact of heterogeneous edges.
Unsupervised Concept Discovery Mitigates Spurious Correlations
Md Rifat Arefin (Mila University of Montreal), Kenji Kawaguchi (National University of Singapore)
ClassificationRepresentation LearningContrastive LearningImage
🎯 What it does: This paper proposes a concept balancing technique called CoBalT based on unsupervised object-centric learning, aimed at alleviating short-sighted correlations in training data without the need for manual subgroup labels.
Unsupervised Domain Adaptation for Anatomical Structure Detection in Ultrasound Images
Bin Pu (Hong Kong University of Science and Technology), Xiaomeng Li (Hong Kong University of Science and Technology)
Object DetectionDomain AdaptationGraph Neural NetworkImageBiomedical DataUltrasound
🎯 What it does: This paper proposes an unsupervised domain adaptation method for ultrasound fetal structure detection, ToMo-UDA, which combines topological and morphological knowledge transfer.
Unsupervised Episode Generation for Graph Meta-learning
Jihyeong Jung (KAIST), Chanyoung Park (KAIST)
Meta LearningGraph Neural NetworkGraph
🎯 What it does: This paper proposes an unsupervised episode generation method called NAQ for solving few-shot node classification (FSNC) tasks in graph meta-learning.
Unsupervised Evaluation of Code LLMs with Round-Trip Correctness
Miltiadis Allamanis (Google DeepMind), Pengcheng Yin (Google DeepMind)
AI Code AssistantTransformerLarge Language ModelText
🎯 What it does: This paper proposes an unsupervised evaluation method that does not require manual annotation—Round-Trip Correctness (RTC), which can be used to assess the code synthesis and code editing capabilities of code language models.
Unsupervised Parameter-free Simplicial Representation Learning with Scattering Transforms
Hiren Madhu (Indian Institute of Science), Sundeep Prabhakar Chepuri (Indian Institute of Science)
Representation LearningGraph Neural NetworkGraph
🎯 What it does: An unsupervised, non-parametric simplicial scattering network (SSN) is proposed for extracting task-independent representations from simplicial complexes.
Unsupervised Representation Learning of Brain Activity via Bridging Voxel Activity and Functional Connectivity
Ali Behrouz (Cornell University), Farnoosh Hashemi (University of British Columbia)
Anomaly DetectionRepresentation LearningGraph Neural NetworkTransformerContrastive LearningTime SeriesBiomedical DataMagnetic Resonance ImagingAlzheimer's Disease
🎯 What it does: An unsupervised brain activity representation learning framework named BRAINMIXER is proposed, which can simultaneously utilize voxel-level time series and functional connectivity graphs to learn low-dimensional brain representations.
Unsupervised Zero-Shot Reinforcement Learning via Functional Reward Encodings
Kevin Frans (University of California), Sergey Levine (University of California)
TransformerReinforcement LearningSequential
🎯 What it does: This paper proposes an unsupervised zero-shot reinforcement learning method called Functional Reward Encoding (FRE), which pre-trains a general policy from unlabeled offline trajectories and achieves zero-shot adaptation to new tasks with only a few reward samples.
Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibration
Zhongzhi Yu (Georgia Institute of Technology), Yingyan Celine Lin
ClassificationTransformerLarge Language ModelText
🎯 What it does: This paper studies the distribution and impact of 'attention sinks' in large language models (LLMs), finding that attention sinks can occur not only at the first input word but also at intermediate words, and not all sinks are beneficial; it then proposes a training-free technique for real-time attention calibration during inference—Attention Calibration Technique (ACT)—which enhances the accuracy of LLMs on various tasks by weakening the attention at sinks and redistributing it to other words.
Unveiling Privacy, Memorization, and Input Curvature Links
Deepak Ravikumar (Purdue University), Kaushik Roy (Purdue University)
ClassificationSafty and PrivacyConvolutional Neural NetworkImage
🎯 What it does: This study investigates the theoretical relationship between memorization of deep learning models, input loss curvature, and differential privacy, and experimentally verifies three upper bound relationships.
Unveiling the Cycloid Trajectory of EM Iterations in Mixed Linear Regression
Zhankun Luo (Purdue University), Abolfazl Hashemi (Purdue University)
Tabular
🎯 What it does: This paper provides an analysis of the EM algorithm for two groups of mixed linear regression (2MLR), presenting closed-form updates for the first-order EM iteration at all signal-to-noise ratios (SNR) (using Bessel functions). It proves that the EM iteration trajectory moves along a cycloid curve in the noise-free limit, further demonstrating that the convergence rate of EM in this limit is superlinear (essentially quadratic convergence) and insensitive to mixed weights, while also providing an upper bound for statistical error.
Unveiling the Dynamics of Information Interplay in Supervised Learning
Kun Song (University of Science and Technology Beijing), Weiran Huang (Shanghai Jiao Tong University)
ClassificationOptimizationConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: This paper introduces the Matrix Information Ratio (MIR) and Matrix Entropy Difference Ratio (HDR) through matrix information theory, studying the dynamic mutual information between sample representations and classification head vectors in supervised learning, and uses it as a loss constraint to enhance the performance of supervised and semi-supervised learning.
Unveiling the Potential of AI for Nanomaterial Morphology Prediction
Ivan Dubrovsky (ITMO University), Vladimir Vinogradov (ITMO University)
ClassificationGenerationTransformerLarge Language ModelAuto EncoderImage
🎯 What it does: A dataset of 215 experimental synthesis data of calcium carbonate nanomaterials was constructed, and machine learning and large language models were used to predict the shape and size of nanoparticles.
UP2ME: Univariate Pre-training to Multivariate Fine-tuning as a General-purpose Framework for Multivariate Time Series Analysis
Yunhao Zhang (Shanghai Jiao Tong University), Junchi Yan (Shanghai Jiao Tong University)
Anomaly DetectionGraph Neural NetworkTransformerAuto EncoderTime Series
🎯 What it does: A general multivariate time series analysis framework UP2ME is proposed, which achieves multi-tasking such as prediction, imputation, and anomaly detection without parameter changes by first performing self-supervised Masked AutoEncoder pre-training on univariate data and then fine-tuning on multivariate data.
UPAM: Unified Prompt Attack in Text-to-Image Generation Models Against Both Textual Filters and Visual Checkers
Duo Peng (Singapore University of Technology and Design), Jun Liu (Lancaster University)
GenerationAdversarial AttackTransformerLarge Language ModelPrompt EngineeringImageText
🎯 What it does: A unified prompt attack framework UPAM is proposed, capable of simultaneously deceiving text filters and visual inspectors in black-box text-image generation models, generating natural adversarial prompts to induce the generation of target harmful images.
UPOCR: Towards Unified Pixel-Level OCR Interface
Dezhi Peng (South China University of Technology), Lianwen Jin (South China University of Technology)
RecognitionSegmentationGenerationTransformerImage
🎯 What it does: Proposes the UPOCR unified pixel-level OCR interface, integrating text removal, text segmentation, and tampered text detection into a RGB→RGB image transformation task, using a ViT encoder-decoder with learnable task prompts.
Use Your INSTINCT: INSTruction optimization for LLMs usIng Neural bandits Coupled with Transformers
Xiaoqiang Lin (National University of Singapore), Bryan Kian Hsiang Low (National University of Singapore)
OptimizationTransformerLarge Language ModelReinforcement LearningPrompt EngineeringText
🎯 What it does: To optimize instructions for black-box large language models, the INSTINCT algorithm is proposed.
Using AI Uncertainty Quantification to Improve Human Decision-Making
Laura Marusich, Murat Kantarcioglu (University of Texas at Dallas)
ClassificationTabular
🎯 What it does: This study evaluates the impact of providing human decision-makers with high-quality, calibrated instance-level AI uncertainty quantification (UQ) information on decision accuracy and confidence calibration.
Using Left and Right Brains Together: Towards Vision and Language Planning
Jun CEN, Jianguo Zhang (Southern University of Science and Technology)
GenerationAutonomous DrivingTransformerLarge Language ModelVision Language ModelDiffusion modelVideoTextMultimodalityChain-of-Thought
🎯 What it does: A framework called VLP is proposed, which simultaneously conducts visual planning (future video generation) and language planning (Chain-of-Thought) and integrates both through a large multimodal model to accomplish multimodal tasks.
Using Uncertainty Quantification to Characterize and Improve Out-of-Domain Learning for PDEs
S Chandra Mouli, Bernie Wang
Domain AdaptationOptimizationNeural Radiance FieldTime SeriesPhysics Related
🎯 What it does: This study investigates the out-of-distribution (OOD) learning capabilities of neural operators in handling partial differential equations (PDEs) and proposes a new uncertainty quantification method.
USTAD: Unified Single-model Training Achieving Diverse Scores for Information Retrieval
Seungyeon Kim (Google Research), Sanjiv Kumar (Google Research)
RetrievalRecommendation SystemKnowledge DistillationTransformerScore-based ModelText
🎯 What it does: USTAD is a unified Transformer single model that can simultaneously perform retrieval (dual-encoder) and ranking (cross-encoder) tasks on the same network;
Vague Prototype-Oriented Diffusion Model for Multi-Class Anomaly Detection
Yuxin Li (Xidian University), Mingyuan Zhou (University of Texas at Austin)
Anomaly DetectionDiffusion modelImage
🎯 What it does: This paper proposes a new framework for multi-class unsupervised anomaly detection called the Vague Prototype-Oriented Diffusion Model (VPDM). It achieves modeling of normal distributions and anomaly localization by first extracting vague prototypes and then gradually refining them using a diffusion model.
Value-Evolutionary-Based Reinforcement Learning
Pengyi Li (Tianjin University), Fazl Barez (University of Oxford)
Reinforcement Learning
🎯 What it does: The VEB-RL framework is proposed, which combines evolutionary algorithms with value-based reinforcement learning by maintaining a Q-network within the population, using negative TD error to evaluate individuals, and employing elite interaction to enhance sample quality, thereby improving value function learning and policy performance.
Vanilla Bayesian Optimization Performs Great in High Dimensions
Carl Hvarfner (Lund University), Luigi Nardi (DBTune)
OptimizationTabular
🎯 What it does: The research found that the fundamental reason for the poor performance of traditional Bayesian optimization in high-dimensional problems is the excessive model complexity. It proposes to reduce complexity by scaling the Gaussian process kernel length scale prior by dimension, thereby improving the performance of vanilla BO.
Variance-reduced Zeroth-Order Methods for Fine-Tuning Language Models
Tanmay Gautam (University of California), Wooseok Ha (Amazon AI Labs)
OptimizationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: A variant of the stochastic variance-reduced gradient (SVRG) method called MeZO-SVRG is proposed for fine-tuning large-scale language models under memory-constrained conditions.
Variational Inference with Coverage Guarantees in Simulation-Based Inference
Yash Patel (University of Michigan), Ambuj Tewari (University of Michigan)
Flow-based ModelTabularBenchmark
🎯 What it does: This paper proposes the CANVI framework, which adds distribution-independent marginal coverage guarantees to variational posterior distributions and selects the most informative predictive regions among various candidate posteriors.
Variational Learning is Effective for Large Deep Networks
Yuesong Shen (Technical University of Munich), Thomas Möllenhoff (RIKEN Center for AI Project)
OptimizationTransformerLarge Language ModelImageText
🎯 What it does: An improved variational online Newton optimizer IVON is proposed and implemented, capable of directly optimizing variational objectives on large-scale deep networks (such as GPT-2, ResNet), while enhancing accuracy and uncertainty estimation with a computational cost similar to Adam.
Variational Linearized Laplace Approximation for Bayesian Deep Learning
Luis A. Ortega (Universidad Autónoma de Madrid), Daniel Hernández-Lobato (Universidad Pontificia Comillas)
ClassificationOptimizationComputational EfficiencyConvolutional Neural NetworkImageTabular
🎯 What it does: A variational linearized Laplace approximation (VaLLA) is proposed, which uses sparse Gaussian processes to maintain the predictions of pre-trained deep networks as means in RKHS and quickly estimate uncertainty.
Variational Partial Group Convolutions for Input-Aware Partial Equivariance of Rotations and Color-Shifts
Hyunsu Kim (Korea Advanced Institute of Science and Technology), Juho Lee (Korea Advanced Institute of Science and Technology)
ClassificationConvolutional Neural NetworkImage
🎯 What it does: Proposed an input-aware Variational Partial G-CNN, achieving adaptive partial equivariance for each input sample;
Variational Schrödinger Diffusion Models
Wei Deng (Morgan Stanley), Ricky T. Q. Chen (Meta AI)
GenerationData SynthesisOptimizationDiffusion modelTime SeriesPhysics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: This paper proposes a Variational Schrödinger Diffusion Model (VSDM), which linearizes the forward score through variational inference to eliminate the simulation overhead of backward training, and constructs a scalable multivariate diffusion generative framework by utilizing stochastic approximation for adaptive optimization of the forward score.
Various Lengths, Constant Speed: Efficient Language Modeling with Lightning Attention
Zhen Qin (TapTap), Yiran Zhong (OpenNLPLab)
OptimizationComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: A linear attention implementation named Lightning Attention is designed, and a TNL language model architecture for long sequences is proposed.
Vector Quantization Pretraining for EEG Time Series with Random Projection and Phase Alignment
Haokun GUI, Xinyang Chen (Harbin Institute of Technology)
ClassificationAnomaly DetectionRepresentation LearningTransformerSupervised Fine-TuningTime SeriesBiomedical DataElectrocardiogram
🎯 What it does: A BERT-style self-supervised pre-training model VQ-MTM is proposed for semantic unit generation and context representation learning of electroencephalogram (EEG) time series.
Vectorized Conditional Neural Fields: A Framework for Solving Time-dependent Parametric Partial Differential Equations
Jan Hagnberger (University of Stuttgart), Mathias Niepert (University of Stuttgart)
TransformerTime SeriesPhysics Related
🎯 What it does: The Vectorized Conditional Neural Fields (VCNeF) model is proposed for solving parameterized partial differential equations (PDEs) in continuous time.
Verification of Machine Unlearning is Fragile
Binchi Zhang (University of Virginia), Jundong Li (University of Virginia)
ClassificationComputational EfficiencyImage
🎯 What it does: This paper studies the verification vulnerabilities of machine unlearning and proposes two adversarial unlearning methods that can retain the model information of deleted data while satisfying existing backdoor verification and retraining verification.
Verifying message-passing neural networks via topology-based bounds tightening
Christopher Hojny (Eindhoven University of Technology), Ruth Misener (Imperial College London)
OptimizationComputational EfficiencyAdversarial AttackGraph Neural NetworkReinforcement LearningGraph
🎯 What it does: This paper proposes a boundary tightening method based on graph structures to provide verifiable proofs of the robustness of message passing neural networks (MPNNs).
Video-LaVIT: Unified Video-Language Pre-training with Decoupled Visual-Motional Tokenization
Yang Jin (Peking University), Yadong MU
RecognitionGenerationTransformerLarge Language ModelVision Language ModelVideoTextMultimodality
🎯 What it does: Proposes Video-LaVIT, which utilizes video decomposition into key frames and motion vectors, performing large-scale autoregressive pre-training of videos, images, and text in a unified discretized token space, thereby achieving multimodal understanding and generation;
Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognition
Hao Fei (National University of Singapore), Wynne Hsu (National University of Singapore)
GenerationData SynthesisRetrievalTransformerLarge Language ModelPrompt EngineeringVideoMultimodalityChain-of-Thought
🎯 What it does: This paper proposes the MotionEpic video multimodal large language model and the Video-of-Thought (VoT) step-by-step reasoning framework, which can provide complete answers to complex video question-answering problems from pixel-level perception to cognitive reasoning.
video-SALMONN: Speech-Enhanced Audio-Visual Large Language Models
Guangzhi Sun (Tsinghua University), Chao Zhang (Tsinghua University)
RecognitionGenerationTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelVideoTextMultimodalityBenchmarkAudio
🎯 What it does: An end-to-end multimodal large language model, video-SALMONN, is proposed, capable of simultaneously understanding video frames, speech, non-speech audio, and music, achieving comprehensive video semantic analysis and reasoning.
VideoPoet: A Large Language Model for Zero-Shot Video Generation
Dan Kondratyuk (Google), Lu Jiang (Carnegie Mellon University)
GenerationData SynthesisTransformerLarge Language ModelImageVideoTextMultimodalityAudio
🎯 What it does: We propose VideoPoet, a multimodal large language model based on a decoder-only transformer that can generate high-quality videos from various inputs such as text, images, videos, and audio in a zero-shot manner.
VideoPrism: A Foundational Visual Encoder for Video Understanding
Long Zhao (Google), Boqing Gong (Google)
ClassificationRetrievalTransformerContrastive LearningVideoText
🎯 What it does: This paper presents VideoPrism, a freezeable foundational video encoder that achieves optimal performance across various video understanding tasks (classification, localization, retrieval, captioning, and question answering).
Viewing Transformers Through the Lens of Long Convolutions Layers
Itamar Zimerman (Blavatnik School of Computer Science), Lior Wolf (Blavatnik School of Computer Science)
TransformerTextSequentialBenchmark
🎯 What it does: Structural improvements to the Transformer are made by introducing Local Smoothing Attention (LaS‑Attention), which adds smoothing and locality bias to attention through exponential decay and 1D average pooling without learning parameters.
VinT-6D: A Large-Scale Object-in-hand Dataset from Vision, Touch and Proprioception
Zhaoliang Wan (Sun Yat-sen University), Hui Cheng (Sun Yat-sen University)
Pose EstimationRobotic IntelligenceMultimodalityBenchmark
🎯 What it does: This paper constructs a multimodal large object handheld pose estimation dataset VinT-6D, which includes visual, tactile, and proprioceptive data, and proposes a benchmark method VinT-Net.
ViP: A Differentially Private Foundation Model for Computer Vision
Yaodong Yu (University of California Berkeley), Chuan Guo (Meta)
RecognitionObject DetectionSegmentationData SynthesisSafty and PrivacyTransformerAuto EncoderContrastive LearningImage
🎯 What it does: This paper proposes a complete process for training visual foundation models using differential privacy (DP), implementing self-supervised learning with Masked Autoencoders (MAE), and accelerating DP-SGD training through pre-training on synthetic data, ultimately achieving a privacy-preserving Vision Transformer (ViP) on LAION400M.
Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model
Lianghui Zhu (Huazhong University of Science and Technology), Xinggang Wang (Huazhong University of Science and Technology)
ClassificationObject DetectionSegmentationRepresentation LearningConvolutional Neural NetworkTransformerImage
🎯 What it does: Proposes Vision Mamba (Vim), a visual representation learning framework purely based on a bidirectional state space model (SSM);
Vision Transformers as Probabilistic Expansion from Learngene
Qiufeng Wang (Southeast University), Xin Geng (Southeast University)
ClassificationRecognitionTransformerImage
🎯 What it does: Proposes the PEG method, which performs probabilistic mixed sampling and scalable initialization for Vision Transformers.
VisionGraph: Leveraging Large Multimodal Models for Graph Theory Problems in Visual Context
yunxin li, Min Zhang (Harbin Institute of Technology)
Large Language ModelPrompt EngineeringMultimodalityGraphBenchmark
🎯 What it does: This paper proposes a benchmark for visual graph theory problems called VisionGraph, and evaluates it using multimodal large models (such as GPT-4V and Gemini). It further designs a Description-Program-Reasoning (DPR) agent to enhance multi-step reasoning capabilities.
Visual Representation Learning with Stochastic Frame Prediction
Huiwon Jang (Korea Advanced Institute of Science and Technology), Younggyo Seo
GenerationRepresentation LearningRobotic IntelligenceTransformerContrastive LearningVideo
🎯 What it does: A visual representation learning framework based on random frame prediction, RSP, has been developed, which combines random priors from video generation and MAE auxiliary tasks to learn temporal information across frames and spatial details.
Visual Transformer with Differentiable Channel Selection: An Information Bottleneck Inspired Approach
Yancheng Wang (Arizona State University), Yingzhen Yang (Arizona State University)
ClassificationObject DetectionSegmentationCompressionOptimizationTransformerImage
🎯 What it does: A differentiable channel selection Transformer block, DCS-Transformer, is proposed and integrated into visual Transformers such as MobileViT and EfficientViT to compress the model and enhance performance.
Visual-Text Cross Alignment: Refining the Similarity Score in Vision-Language Models
Jinhao Li (University of Melbourne), Feng Liu (University of Melbourne)
ClassificationTransformerLarge Language ModelVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: Proposes the Weighted Visual-Text Cross Alignment (WCA) method, which utilizes pre-trained Visual-Language Models (VLM) and fine-grained text descriptions generated by large language models (LLM) to perform weighted cross alignment on local regions of images, enhancing zero-shot image classification performance.
VNN: Verification-Friendly Neural Networks with Hard Robustness Guarantees
Anahita Baninajjar (Lund University), Amir Aminifar (Lund University)
OptimizationBiomedical DataElectrocardiogram
🎯 What it does: By performing layer-wise sparsification optimization on a pre-trained deep neural network, a verifiable VNN is obtained while maintaining the original prediction accuracy;
Vocabulary for Universal Approximation: A Linguistic Perspective of Mapping Compositions
Yongqiang Cai (Beijing Normal University)
Ordinary Differential Equation
🎯 What it does: A finite mapping vocabulary is proposed, allowing any continuous mapping on any compact domain to be approximated by combinations of mappings from this vocabulary (i.e., 'sentences').
VoroNav: Voronoi-based Zero-shot Object Navigation with Large Language Model
Pengying Wu (Peking University), Chang Liu (Peking University)
Robotic IntelligenceReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningPrompt EngineeringImageText
🎯 What it does: We propose VoroNav, a zero-shot target navigation framework based on Voronoi diagrams, which utilizes LLM to infer paths and scene descriptions.
VQDNA: Unleashing the Power of Vector Quantization for Multi-Species Genomic Sequence Modeling
Siyuan Li (Westlake University), Stan Z. Li (Westlake University)
TransformerAuto EncoderBiomedical DataBenchmark
🎯 What it does: This paper proposes the VQDNA framework, which achieves end-to-end tokenization of gene sequences through vector quantization learning of a variable vocabulary, and is pre-trained on multi-species genomes.
WARM: On the Benefits of Weight Averaged Reward Models
Alexandre Rame (Google DeepMind), Johan Ferret (Google DeepMind)
Reinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextBenchmarkChain-of-Thought
🎯 What it does: This study investigates a method for enhancing the reliability and robustness of large language models (LLMs) in aligning with human preferences by averaging multiple reward models in the weight space (WARM).
Wasserstein Wormhole: Scalable Optimal Transport Distance with Transformer
Doron Haviv (Sloan Kettering Institute), Dana Pe'er (Memorial Sloan Kettering Cancer Center)
TransformerAuto EncoderPoint Cloud
🎯 What it does: This paper proposes a Transformer autoencoder called Wasserstein Wormhole, which embeds point cloud distributions of arbitrary dimensions and variable lengths into Euclidean latent space, allowing the Euclidean distance to approximate the Wasserstein distance. This enables linear time optimal transport (OT) computation on large-scale distribution sets and supports decoding and reconstruction.
Watermark Stealing in Large Language Models
Nikola Jovanović (ETH Zurich), Martin Vechev (ETH Zurich)
GenerationAdversarial AttackTransformerLarge Language ModelText
🎯 What it does: This paper studies the security of watermarking in large language models (LLMs), proposing and implementing an automated watermark-stealing attack, and evaluating the impact of this theft on spoofing and scrubbing attacks in real-world scenarios.
Watermarks in the Sand: Impossibility of Strong Watermarking for Language Models
Hanlin Zhang (Harvard University), Boaz Barak (Harvard University)
Adversarial AttackTransformerLarge Language ModelImageText
🎯 What it does: A general quality-preserving random walk attack is proposed, proving that under the assumptions of quality oracle and perturbation oracle, strong watermarking is unachievable for language models;
WAVES: Benchmarking the Robustness of Image Watermarks
Bang An (University of Maryland), Furong Huang (University of Maryland)
Diffusion modelAuto EncoderImageBenchmark
🎯 What it does: A WAVES benchmark has been constructed for the systematic evaluation of the robustness of image watermarks under various realistic attacks.
Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision
Collin Burns (OpenAI), Jeffrey Wu
ClassificationReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This study investigates whether weak supervision can enhance the capabilities of stronger models by fine-tuning a strong model with weak model labels and observing its generalization performance.
Weakly Convex Regularisers for Inverse Problems: Convergence of Critical Points and Primal-Dual Optimisation
Zakhar Shumaylov (University of Cambridge), Carola-Bibiane Schönlieb
RestorationOptimizationConvolutional Neural NetworkGenerative Adversarial NetworkImageComputed Tomography
🎯 What it does: A weakly convex regularization framework is proposed, and its convergence at critical points is proven. Additionally, the convergence and asymptotic rate of the forward-backward gradient algorithm based on weakly convex regularization are provided, and this framework is implemented as an Input Weakly Convex Neural Network (IWCNN) for CT reconstruction.
Weakly-Supervised Residual Evidential Learning for Multi-Instance Uncertainty Estimation
Pei Liu (University of Electronic Science and Technology of China), Luping Ji (University of Electronic Science and Technology of China)
ClassificationAnomaly DetectionConvolutional Neural NetworkImageBiomedical Data
🎯 What it does: This paper proposes a weakly supervised uncertainty estimation framework for multi-instance learning (MIL) called MIREL, which models uncertainty at both the bag level and instance level using residual evidence learning under the condition of not having complete instance labels.
WebLINX: Real-World Website Navigation with Multi-Turn Dialogue
Xing Han Lu (McGill University), Siva Reddy (Mila Quebec AI Institute)
RetrievalCompressionRecommendation SystemTransformerLarge Language ModelSupervised Fine-TuningTextMultimodalityBenchmarkRetrieval-Augmented Generation
🎯 What it does: The WEBLINX benchmark is proposed, covering 2,337 real website multi-turn dialogue navigation examples, and a Dense Markup Ranking (DMR) method is designed to effectively compress the DOM.
Weighted distance nearest neighbor condensing
Lee-Ad Gottlieb (Ariel University), Roi Weiss (Ariel University)
CompressionTabular
🎯 What it does: A weighted nearest neighbor (WNN) compression algorithm is proposed, assigning weights to each point in the compressed set to improve compression effectiveness;
Weisfeiler Leman for Euclidean Equivariant Machine Learning
Snir Hordan (Technion Institute of Technology), Nadav Dym (Technion Institute of Technology)
Graph Neural NetworkPoint Cloud
🎯 What it does: This paper proposes a Euclidean equivariant graph neural network framework based on the 2-WL test, and achieves unified separation of 3D point clouds and their position-velocity pairs through PPGN, ultimately constructing the WeLNet model that can achieve full equivariance generality.
Weisfeiler-Leman at the margin: When more expressivity matters
Billy Joe Franks, Floris Geerts (University of Antwerp)
Graph Neural NetworkGraph
🎯 What it does: The theoretical and experimental analysis of the expressive power and generalization performance of the extended 1-WL and MPNN from the perspective of graph isomorphism is conducted, providing upper and lower bounds on the VC dimension based on distance.
What Can Transformer Learn with Varying Depth? Case Studies on Sequence Learning Tasks
Xingwu Chen (University of Hong Kong), Difan Zou (University of Hong Kong)
TransformerSequential
🎯 What it does: This study investigates the learning capabilities of the Transformer at different depths (i.e., number of attention layers), designs four types of sequence tasks (memory, reasoning, generalization, context generalization), and demonstrates the impact of different layer counts on each task from both theoretical and experimental perspectives.
What Improves the Generalization of Graph Transformers? A Theoretical Dive into the Self-attention and Positional Encoding
Hongkang Li (Rensselaer Polytechnic Institute), Pin-Yu Chen (IBM)
Graph Neural NetworkTransformerGraph
🎯 What it does: This paper provides a theoretical analysis of shallow graph Transformers, offering quantifiable bounds on sample complexity and iteration count for semi-supervised node classification tasks, and proving that self-attention and positional encoding can enhance generalization performance.
What is Dataset Distillation Learning?
William Yang (Princeton University), Olga Russakovsky (Princeton University)
Knowledge DistillationImage
🎯 What it does: This paper studies the information storage mechanism of dataset distillation and systematically analyzes the performance of distilled data in training, inference, information content, and semantic expression.
What is the Long-Run Distribution of Stochastic Gradient Descent? A Large Deviations Analysis
Waïss Azizian (University of Grenoble Alpes), Panayotis Mertikopoulos (University of Grenoble Alpes)
OptimizationStochastic Differential Equation
🎯 What it does: This study investigates the long-term distribution of Stochastic Gradient Descent (SGD) in non-convex problems, revealing its nature concentrated around critical points and controlled by energy potential traps;
What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation
Aaditya K Singh, Andrew M Saxe
TransformerImage
🎯 What it does: Trained a two-layer attention Transformer on synthetic few-shot learning tasks to study how the induction head forms and functions during the training process, along with its related circuits.
What Will My Model Forget? Forecasting Forgotten Examples in Language Model Refinement
Xisen Jin (University of Southern California), Xiang Ren (University of Southern California)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: The study investigates how to predict which pre-training samples will be forgotten during the fine-tuning of language models, in order to select appropriate replay samples to mitigate catastrophic forgetting during model correction.
What Would Gauss Say About Representations? Probing Pretrained Image Models using Synthetic Gaussian Benchmarks
Ching-Yun Ko (Massachusetts Institute of Technology), Luca Daniel (Massachusetts Institute of Technology)
Representation LearningAdversarial AttackTransformerGaussian SplattingImageBenchmark
🎯 What it does: This paper proposes a label-free, task-agnostic evaluation framework called SynBench, based on synthetic Gaussian mixtures, to quantify the representation quality of pre-trained image models and measure the trade-off between robustness and accuracy.
What’s the score? Automated Denoising Score Matching for Nonlinear Diffusions
Raghav Singhal (New York University), Rajesh Ranganath (New York University)
Diffusion modelScore-based ModelImagePhysics RelatedOrdinary Differential Equation
🎯 What it does: An automated denoising score matching (Automated DSM) framework suitable for nonlinear diffusion processes is proposed, along with a local-DSM (local-DSM) variational lower bound;
When and How Does In-Distribution Label Help Out-of-Distribution Detection?
Xuefeng Du (University of Wisconsin Madison), Yixuan Li (University of Wisconsin Madison)
Anomaly DetectionGraph Neural NetworkContrastive LearningImage
🎯 What it does: This paper studies the impact of using ID labels on OOD detection performance in discrete distribution detection and provides a theoretical lower bound on error.
When Do Skills Help Reinforcement Learning? A Theoretical Analysis of Temporal Abstractions
Zhening Li (Massachusetts Institute of Technology), Armando Solar-Lezama (Massachusetts Institute of Technology)
Reinforcement Learning
🎯 What it does: In deterministic sparse reward MDPs (DSMDP), the theoretical analysis of skills (temporal abstractions) on reinforcement learning is conducted, proposing two types of difficulty metrics (p-learning, p-exploration) and introducing a non-compressibility metric to measure the compressibility of skills in the environment.
When is Transfer Learning Possible?
My Phan (Cornell University), Geoffrey J. Gordon (Carnegie Mellon University)
Domain AdaptationReinforcement LearningImage
🎯 What it does: A unified Structural Causal Model (SCM) framework has been constructed to describe and analyze the feasibility of cross-environment transfer learning, proposing a method to update the unknown parameter space through constraint propagation and computation trees.
When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models
Haoran You (Georgia Institute of Technology), Yingyan Celine Lin (Georgia Institute of Technology)
GenerationComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: This paper systematically evaluates and improves the application of linear attention in autoregressive large language models, combining it with speculative decoding to enhance training and inference efficiency.
When Representations Align: Universality in Representation Learning Dynamics
Loek van Rossem (University College London), Andrew M Saxe
Representation LearningConvolutional Neural NetworkTransformerImageStochastic Differential Equation
🎯 What it does: An effective theory has been constructed to describe the dynamics of representation learning between two points during the gradient descent process in large-scale, expressive neural networks, and its universality has been validated across various network architectures and activation functions.
When Will Gradient Regularization Be Harmful?
Yang Zhao (Tsinghua University), Xiuyuan Hu (Tsinghua University)
OptimizationTransformerImage
🎯 What it does: This paper investigates the reasons for performance degradation when gradient regularization (GR) is used in conjunction with adaptive optimizers (such as Adam and RMSProp) and learning rate warmup (LR warmup), and proposes three GR warmup strategies to mitigate this issue.
Which Frequencies do CNNs Need? Emergent Bottleneck Structure in Feature Learning
Yuxiao Wen (New York University), Arthur Jacot (New York University)
ClassificationRepresentation LearningConvolutional Neural NetworkImage
🎯 What it does: This paper studies and proves that a Convolution Bottleneck (CBN) structure naturally arises in deep convolutional neural networks with L2 regularization, where the network retains only a limited number of Fourier frequencies in the intermediate layers, thereby reducing the representation dimension and enhancing generalization.
Whispering Experts: Neural Interventions for Toxicity Mitigation in Language Models
Xavier Suau (Apple), Pau Rodriguez
GenerationOptimizationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes a hyperparameter-free AURA intervention method to reduce toxic generation by identifying and suppressing expert neurons in language models responsible for producing toxic language, while maintaining overall model performance.
Why Do Animals Need Shaping? A Theory of Task Composition and Curriculum Learning
Jin Hwa Lee (University College London), Andrew M Saxe
Robotic IntelligenceReinforcement LearningOrdinary Differential Equation
🎯 What it does: This study investigates the 'shaping' process in animal learning within a theoretical framework, constructing a linear combination subtask model based on a teacher-student setup, and deriving the learning dynamics ODE in high-dimensional limits.
Why do Variational Autoencoders Really Promote Disentanglement?
Pratik Bhowal (NVIDIA), Sirisha Rambhatla (University of Waterloo)
GenerationRepresentation LearningAuto EncoderImage
🎯 What it does: This paper conducts theoretical and experimental research on the mechanism of Variational Autoencoders (VAE) in learning separable representations, focusing on how local nonlinearity and orthogonality in the decoder promote separability.
Why Do You Grok? A Theoretical Analysis on Grokking Modular Addition
Mohamad Amin Mohamadi (Toyota Technological Institute at Chicago), Danica J. Sutherland (University of British Columbia)
Supervised Fine-TuningTabular
🎯 What it does: This paper provides a theoretical analysis of the 'grokking' phenomenon, particularly exploring the reasons why models can still achieve good generalization after overfitting in the modular addition problem.
Why Larger Language Models Do In-context Learning Differently?
Zhenmei Shi (University of Wisconsin Madison), Yingyu Liang (University of Hong Kong)
TransformerLarge Language ModelText
🎯 What it does: The paper explores the different performances of large language models in in-context learning (ICL) through theoretical analysis and experimental investigation, demonstrating that larger models are more susceptible to noise, resulting in poorer performance in ICL tasks compared to smaller models.
Winner-takes-all learners are geometry-aware conditional density estimators
Victor Letzelter (Valeo), Patrick Perez
Mixture of ExpertsTabularAudio
🎯 What it does: A conditional density estimation method based on Winner-takes-all (WTA) training is proposed—Voronoi-WTA, which constructs adaptive Voronoi partitions using the multi-head outputs predicted by WTA, and employs truncated kernel density estimation within each partition;