ICLR 2025 Papers — Page 36
International Conference on Learning Representations · 3704 papers
Understanding Factual Recall in Transformers via Associative Memories
Eshaan Nichani (Princeton University), Alberto Bietti (Flatiron Institute)
TransformerText
🎯 What it does: This paper studies the memory capacity of Transformers in factual memory tasks, and constructs a shallow Transformer that achieves near-optimal storage capacity in synthetic factual recall tasks by treating the attention matrix and MLP weights as associative memory, achieving 100% accuracy when the parameter scale is linearly proportional to the number of facts.
Understanding Long Videos with Multimodal Language Models
Kanchana Ranasinghe (Stony Brook University), Michael S Ryoo
Robotic IntelligenceTransformerLarge Language ModelPrompt EngineeringVision Language ModelVideoTextMultimodality
🎯 What it does: This paper proposes a multimodal video understanding framework MVU, which integrates three types of object-based visual information (global objects, spatial locations, motion trajectories) through natural language, and combines it with large language models to perform long video question answering and robotic domain question answering tasks. It also studies two baselines that use only LLM or only single-frame visual information, revealing the strong performance of LLM in the absence of video information.
Understanding Matrix Function Normalizations in Covariance Pooling through the Lens of Riemannian Geometry
Ziheng Chen (University of Trento), Nicu Sebe (University of Trento)
ClassificationImage
🎯 What it does: Explores and unifies the explanation of the roles of matrix logarithm and power functions in Global Covariance Pooling (GCP), clarifying their implicit construction of SPD polynomial Logistic regression (Riemannian classifier).
Understanding Optimization in Deep Learning with Central Flows
Jeremy Cohen, Jason D. Lee (Princeton University)
OptimizationConvolutional Neural NetworkTransformerFlow-based ModelImageSequentialOrdinary Differential Equation
🎯 What it does: A continuous dynamic model named Central Flow is proposed and validated to describe the time-averaged optimization trajectory of gradient descent and adaptive optimizers (such as Scalar RMSProp and RMSProp) in the 'Edge of Stability' state in deep learning. By using third-order Taylor expansion and time averaging, the corresponding ordinary differential equations are derived and experimentally validated across various network architectures.
Understanding the Generalization of In-Context Learning in Transformers: An Empirical Study
Xingxuan Zhang (Tsinghua University), Peng Cui (Tsinghua University)
TransformerLarge Language ModelText
🎯 What it does: This paper systematically evaluates the generalization ability of Transformers in In-Context Learning (ICL), constructing a task-based three-dimensional framework (inter-question, intra-question, intra-task), and explores its generalization performance across different dimensions through experiments such as function fitting, tool invocation, and translation.
Understanding the Stability-based Generalization of Personalized Federated Learning
Yingqi Liu (Sun Yat-sen University), Xiaochun Cao (Sun Yat-sen University)
Federated LearningConvolutional Neural NetworkImage
🎯 What it does: This paper conducts an algorithm-dependent generalization analysis and excess risk assessment of partially personalized federated learning (PFL) under non-convex conditions for the first time using uniform stability methods.
Understanding Virtual Nodes: Oversquashing and Node Heterogeneity
Joshua Southern (Imperial College London), Johannes F. Lutzeyer (École Polytechnique)
Graph Neural NetworkGraph
🎯 What it does: This study investigates the role of Virtual Nodes in Graph Neural Networks, quantifies their effect on alleviating oversquashing, and proposes an improved MPNN+VN G model that can learn the heterogeneous importance of nodes.
Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape View
Kaiyue Wen (Stanford University), Tengyu Ma (Stanford University)
OptimizationTransformerLarge Language ModelText
🎯 What it does: This study proposes the Warmup-Stable-Decay (WSD) and its improved version WSD-S learning rate schedules, explaining their optimization mechanisms under valley loss curves and achieving multiple budget checkpoints in large language model pre-training.
Unearthing Skill-level Insights for Understanding Trade-offs of Foundation Models
Mazda Moayeri (University of Maryland), Vibhav Vineet (Microsoft Research)
Large Language ModelPrompt EngineeringMultimodalityBenchmark
🎯 What it does: By automatically extracting skill labels from the reasoning steps generated by models, skill slices across benchmarks are constructed, enabling fine-grained capability analysis of large models.
Unhackable Temporal Reward for Scalable Video MLLMs
En Yu (Huazhong University of Science and Technology), Wenbing Tao (Huazhong University of Science and Technology)
RecognitionTransformerLarge Language ModelReinforcement LearningVision Language ModelVideoMultimodality
🎯 What it does: Proposed and validated the theory of 'Temporal Hacking', clarifying that video LLMs tend to focus on only a few frames during training, leading to performance degradation, and introduced a new metric called Temporal Perplexity (TPL) to measure this phenomenon; further designed the 'Unhackable Temporal Reward (UTR)' framework, which enhances video-language alignment capabilities by utilizing spatiotemporal attribute extraction and bidirectional querying; constructed and trained the Video-UTR model, achieving leading or competitive performance on multiple video and image understanding benchmarks.
Uni-Sign: Toward Unified Sign Language Understanding at Scale
Zecheng Li (University of Science and Technology of China), Houqiang Li (University of Science and Technology of China)
RecognitionGenerationGraph Neural NetworkLarge Language ModelSupervised Fine-TuningVideoText
🎯 What it does: A unified pre-training framework called Uni-Sign is proposed, which significantly reduces the gap between pre-training and downstream tasks by treating all SLU tasks as translation tasks through large-scale generative pre-training and unified fine-tuning.
Uni$^2$Det: Unified and Universal Framework for Prompt-Guided Multi-dataset 3D Detection
Yubin Wang (Tongji University), Cairong Zhao (Tongji University)
Object DetectionAutonomous DrivingPrompt EngineeringPoint Cloud
🎯 What it does: This paper proposes a unified and general multi-dataset 3D detection framework called Uni 2 Det, which utilizes a multi-stage prompting module to achieve feature unification and sharing across different datasets.
UniCBE: An Uniformity-driven Comparing Based Evaluation Framework with Unified Multi-Objective Optimization
Peiwen Yuan (Beijing Institute of Technology), Kan Li (Beijing Institute of Technology)
OptimizationLarge Language ModelSupervised Fine-TuningText
🎯 What it does: A unified comparative evaluation framework called UNICBE is proposed, which significantly reduces evaluation budget while maintaining accuracy, convergence speed, and scalability.
UniCO: On Unified Combinatorial Optimization via Problem Reduction to Matrix-Encoded General TSP
Wenzheng Pan (Shanghai Jiao Tong University), Junchi Yan (Shanghai Jiao Tong University)
OptimizationTransformerReinforcement LearningDiffusion model
🎯 What it does: Various combinatorial optimization problems are reduced to a general TSP matrix form in polynomial time, and a neural TSP solver capable of cross-task and cross-scale training is developed.
UniCon: Unidirectional Information Flow for Effective Control of Large-Scale Diffusion Models
Fanghua Yu (Shenzhen Institutes of Advanced Technology), Chao Dong (Shenzhen Institutes of Advanced Technology)
GenerationData SynthesisComputational EfficiencyTransformerDiffusion modelImage
🎯 What it does: A unidirectional information flow control adapter architecture named UniCon is proposed for conditional generation tasks of large-scale diffusion models, significantly reducing GPU memory usage and accelerating training speed.
UniCoTT: A Unified Framework for Structural Chain-of-Thought Distillation
Xianwei Zhuang (Peking University), Yuexian Zou (Peking University)
Knowledge DistillationTransformerLarge Language ModelContrastive LearningTextChain-of-Thought
🎯 What it does: By constructing and unifying diverse structured chains-of-thought (UniCoT) such as chain, tree, and graph structures, and using it as a bridge to transfer the reasoning capabilities of large models (LLM) to small models (SLM).
UniDetox: Universal Detoxification of Large Language Models via Dataset Distillation
Huimin LU, Ichiro Sakata (University of Tokyo)
Knowledge DistillationData-Centric LearningTransformerLarge Language ModelContrastive LearningText
🎯 What it does: We propose UNIDETOX, a knowledge distillation technique implemented through contrastive decoding, to generate detoxified text that can be fine-tuned on any large language model, achieving a unified detoxification effect.
UniDrive: Towards Universal Driving Perception Across Camera Configurations
Ye Li (University of Michigan), Kurt Keutzer (University of California)
Object DetectionAutonomous DrivingOptimizationVideo
🎯 What it does: The UniDrive framework is proposed, which achieves the generalization of vision-driven 3D perception models across different camera configurations by unifying virtual camera space, ground perception projection, and projection error optimization.
Unified Convergence Analysis for Score-Based Diffusion Models with Deterministic Samplers
Runjia Li (Peking University), Quanquan Gu (University of California)
GenerationData SynthesisOptimizationDiffusion modelScore-based ModelOrdinary Differential Equation
🎯 What it does: A unified convergence analysis framework is proposed, specifically for the theoretical study of distribution convergence of deterministic samplers, and two typical sampling methods—VP+EI and VE+DDIM—are analyzed for their iterative complexity.
Unified Parameter-Efficient Unlearning for LLMs
Chenlu Ding (Hong Kong Polytechnic University), Xiangnan He (University of Science and Technology of China)
Recommendation SystemOptimizationData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes a unified parameter-efficient model forgetting framework called LLMEraser, which enables rapid parameter adjustment for instance-level forgetting tasks of LLMs (instance deletion, query modification, answer correction);
Unify ML4TSP: Drawing Methodological Principles for TSP and Beyond from Streamlined Design Space of Learning and Search
Yang Li (Shanghai Jiao Tong University), Junchi Yan (Shanghai Jiao Tong University)
OptimizationMeta LearningGraph Neural NetworkReinforcement LearningDiffusion modelGraph
🎯 What it does: Designed and unified a modular ML4TSP framework, systematically breaking down and evaluating the roles of learning and search in TSP solving, and based on this, proposed improvement and restructuring methods.
Unifying Causal Representation Learning with the Invariance Principle
Dingling Yao (Institute of Science and Technology Austria), Francesco Locatello (Institute of Science and Technology Austria)
Representation LearningAuto EncoderTabular
🎯 What it does: A unified causal representation learning framework is proposed, utilizing the invariance principle to achieve the identifiability of latent variables.
Unifying Unsupervised Graph-Level Anomaly Detection and Out-of-Distribution Detection: A Benchmark
Yili Wang (Jilin University), Xin Wang (Jilin University)
Anomaly DetectionGraph Neural NetworkContrastive LearningGraphBiomedical DataBenchmark
🎯 What it does: A unified benchmark for unsupervised graph-level anomaly detection and graph-level out-of-distribution (OOD) detection (UB-GOLD) has been constructed, and a systematic evaluation of 18 mainstream methods has been conducted on 35 multi-scenario datasets.
UniGEM: A Unified Approach to Generation and Property Prediction for Molecules
Shikun Feng (Institute for AI Industry Research), Yanyan Lan (Institute for AI Industry Research)
GenerationDrug DiscoveryGraph Neural NetworkDiffusion modelGraph
🎯 What it does: A unified diffusion model, UniGEM, has been designed and implemented to simultaneously perform molecular generation and molecular property prediction.
UniGS: Unified Language-Image-3D Pretraining with Gaussian Splatting
Haoyuan Li (Shenzhen campus of Sun Yat-sen University), Xiaodan Liang (Guangdong Key Laboratory of Big Data Analysis and Processing)
RetrievalRepresentation LearningTransformerContrastive LearningGaussian SplattingMultimodalityPoint Cloud
🎯 What it does: The UniGS framework is proposed, which for the first time uses 3D Gaussian Splatting (3DGS) as a unified tri-modal (text-image-3D) pre-trained representation, addressing the sparsity of point cloud representation and the continuity gap with images.
uniINF: Best-of-Both-Worlds Algorithm for Parameter-Free Heavy-Tailed MABs
Yu Chen (Tsinghua University), Longbo Huang (Tsinghua University)
Optimization
🎯 What it does: A novel parameter-free algorithm, uniINF, is designed to address the heavy-tailed multi-armed bandit problem in both stochastic and adversarial environments, achieving the best-of-both-worlds performance.
UniMatch: Universal Matching from Atom to Task for Few-Shot Drug Discovery
Ruifeng Li (Zhejiang University), Hongyang Chen (Zhejiang Lab)
Meta LearningDrug DiscoveryGraph Neural NetworkGraphBiomedical Data
🎯 What it does: A dual matching framework called UniMatch is proposed for few-shot drug discovery tasks;
Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization
Noam Razin (Princeton University), Boris Hanin (Princeton University)
OptimizationSafty and PrivacyReinforcement Learning from Human FeedbackText
🎯 What it does: This paper studies the phenomenon of 'likelihood shift' in Direct Preference Optimization (DPO, etc.), elucidating the underlying geometric mechanisms of embeddings. It proposes and validates the Centralized Hidden Embedding Similarity (CHES) score, which is used to identify and filter training samples that lead to severe shifts, thereby alleviating unintentional mismatches in the model's safety alignment tasks.
Union-over-Intersections: Object Detection beyond Winner-Takes-All
Aritra Bhowmik (University of Amsterdam), Cees G. M. Snoek (University of Amsterdam)
Object DetectionImage
🎯 What it does: A strategy is proposed in object detection that first regresses the predicted boxes to the intersection area with the ground truth, and then uses the union of all intersection areas as the final detection box, accommodating two-stage, single-stage, and transformer-based detection frameworks with almost no modification to the original network;
UNIP: Rethinking Pre-trained Attention Patterns for Infrared Semantic Segmentation
Tao Zhang (Institute of Automation Chinese Academy of Sciences), Chunhong Pan (Institute of Automation Chinese Academy of Sciences)
SegmentationKnowledge DistillationTransformerContrastive LearningImage
🎯 What it does: This study investigates the transfer performance of pre-trained models in infrared image semantic segmentation, systematically comparing various pre-training methods, and proposes a unified pre-training framework called UNIP based on attention pattern analysis.
UniRestore3D: A Scalable Framework For General Shape Restoration
Yuang Wang (Zhejiang University), Xiaowei Zhou (Zhejiang University)
RestorationGenerationSuper ResolutionDiffusion modelAuto EncoderPoint CloudMesh
🎯 What it does: A unified 3D shape recovery framework called UniRestore3D is proposed, capable of handling various shape defects such as missing parts, noise, and low resolution all at once.
Universal generalization guarantees for Wasserstein distributionally robust models
Tam Le (University Grenoble Alpes), Jerome Malick (University Grenoble Alpes)
Optimization
🎯 What it does: This paper studies and proves the exact generalization guarantees of the Wasserstein distributionally robust optimization model, applicable to any continuous cost and loss functions.
Universal Image Restoration Pre-training via Degradation Classification
JiaKui Hu (Peking University), Yanye Lu (Peking University)
RestorationConvolutional Neural NetworkTransformerSupervised Fine-TuningImage
🎯 What it does: A Degradation Classification Pre-Training (DCPT) framework is proposed, allowing the restoration network to first learn to identify image degradation types through pre-training before downstream tasks, thereby enhancing global restoration capabilities.
Universal Sharpness Dynamics in Neural Network Training: Fixed Point Analysis, Edge of Stability, and Route to Chaos
Dayal Singh Kalra (University of Maryland), Maissam Barkeshli (University of Maryland)
OptimizationConvolutional Neural NetworkTransformerImageText
🎯 What it does: During the gradient descent training process, the evolution of the maximum eigenvalue of the loss Hessian (sharpness) was studied. A fixed point and linear stability analysis was conducted using a 2-layer linear UV model in a closed dynamical equation in function space, explaining the four training stages of sharpness (early decay, saturation, gradual sharpening, edge stability) and the periodic doubling to chaotic routing on the EoS trajectory. The theoretical predictions were then transferred to real networks (FCN, CNN, ResNet, Transformer) for validation.
UniWav: Towards Unified Pre-training for Speech Representation Learning and Generation
Alexander H. Liu (Massachusetts Institute of Technology), Bryan Catanzaro (NVIDIA)
GenerationKnowledge DistillationRepresentation LearningTransformerAudio
🎯 What it does: This paper proposes and implements UniWav, a unified encoder-decoder pre-training framework that can be used for speech recognition, text-to-speech synthesis, and low-bitrate speech segmentation/resynthesis.
Unlearn and Burn: Adversarial Machine Unlearning Requests Destroy Model Accuracy
Yangsibo Huang (Google), Chiyuan Zhang (Google)
Adversarial AttackConvolutional Neural NetworkImage
🎯 What it does: Research on the security of machine unlearning systems demonstrates that submitting adversarial unlearning requests leads to a sharp decline in model accuracy.
Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearning
Shengyuan Hu (Carnegie Mellon University), Virginia Smith (Carnegie Mellon University)
GenerationSafty and PrivacyAdversarial AttackTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: The paper proposes and validates a lightweight relearning attack for large language models that have 'forgotten' information. By fine-tuning on a dataset that has not been deleted or through fine-tuning with publicly available information, it is possible to restore the model's generative ability for the knowledge that has been forgotten.
Unlearning-based Neural Interpretations
Ching Lam Choi (Massachusetts Institute of Technology), Serge Belongie (University of Copenhagen)
Explainability and InterpretabilityConvolutional Neural NetworkTransformerImageText
🎯 What it does: This paper proposes a machine unlearning-based neural network explanation framework called UNI, which utilizes local first-order unlearning on the model and matches activations in the input space to generate adaptive, featureless baselines for specific task-model-input combinations, thereby achieving more trustworthy and robust gradient path attribution.
Unleashing the Potential of Vision-Language Pre-Training for 3D Zero-Shot Lesion Segmentation via Mask-Attribute Alignment
Yankai Jiang (Shanghai AI Laboratory), Shaoting Zhang (Shanghai AI Laboratory)
SegmentationTransformerVision Language ModelContrastive LearningImageMultimodalityBiomedical DataMagnetic Resonance Imaging
🎯 What it does: This paper presents Malenia, a multi-scale mask-attribute alignment framework for 3D zero-shot lesion segmentation, and designs a cross-modal knowledge injection (CMKI) module.
Unleashing the Power of Task-Specific Directions in Parameter Efficient Fine-tuning
Chongjie Si (Shanghai Jiao Tong University), Wei Shen (Shanghai Jiao Tong University)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes LoRA-Dash, a method for parameter-efficient fine-tuning that utilizes Task-Specific Directions (TSD) through a two-phase framework (pre-launch and sprint) to actively mine and amplify the impact of TSD;
Unlocking Efficient, Scalable, and Continual Knowledge Editing with Basis-Level Representation Fine-Tuning
Tianci Liu (Purdue University), Jing Gao (SUNY Albany)
TransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: A knowledge editing method based on basis-level representation fine-tuning, BaFT, is proposed, which utilizes input-related basis weights for nonlinear updates of representations.
Unlocking Global Optimality in Bilevel Optimization: A Pilot Study
Quan Xiao (Rensselaer Polytechnic Institute), Tianyi Chen (Rensselaer Polytechnic Institute)
OptimizationRepresentation LearningReinforcement LearningTabular
🎯 What it does: This paper proposes two types of landscapes beneficial for second-order optimization problems (joint PL and blockwise PL) and utilizes a penalty reconstruction method to prove that under these conditions, the penalty-based second-order gradient descent (PBGD) can achieve global optimal convergence in two typical machine learning applications: representation learning and data denoising.
Unlocking Guidance for Discrete State-Space Diffusion and Flow Models
Hunter Nisonoff (University of California, Berkeley), Jennifer Listgarten (University of California, Berkeley)
GenerationData SynthesisDrug DiscoveryDiffusion modelFlow-based ModelSequentialBiomedical Data
🎯 What it does: This paper proposes the Discrete Guidance method, which enables controllable conditional generation on discrete state space diffusion (CTDD) and flow (DFM) generative models. By performing Bayesian inference on the continuous-time Markov chain (CTMC) rate matrix, it achieves both predictor-guided and non-predictor-guided generation, and provides a Taylor approximation for accelerated implementation.
Unlocking Point Processes through Point Set Diffusion
David Lüdke (Munich Data Science Institute), Stephan Günnemann (Munich Data Science Institute)
GenerationData SynthesisTransformerDiffusion modelPoint CloudTime Series
🎯 What it does: A diffusion latent variable model named POINT SET DIFFUSION is proposed to directly model point processes in arbitrary metric spaces (including spatial, temporal, and spatiotemporal point processes), enabling unconditional generation and point set sampling under arbitrary conditions.
Unlocking State-Tracking in Linear RNNs Through Negative Eigenvalues
Riccardo Grazzi (Istituto Italiano di Tecnologia), Massimiliano Pontil
Recurrent Neural NetworkTextSequential
🎯 What it does: This paper studies the expressive power of Linear Recursive Neural Networks (LRNN) in state tracking tasks, proving that using only positive eigenvalues leads to the inability to solve problems such as parity and modular arithmetic. By extending the range of the eigenvalues of the state transition matrix to [-1, 1], the model's expressiveness is significantly enhanced, achieving perfect parity and significantly improved performance in modular arithmetic, word formation problems, and code/mathematical language modeling in multiple experiments.
Unlocking the Potential of Model Calibration in Federated Learning
Yun-Wei Chu (Purdue University), Christopher Brinton
Federated LearningImage
🎯 What it does: This paper proposes a universal framework for model calibration in federated learning, called NUCFL, and experiments with it in combination with various existing federated learning algorithms.
Unlocking the Power of Function Vectors for Characterizing and Mitigating Catastrophic Forgetting in Continual Instruction Tuning
Gangwei Jiang (University of Science and Technology of China), Ying Wei (Zhejiang University)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper analyzes catastrophic forgetting in continuous instruction fine-tuning of LLMs through functional vector analysis and proposes a functional vector-guided training method to alleviate this issue.
Unposed Sparse Views Room Layout Reconstruction in the Age of Pretrain Model
Yaxuan Huang (Hong Kong Center for Construction Robotics), Xiangyu Yue (Chinese University of Hong Kong)
SegmentationGenerationDomain AdaptationSupervised Fine-TuningImagePoint Cloud
🎯 What it does: This paper proposes a complete pipeline for indoor layout estimation based on Plane-DUSt3R, capable of directly generating the 3D layout of walls, ceilings, and floors from multiple perspective images without pose constraints.
Unsupervised Disentanglement of Content and Style via Variance-Invariance Constraints
Yuxuan Wu (Mohamed bin Zayed University of Artificial Intelligence), Gus Xia (Mohamed bin Zayed University of Artificial Intelligence)
GenerationRepresentation LearningAuto EncoderImageVideoMultimodalityAudio
🎯 What it does: A novel unsupervised content-style separation method V3 is proposed, which learns interpretable symbolic representations using variance and invariance constraints.
Unsupervised Meta-Learning via In-Context Learning
Anna Vettoruzzo (Halmstad University), Marlena Nowaczyk (Halmstad University)
Domain AdaptationMeta LearningTransformerContrastive LearningImage
🎯 What it does: This paper proposes CAMeLU, a Transformer framework that reformulates unsupervised meta-learning as context learning, capable of performing few-shot classification in a single forward inference.
Unsupervised Model Tree Heritage Recovery
Eliahu Horwitz (Hebrew University of Jerusalem), Yedid Hoshen (Hebrew University of Jerusalem)
Graph
🎯 What it does: An unsupervised method called MoTHer is proposed, which recovers the inheritance relationships of model trees/graphs using model weights.
Unsupervised Multiple Kernel Learning for Graphs via Ordinality Preservation
Yan Sun (National University of Singapore), Stanley Kok (National University of Singapore)
Graph Neural NetworkGraph
🎯 What it does: This paper proposes an unsupervised multiple kernel learning framework UMKL-G, which automatically combines multiple graph kernels while preserving the topological structure of the graphs, thereby enhancing graph-level clustering performance without the need for labels or predefined neighbors.
Unsupervised Zero-Shot Reinforcement Learning via Dual-Value Forward-Backward Representation
Jingbo Sun (Institute of Automation, Chinese Academy of Sciences), Dongbin Zhao (Institute of Automation, Chinese Academy of Sciences)
Reinforcement LearningContrastive LearningSequential
🎯 What it does: A dual-value forward-backward representation (DVFB) framework is proposed, utilizing parallel training of exploration value and skill value to achieve zero-shot generalization and fine-tuning adaptation in unsupervised online reinforcement learning.
UNSURE: self-supervised learning with Unknown Noise level and Stein's Unbiased Risk Estimate
Julián Tachella (Centre National de la Recherche Scientifique), Laurent Jacques (UCLouvain)
Image TranslationRestorationContrastive LearningImageMagnetic Resonance ImagingComputed Tomography
🎯 What it does: A self-supervised learning framework (UNSURE) is proposed, which can train image reconstruction networks under unknown noise levels without the need for ground truth labels.
Unveiling the Magic of Code Reasoning through Hypothesis Decomposition and Amendment
Yuze Zhao (University of Science and Technology of China), Enhong Chen (University of Science and Technology of China)
AI Code AssistantTransformerLarge Language ModelReinforcement LearningTextBenchmark
🎯 What it does: This paper proposes a code reasoning task that unifies three types of logical reasoning—inductive, deductive, and abductive—into a code format. Based on this, a reflective hypothesis decomposition and revision (RHDA) process is designed, utilizing LLMs and external execution tools (compilers, VirtualHome) to achieve multi-round reasoning, verification, and improvement, significantly enhancing the reasoning performance of LLMs.
Unveiling the Secret Recipe: A Guide For Supervised Fine-Tuning Small LLMs
Aldo Pareja (Red Hat AI Innovation), Akash Srivastava (Red Hat AI Innovation)
OptimizationHyperparameter SearchTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Conducted systematic experiments on supervised instruction fine-tuning of open-source LLMs in the 3B–7B scale, systematically evaluating the impact of hyperparameters such as batch size, learning rate, warmup, and training strategies (staged vs. stacked) on final performance.
URLOST: Unsupervised Representation Learning without Stationarity or Topology
Zeyu Yun (University of California Berkeley), Yubei Chen (University of California Davis)
Representation LearningTransformerAuto EncoderContrastive LearningImageBiomedical Data
🎯 What it does: Proposes an unsupervised representation learning framework URLOST that does not rely on data topology and temporal stationarity.
Utilitarian Algorithm Configuration for Infinite Parameter Spaces
Devon R. Graham (University of British Columbia), Kevin Leyton-Brown (University of British Columbia)
OptimizationHyperparameter SearchTabular
🎯 What it does: This paper proposes COUP (Continuous, Optimistic Utilitarian Procrastination) — a novel framework for configurability algorithms that can operate in an infinite parameter space. It demonstrates that COUP is faster with finite configuration sets and can effectively explore and provide approximate optimal guarantees in infinite sets.
Utility-Directed Conformal Prediction: A Decision-Aware Framework for Actionable Uncertainty Quantification
Santiago Cortes-Gomez (Carnegie Mellon University), Bryan Wilder (Carnegie Mellon University)
OptimizationImageBiomedical Data
🎯 What it does: This paper proposes a decision-oriented conformal prediction framework that integrates downstream cost functions into the prediction set generation while ensuring statistical coverage.
UTILITY: Utilizing Explainable Reinforcement Learning to Improve Reinforcement Learning
Shicheng Liu (Pennsylvania State University), Minghui Zhu (Pennsylvania State University)
OptimizationExplainability and InterpretabilityReinforcement LearningSequential
🎯 What it does: Identify the reasons for the non-optimality of the RL agent through two layers of interpretability analysis and improve RL using that explanation.
UV-Attack: Physical-World Adversarial Attacks on Person Detection via Dynamic-NeRF-based UV Mapping
Yanjie Li (Hong Kong Polytechnic University), Bin Xiao (Hong Kong Polytechnic University)
Object DetectionPose EstimationAdversarial AttackDiffusion modelNeural Radiance FieldVideo
🎯 What it does: Utilizing dynamic NeRF and UV mapping to achieve physical adversarial attacks on person detection;
VAE-Var: Variational Autoencoder-Enhanced Variational Methods for Data Assimilation in Meteorology
Yi Xiao (Tsinghua University), Wei Xue (Tsinghua University)
OptimizationAuto EncoderTime Series
🎯 What it does: A VAE-Var data assimilation algorithm based on variational autoencoders is proposed to capture the non-Gaussian distribution of atmospheric background errors and achieve observation fusion within a traditional variational framework.
Valid Conformal Prediction for Dynamic GNNs
Ed Davis (University of Bristol), Patrick Rubin-Delanchy (University of Edinburgh)
Graph Neural NetworkGraph
🎯 What it does: This paper uses tensor decomposition of dynamic graphs to map the node-time pairs of dynamic graphs to interchangeable embeddings, enabling graph neural networks (GNNs) to achieve effective confidence set prediction in both transductive and semi-inductive inference scenarios.
Value-aligned Behavior Cloning for Offline Reinforcement Learning via Bi-level Optimization
Xingyu Jiang (Beihang University), Yue Deng (Beihang University)
Reinforcement LearningTabularBenchmark
🎯 What it does: A dual-layer optimization-based offline reinforcement learning framework VACO is proposed, which performs weighted training of behavior cloning in the inner layer and aligns the value function in the outer layer, thereby improving policy performance without interacting with the environment.
Value-Incentivized Preference Optimization: A Unified Approach to Online and Offline RLHF
Shicong Cen (Carnegie Mellon University), Bo Dai (Google)
OptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: This paper proposes a unified Value Preference Optimization (VPO) framework, which achieves implicit optimistic/pessimistic policy optimization in online and offline RLHF by incorporating a value regularization term into the maximum likelihood estimation of the reward function.
Variance-Reducing Couplings for Random Features
Isaac Reid (University of Cambridge), Adrian Weller (University of Cambridge)
OptimizationGraph Neural NetworkTransformerGraphTabular
🎯 What it does: This paper models the variance reduction problem of random features (RFF, RLF, GRF) as a multi-marginal optimal transport (OT) problem, and designs coupling strategies in Euclidean space and graph space, significantly improving the approximation accuracy of random features and their performance in downstream tasks.
Variational Bayesian Pseudo-Coreset
Hyungi Lee (Korea Advanced Institute of Science and Technology), Juho Lee (Korea Advanced Institute of Science and Technology)
OptimizationComputational EfficiencyKnowledge DistillationConvolutional Neural NetworkImage
🎯 What it does: This paper proposes the Variational Bayesian Pseudo-Coreset (VBPC) method, which utilizes variational inference to construct a pseudo-core set on Bayesian neural networks with only the last layer randomized, achieving efficient inference for Bayesian model averaging (BMA) on large-scale datasets.
Variational Best-of-N Alignment
Afra Amini (ETH Zurich), Ryan Cotterell (ETH Zurich)
OptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
🎯 What it does: By deriving the distribution generated by Best-of-N (BoN) and using the inverse KL variational method (vBoN) to fine-tune the language model, it approximates the high-reward outputs of BoN during single sampling, thus achieving efficient aligned inference.
Variational Diffusion Posterior Sampling with Midpoint Guidance
Badr MOUFAD, Jimmy Olsson (KTH Royal Institute of Technology)
RestorationGenerationData SynthesisDiffusion modelImageBiomedical DataElectrocardiogram
🎯 What it does: A posterior sampling method MGPS based on Denoising Diffusion Models is proposed, which improves the estimation of the guiding term for observations using midpoint decomposition.
Variational Search Distributions
Daniel M. Steinberg (Data61 CSIRO), Edwin V. Bonilla (Data61 CSIRO)
OptimizationDrug DiscoveryRecurrent Neural NetworkTransformerReinforcement LearningSequentialBiomedical Data
🎯 What it does: A method called Variational Search Distributions (VSD) is proposed for sequentially learning and generating generative models of rare target classes (such as highly active protein/nucleic acid sequences) in high-dimensional discrete combinatorial design spaces, and for batch evaluation through black-box experiments/simulations.
Varying Shades of Wrong: Aligning LLMs with Wrong Answers Only
Jihan Yao (University of Washington), Yulia Tsvetkov (University of Washington)
TransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextBiomedical Data
🎯 What it does: In task scenarios lacking reliable correct answers, this study explores the use of LLMs to generate incorrect answers and improve the quality and calibration of model answers through 'wrong-over-wrong' preference alignment.
VCR: A Task for Pixel-Level Complex Reasoning in Vision Language Models via Restoring Occluded Text
Tianyu Zhang (Mila), Yoshua Bengio (Mila)
RestorationData SynthesisTransformerSupervised Fine-TuningVision Language ModelImageTextMultimodality
🎯 What it does: The task of Visual Text Recovery (VCR) is proposed, and the corresponding dataset VCR-WIKI is constructed, exploring pixel-level recovery of occluded text in images and complex reasoning of visual context.
VD3D: Taming Large Video Diffusion Transformers for 3D Camera Control
Sherwin Bahmani (University of Toronto), Sergey Tulyakov (Snap Inc.)
GenerationData SynthesisTransformerDiffusion modelVideo
🎯 What it does: A technology called VD3D is proposed to achieve three-dimensional camera control for large-scale video diffusion transformers, injecting spatiotemporal camera embeddings based on Plücker coordinates into the video generation process using a ControlNet-style conditional mechanism.
Vec2Face: Scaling Face Dataset Generation with Loosely Constrained Vectors
Haiyu Wu (University of Notre Dame), Kevin Bowyer
GenerationData SynthesisAuto EncoderGenerative Adversarial NetworkImage
🎯 What it does: This work proposes Vec2Face, which can efficiently synthesize datasets of synthetic faces for hundreds of thousands of identities by sampling and perturbing high-dimensional facial feature vectors.
Vector-ICL: In-context Learning with Continuous Vector Representations
Yufan Zhuang (University of California San Diego), Jianfeng Gao (Microsoft Research)
ClassificationGenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextMultimodalityTime SeriesMagnetic Resonance Imaging
🎯 What it does: The Vector-ICL method is proposed, which projects arbitrary continuous vectors into 'box tokens' that can be directly processed by LLMs, enabling cross-modal contextual learning.
VEDIT: Latent Prediction Architecture For Procedural Video Representation Learning
Han Lin (University of North Carolina Chapel Hill), Koustuv Sinha (Meta)
Representation LearningTransformerDiffusion modelFlow-based ModelVideo
🎯 What it does: Proposes the VEDIT framework, utilizing the latent embeddings of a frozen visual encoder combined with a diffusion transformer for step prediction and planning.
Verifying Properties of Binary Neural Networks Using Sparse Polynomial Optimization
Jianting Yang (Academy of Mathematics and Systems Science Chinese Academy of Sciences), Jun Zhao (Nanyang Technological University)
OptimizationAdversarial AttackImage
🎯 What it does: This paper proposes a semi-definite programming (SDP) relaxation method based on sparse polynomial optimization to verify the robustness of binary neural networks (BNN) in continuous input spaces, overcoming the shortcomings of traditional SAT/SMT and MILP methods in terms of numerical errors and scalability.
Vertical Federated Learning with Missing Features During Training and Inference
Pedro Valdeira (Carnegie Mellon University), Yuejie Chi (Carnegie Mellon University)
Federated LearningTabularBiomedical Data
🎯 What it does: This paper proposes LASER-VFL, a vertical federated learning method capable of handling arbitrary missing feature blocks during both training and inference, avoiding data waste and achieving scalability.
Vevo: Controllable Zero-Shot Voice Imitation with Self-Supervised Disentanglement
Xueyao Zhang (Chinese University of Hong Kong), Mingbo Ma (Meta AI)
GenerationData SynthesisTransformerAuto EncoderAudio
🎯 What it does: This paper presents the Vevo framework, which achieves controllable voice imitation in zero-shot scenarios, allowing for separate control of timbre, pitch, and content.
VibeCheck: Discover and Quantify Qualitative Differences in Large Language Models
Lisa Dunlap (University of California Berkeley), Joseph E. Gonzalez
Recommendation SystemExplainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: VibeCheck has been developed, a method for automatically discovering and quantifying interpretable and measurable 'vibes' in the outputs of large language models, helping to compare subtle differences between models.
ViBiDSampler: Enhancing Video Interpolation Using Bidirectional Diffusion Sampler
Serin Yang (KAIST), Jong Chul Ye (KAIST)
GenerationData SynthesisDiffusion modelVideo
🎯 What it does: The ViBiDSampler keyframe interpolation method is proposed, utilizing bidirectional sampling and advanced manifold guidance to generate high-quality, temporally coherent video frames.
VICtoR: Learning Hierarchical Vision-Instruction Correlation Rewards for Long-horizon Manipulation
Kuo-Han Hung (National Taiwan University), Winston H. Hsu (National Taiwan University)
Robotic IntelligenceReinforcement Learning from Human FeedbackLarge Language ModelReinforcement LearningVision Language ModelContrastive LearningVideoMultimodality
🎯 What it does: Learning reward models for long-term manipulation tasks, using visual-instruction correlation for hierarchical rewards.
Video Action Differencing
James Burgess (Stanford), Serena Yeung-Levy (Stanford)
RecognitionGenerationTransformerLarge Language ModelAgentic AIVision Language ModelVideoBenchmark
🎯 What it does: This paper proposes the video action differentiation (VidDiff) task and its benchmark dataset VidDiffBench, and designs a three-stage zero-shot proxy workflow called the VidDiff method, which can provide fine-grained differentiation descriptions of two video segments of the same action without requiring additional training.
Video In-context Learning: Autoregressive Transformers are Zero-Shot Video Imitators
Wentao Zhang (University of Science and Technology of China), Jiang Bian (Microsoft Research Asia)
GenerationData SynthesisTransformerGenerative Adversarial NetworkVideoMultimodality
🎯 What it does: A video imitation model called VidIT based on autoregressive Transformer is proposed, which can generate new videos consistent with the demonstration semantics in a zero-shot scenario by watching demonstration videos.
Video-STaR: Self-Training Enables Video Instruction Tuning with Any Supervision
Orr Zohar (Stanford University), Serena Yeung-Levy (Stanford University)
GenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningVideo
🎯 What it does: This paper proposes Video-STAR, a self-training framework that utilizes arbitrarily labeled video data for video instruction fine-tuning.
VideoGrain: Modulating Space-Time Attention for Multi-Grained Video Editing
Xiangpeng Yang (University of Technology Sydney), Yi Yang (Zhejiang University)
SegmentationGenerationTransformerDiffusion modelVideoText
🎯 What it does: This paper presents VideoGrain, a zero-parameter multi-granularity video editing framework that enables precise editing at the class, instance, and part levels.
VideoPhy: Evaluating Physical Commonsense for Video Generation
Hritik Bansal (University of California Los Angeles), Aditya Grover (University of California Los Angeles)
GenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringVideoTextMultimodalityBenchmarkPhysics Related
🎯 What it does: The VIDEOPHY benchmark was constructed to evaluate the semantic adherence and physical common sense of text-to-video models using human annotations, and an automatic evaluator, VIDEOCON-PHYSICS, was proposed.
VideoShield: Regulating Diffusion-based Video Generation Models via Watermarking
Runyi Hu (Nanyang Technological University), Tianwei Zhang (Nanyang Technological University)
GenerationData SynthesisSafty and PrivacyDiffusion modelVideo
🎯 What it does: Achieve untrained watermark embedding in the diffusion-based video generation process, and implement a complete framework VIDEOSHIELD for watermark extraction and spatiotemporal tampering localization through DDIM inversion.
VideoWebArena: Evaluating Long Context Multimodal Agents with Video Understanding Web Tasks
Lawrence Keunho Jang (Carnegie Mellon University), Kazuhito Koishida (Microsoft)
Large Language ModelAgentic AIVideoMultimodalityBenchmark
🎯 What it does: Proposes the VideoWebArena benchmark to evaluate multimodal long-context video understanding and agent execution capabilities;
ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation
Tianchen Zhao (Tsinghua University), Yu Wang (Tsinghua University)
GenerationData SynthesisCompressionComputational EfficiencyTransformerDiffusion modelImageVideo
🎯 What it does: A post-training quantization method for diffusion transformers (DiT) called ViDiT-Q is proposed, which achieves low bit-width (e.g., W4A8) in text-to-image/video generation tasks with minimal impact on generation quality.
VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generation
Yecheng Wu (Tsinghua University), Yao Lu (NVIDIA)
RecognitionGenerationTransformerLarge Language ModelVision Language ModelContrastive LearningImageVideoTextMultimodality
🎯 What it does: We propose VILA-U, a unified visual language model capable of predicting the next token in both visual and textual modalities under the same autoregressive framework, achieving both visual understanding and visual generation.
ViSAGe: Video-to-Spatial Audio Generation
Jaeyeon Kim (Seoul National University), Gunhee Kim (Seoul National University)
GenerationData SynthesisTransformerVideoAudio
🎯 What it does: A framework called ViSAGe is proposed to directly generate first-order Ambisonics spatial audio from silent videos, and a corresponding dataset YT-Ambigen is created.
Vision and Language Synergy for Rehearsal Free Continual Learning
Muhammad Anwar Ma'sum (University of South Australia), Ryszard Kowalczyk (University of South Australia)
GenerationTransformerPrompt EngineeringImage
🎯 What it does: A replay-free continual learning framework LEAPGen is proposed, which utilizes language descriptions to generate prompts for adapting to new tasks.
Vision CNNs trained to estimate spatial latents learned similar ventral-stream-aligned representations
Yudi Xie (Massachusetts Institute of Technology), James J. DiCarlo (Massachusetts Institute of Technology)
Pose EstimationRepresentation LearningConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: Train convolutional networks on synthetic images to estimate the spatial latent variables of objects (such as position, pose, distance, etc.) and evaluate their alignment with neural data from the monkey visual pathway.
Vision Language Models are In-Context Value Learners
Yecheng Jason Ma (University of Pennsylvania), Fei Xia (Google DeepMind)
Robotic IntelligenceTransformerVision Language ModelVideoMultimodality
🎯 What it does: Proposes the Generative Value Learning (GVL) framework, which utilizes large visual language models (VLM) to autoregressively generate task progress values on shuffled frame sequences, thereby achieving general value function prediction across tasks and robots with zero/few examples;
Vision-LSTM: xLSTM as Generic Vision Backbone
Benedikt Alkin (EMMI AI), Johannes Brandstetter (EMMI AI)
ClassificationSegmentationDomain AdaptationComputational EfficiencyRecurrent Neural NetworkVision Language ModelImage
🎯 What it does: Proposes Vision-LSTM (ViL), a linear complexity visual backbone network based on xLSTM.
Vision-RWKV: Efficient and Scalable Visual Perception with RWKV-Like Architectures
Yuchen Duan (Chinese University of Hong Kong), Wenhai Wang (Chinese University of Hong Kong)
ClassificationObject DetectionSegmentationComputational EfficiencyTransformerVision Language ModelImage
🎯 What it does: This paper proposes Vision-RWKV (VRWKV), a visual encoder that employs a linear attention mechanism, capable of maintaining global perception while reducing computational complexity, supporting high-resolution image processing and sparse input.
VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documents
Shi Yu (Tsinghua University), Maosong Sun (Tsinghua University)
GenerationRetrievalTransformerVision Language ModelMultimodalityRetrieval-Augmented Generation
🎯 What it does: A visual retrieval-augmented generation system, VisRAG, has been established, utilizing VLM to directly retrieve and generate multimodal documents, eliminating the need for OCR parsing.
Visual Agents as Fast and Slow Thinkers
Guangyan Sun (University of Rochester), Dongfang Liu
SegmentationExplainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodality
🎯 What it does: A visual agent framework named FAST is proposed, which achieves quick answers to simple problems and hierarchical reasoning for complex problems through system switching (Fast/Slow thinking), and enhances interpretability through chain evidence.
Visual Description Grounding Reduces Hallucinations and Boosts Reasoning in LVLMs
Sreyan Ghosh (University of Maryland), Dinesh Manocha (Adobe)
GenerationRetrievalTransformerLarge Language ModelVision Language ModelImageTextMultimodality
🎯 What it does: A training-agnostic Visual Description Guided Decoding (VDGD) method is proposed, which utilizes image descriptions to guide large visual language models (LVLM) in reducing hallucinations and enhancing reasoning capabilities in inference tasks.