NeurIPS 2025 Papers — Page 2
Conference on Neural Information Processing Systems · 5275 papers
A Practical Guide for Incorporating Symmetry in Diffusion Policy
Dian Wang (Stanford University), Robert Platt (Northeastern University)
Robotic IntelligenceReinforcement Learning from Human FeedbackConvolutional Neural NetworkReinforcement LearningDiffusion modelImage
🎯 What it does: This paper proposes a practical method that embeds spatial symmetry (SE(3)) into diffusion strategies, including the use of relative/incremental trajectory actions, eye-in-hand camera observations, equivariant visual encoders, and the use of frame averaging with pre-trained encoders.
A Pre-training Framework for Relational Data with Information-theoretic Principles
Quang Truong (Michigan State University), Jiliang Tang (Michigan State University)
Graph Neural NetworkTabularBenchmark
🎯 What it does: A self-supervised pre-training framework for relational databases called Task Vector Estimation (TVE) is proposed, which generates task vectors by combining schema graphs, temporal dynamics, and task heterogeneity to train the model.
A Principle of Targeted Intervention for Multi-Agent Reinforcement Learning
Anjie Liu (Hong Kong University of Science and Technology), Mengyue Yang (University of Bristol)
Recurrent Neural NetworkReinforcement LearningAgentic AISequential
🎯 What it does: A target intervention paradigm based on Multi-Agent Influence Diagrams (MAID) is proposed, and a Pre-Strategy Intervention (PSI) method is implemented, using interventions from a single agent to guide the entire multi-agent system to achieve predetermined composite goals.
A Principled Approach to Randomized Selection under Uncertainty: Applications to Peer Review and Grant Funding
Alexander Koujianos Goldberg (Carnegie Mellon University), Nihar B Shah
OptimizationTabularReview/Survey Paper
🎯 What it does: The study is based on the interval estimation of the randomized selection mechanism MERIT, which maximizes the worst-case expected selection of Top-k under uncertainty.
A Principled Path to Fitted Distributional Evaluation
Sungee Hong (Texas A&M University), Raymond K. W. Wong (Texas A&M University)
Reinforcement LearningTabular
🎯 What it does: This paper proposes a unified framework that extends Fitted Q Evaluation (FQE) to Distributed Offline Policy Evaluation (FDE), and provides various feasible divergence metrics and convergence analysis.
A Private Approximation of the 2nd-Moment Matrix of Any Subsamplable Input
Bar Mahpud (Bar Ilan University), Or Sheffet (Bar Ilan University)
Anomaly DetectionOptimizationSafty and Privacy
🎯 What it does: This paper studies the second-order moment estimation problem under differential privacy and proposes a new algorithm that can achieve a strong privacy-utility trade-off even in the worst-case scenario under the assumption of subsampling of the data.
A Provable Approach for End-to-End Safe Reinforcement Learning
Akifumi Wachi (LY Corporation), Youhei Akimoto (University of Tsukuba)
Safty and PrivacyRobotic IntelligenceTransformerReinforcement LearningTabular
🎯 What it does: A method called Provably Lifetime Safe RL (PLS) is proposed, which first trains a reward-conditioned policy through offline safe reinforcement learning (reward-conditioned supervised learning), and then safely optimizes the target reward in the real environment using Gaussian Process, thus ensuring safety throughout the entire process from learning to deployment.
A Regularized Newton Method for Nonconvex Optimization with Global and Local Complexity Guarantees
Yuhao Zhou (Tsinghua University), Jun Zhu (Chinese Academy of Sciences)
OptimizationBenchmarkPhysics Related
🎯 What it does: This paper proposes a parameter-free, adaptive regularized Newton method (ARNCG) that constructs a regularization term by combining the current and previous gradients, and solves the regularized Newton equations using CappedCG with negative curvature monitoring; this method achieves global optimal complexity and local second-order convergence in non-convex optimization.
A Reinforcement Learning-based Bidding Strategy for Data Consumers in Auction-based Federated Learning
Xiaoli Tang (Nanyang Technological University), Xiaoxiao Li (University of British Columbia)
Federated LearningReinforcement LearningImage
🎯 What it does: A dynamic bidding strategy based on reinforcement learning, RLB-AFL, is proposed for data consumers in federated learning to select and bid for data owners in an auction market.
A Reliable Cryptographic Framework for Empirical Machine Unlearning Evaluation
Yiwen Tu (University of Michigan), Jiaqi W. Ma (University of Illinois Urbana-Champaign)
Safty and PrivacyConvolutional Neural NetworkGenerative Adversarial NetworkImage
🎯 What it does: A machine unlearning evaluation framework based on cryptographic games is proposed, along with corresponding advantage metrics and an efficient approximate SWAP test.
A Scalable, Causal, and Energy Efficient Framework for Neural Decoding with Spiking Neural Networks
Georgios Mentzelopoulos (University of Pennsylvania), Flavia Vitale (University of Pennsylvania)
Spiking Neural NetworkTransformerTime Series
🎯 What it does: A scalable, causal, low-energy neural decoding framework based on Spiking Neural Networks (SNN) called Spikachu is proposed, supporting online BCI decoding for multiple sessions and multiple subjects.
A Semantic Parsing Framework for End-to-End Time Normalization
Xin Su, Steven Bethard (University of Arizona)
GenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: In this paper, the authors propose an end-to-end temporal normalization method that treats time expression recognition and normalization as a code generation task. They use the executable SCATE Python library to directly convert natural language time vocabulary into executable time expression code, and obtain precise time intervals through code execution.
A Set of Generalized Components to Achieve Effective Poison-only Clean-label Backdoor Attacks with Collaborative Sample Selection and Triggers
Zhixiao Wu (Harbin Institute of Technology), Guangming Lu (Harbin Institute of Technology)
Adversarial AttackImage
🎯 What it does: This paper proposes a set of general components that collaborate sample selection and triggers to enhance the success rate and stealth of clean-label poisoning attacks.
A Signed Graph Approach to Understanding and Mitigating Oversmoothing
Jiaqi WANG, Yifei Wang (Massachusetts Institute of Technology)
ClassificationGraph Neural NetworkGraphBenchmark
🎯 What it does: This study investigates the oversmoothing problem in deep Graph Neural Networks (GNNs) and proposes a method called Structure Balanced Symbolic Graph Propagation (SBP) to alleviate oversmoothing while maintaining the distinguishability of node representations.
A Single-Loop First-Order Algorithm for Linearly Constrained Bilevel Optimization
Wei Shen (University of Virginia), Cong Shen (University of Virginia)
OptimizationHyperparameter SearchTabular
🎯 What it does: This paper studies a bilevel optimization problem with linear coupling constraints and proposes a single-loop first-order algorithm SFLCB.
A Single-Loop Gradient Algorithm for Pessimistic Bilevel Optimization via Smooth Approximation
Qichao Cao (Southern University of Science and Technology), Jin Zhang (Southern University of Science and Technology)
OptimizationImageTabular
🎯 What it does: A smooth approximation for pessimistic bilevel optimization is proposed, and based on this, a single-loop full first-order gradient algorithm SiPBA is designed.
A Smooth Sea Never Made a Skilled SAILOR: Robust Imitation via Learning to Search
Arnav Kumar Jain (Mila - Quebec AI Institute), Gokul Swamy (Carnegie Mellon University)
Robotic IntelligenceReinforcement Learning from Human FeedbackRecurrent Neural NetworkReinforcement LearningWorld ModelImage
🎯 What it does: A robust imitation learning method named SAILOR was trained, utilizing a world model, reward model, and MPPI planning learned from expert demonstrations, searching for residual plans during testing to correct errors in the baseline policy.
A Snapshot of Influence: A Local Data Attribution Framework for Online Reinforcement Learning
Yuzheng Hu (University of Illinois Urbana Champaign), Han Zhao (University of Illinois Urbana Champaign)
OptimizationComputational EfficiencyLarge Language ModelReinforcement LearningSequential
🎯 What it does: This study investigates data attribution in online reinforcement learning, proposing a local attribution framework for PPO and designing an iterative influence filtering algorithm based on it to enhance training efficiency and performance.
A solvable model of learning generative diffusion: theory and insights
Hugo Cui (Harvard University), Yue M. Lu
GenerationData SynthesisDiffusion modelAuto EncoderImageMultimodalityStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: An analytical analysis of diffusion-based generative models (using two-layer DAE and trained via online SGD) is conducted, deriving the low-dimensional projections and evolution laws of the generative distribution in high-dimensional limits;
A Stable Whitening Optimizer for Efficient Neural Network Training
Kevin Frans (University of California Berkeley), Pieter Abbeel (University of California Berkeley)
OptimizationTransformerDiffusion modelImageText
🎯 What it does: This paper proposes and implements SPlus, a stable whitening optimizer based on Shampoo, designed to accelerate the training of Transformer networks.
A Statistical Theory of Contrastive Learning via Approximate Sufficient Statistics
Licong Lin (University of California Berkeley), Song Mei (University of California Berkeley)
Representation LearningContrastive LearningTabular
🎯 What it does: A new theoretical framework is proposed, using approximate sufficient statistics to analyze data augmentation-based contrastive learning (represented by SimCLR), and it is proven that the contrastive learning loss is equivalent to finding an approximate sufficient encoder;
A Tale of Two Symmetries: Exploring the Loss Landscape of Equivariant Models
YuQing Xie (Massachusetts Institute of Technology), Tess Smidt (Massachusetts Institute of Technology)
OptimizationKnowledge DistillationTabular
🎯 What it does: Theoretical and experimental research on the loss landscape of equivariant neural networks reveals local optima caused by hidden symmetries;
A Temporal Difference Method for Stochastic Continuous Dynamics
Haruki Settai (University of Tokyo), Takehisa Yairi (University of Tokyo)
Reinforcement LearningSequentialStochastic Differential Equation
🎯 What it does: A model-free differential temporal difference (dTD) method based on the Hamilton-Jacobi-Bellman equation is proposed for continuous-time reinforcement learning.
A Theoretical Framework for Grokking: Interpolation followed by Riemannian Norm Minimisation
Etienne Boursier (Inria), Radu-Alexandru Dragomir (Telecom Paris Institute)
OptimizationTabular
🎯 What it does: This paper proposes a theoretical framework based on gradient flow and weight decay to explain the grokking phenomenon in deep learning: in the early stages of training, the gradient flow is almost the same as that of an unregularized gradient flow, quickly converging to a critical point manifold of the loss function; subsequently, in a slow phase with a time scale of about 1/λ, weight decay causes parameters to perform Riemannian gradient descent along this manifold, gradually reducing the ‖·‖₂ norm of the parameters, thereby achieving better generalization.
A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning
Zhi Zhou (Nanjing University), Xiaoxing Ma (Nanjing University)
TransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Provides a theoretical framework for sampling-based testing methods for LLM inference and proposes a new RPC method.
A Theory for Worst-Case vs. Average-Case Guarantees for LLMs
Noga Amit (University of California Berkeley), Guy N. Rothblum (Apple)
Large Language ModelReinforcement LearningText
🎯 What it does: Proposes and trains a self-proof model that can self-verify the correctness of outputs in interactive proof systems.
A Token is Worth over 1,000 Tokens: Efficient Knowledge Distillation through Low-Rank Clone
Jitai Hao (Harbin Institute of Technology), Jun Yu (Harbin Institute of Technology)
Computational EfficiencyKnowledge DistillationTransformerLarge Language ModelText
🎯 What it does: An efficient knowledge distillation method called Low-Rank Cloning (LRC) is proposed for constructing Small Language Models (SLMs) to achieve behavior equivalence with powerful teacher models.
A TRIANGLE Enables Multimodal Alignment Beyond Cosine Similarity
Giordano Cicchetti (Sapienza University of Rome), Danilo Comminiello (Sapienza University of Rome)
RetrievalContrastive LearningVideoTextMultimodalityAudio
🎯 What it does: This paper proposes the TRIANGLE method, which achieves tri-modal alignment by calculating the area of triangles formed by tri-modal embeddings in high-dimensional space, thereby directly completing multi-modal matching in the embedding space.
A Unified Analysis of Stochastic Gradient Descent with Arbitrary Data Permutations and Beyond
Yipeng Li (Shenzhen International Graduate School Tsinghua University), Zhenyu Liu (Shenzhen International Graduate School Tsinghua University)
OptimizationFederated LearningImageTabular
🎯 What it does: This paper proposes a unified convergence analysis framework that encompasses all permutation-based stochastic gradient descent (SGD) algorithms (including arbitrary permutations, independent permutations, and dependent permutations such as GraBs) as well as client permutations in federated learning (FL) with regularized client participation;
A Unified Approach to Submodular Maximization Under Noise
Kshipra Bhawalkar (Google), Tao Lin (Harvard University)
Optimization
🎯 What it does: A general framework is proposed to transform any 'robust' exact submodular maximization algorithm into an algorithm that operates in a noisy environment while maintaining the original approximation ratio under noise.
A unified framework for establishing the universal approximation of transformer-type architectures
Jingpu Cheng (National University of Singapore), Qianxiao Li (National University of Singapore)
Transformer
🎯 What it does: The universal approximation properties of Transformer-like architectures are studied, and a unified theoretical framework is provided.
A Unified Framework for Fair Graph Generation: Theoretical Guarantees and Empirical Advances
Zichong Wang (Florida International University), Wenbin Zhang (Florida International University)
GenerationData SynthesisGraph Neural NetworkDiffusion modelScore-based ModelAuto EncoderGraph
🎯 What it does: A one-shot graph generation framework named FairGEM is proposed, aimed at simultaneously alleviating graph structure bias and node feature bias to generate fairer synthetic graphs.
A Unified Framework for Provably Efficient Algorithms to Estimate Shapley Values
Tyler Chen (Global Technology Applied Research JPMorganChase), Niraj Kumar (Global Technology Applied Research JPMorganChase)
Explainability and InterpretabilityComputational EfficiencyImageTabular
🎯 What it does: This work proposes a unified framework for the theoretical analysis of random estimators of Shapley values, providing non-asymptotic sample complexity guarantees, while implementing estimation methods that can be scaled to high-dimensional data.
A Unified Framework for the Transportability of Population-Level Causal Measures
Ahmed BOUGHDIRI, Erwan Scornet (Sorbonne Université and Université Paris Cité)
TabularBiomedical Data
🎯 What it does: A unified framework is proposed to translate various first-order causal effect measures (such as risk difference, risk ratio, odds ratio, etc.) from randomized controlled trials to target populations under covariate distribution shift.
A Unified Framework for Variable Selection in Model-Based Clustering with Missing Not at Random
Binh Huu Ho, Christopher Drovandi (Queensland University of Technology)
TabularBiomedical Data
🎯 What it does: A unified model-based clustering framework is proposed, which achieves variable selection and handles missing not at random (MNAR) data;
A Unified Reasoning Framework for Holistic Zero-Shot Video Anomaly Analysis
Dongheng Lin (Beijing Jiaotong University), Yunchao Wei (Beijing Jiaotong University)
Anomaly DetectionTransformerLarge Language ModelPrompt EngineeringVision Language ModelVideoMultimodality
🎯 What it does: A unified, zero-shot reasoning framework is proposed, capable of completing three tasks in a single reasoning process: video anomaly detection (VAD), anomaly localization (VAL), and anomaly understanding (VAU);
A Unified Solution to Video Fusion: From Multi-Frame Learning to Benchmarking
Zixiang Zhao (ETH Zurich), Konrad Schindler (ETH Zurich)
RestorationSegmentationData SynthesisTransformerOptical FlowVideoBiomedical DataMagnetic Resonance ImagingComputed TomographyPositron Emission TomographyBenchmark
🎯 What it does: A unified video fusion framework called UniVF is proposed, along with the construction of a video fusion benchmark VF-Bench that covers four types of tasks (multi-exposure, multi-focus, infrared-visible, and medical), providing high-quality, well-aligned video pairs and a unified evaluation process.
A Unified Stability Analysis of SAM vs SGD: Role of Data Coherence and Emergence of Simplicity Bias
WEI-KAI CHANG, Rajiv Khanna (Purdue University)
OptimizationTabular
🎯 What it does: This paper analyzes the behavior of two optimizers—stochastic gradient descent (SGD) with random perturbations and Sharpness-Aware Minimization (SAM)—within a linear stability framework in two-layer ReLU networks, revealing how data coherence determines which minima are more easily converged during training, thus explaining the implicit simplicity bias of SGD and SAM.
A Unifying View of Linear Function Approximation in Off-Policy RL Through Matrix Splitting and Preconditioning
Zechen Wu (Duke University), Ronald Parr (Duke University)
Reinforcement Learning
🎯 What it does: This paper proposes a unified framework for matrix splitting and preconditioning, reformulating the convergence issues of TD, FQI, and PFQI under linear function approximation, viewing the three as different preconditioned iterations of the same linear system.
A-Mem: Agentic Memory for LLM Agents
Wujiang Xu (Rutgers University), Yongfeng Zhang (AIOS Foundation)
TransformerLarge Language ModelAgentic AIText
🎯 What it does: A proxy memory system named A-MEM is proposed, allowing large language model agents to dynamically create, link, and evolve memory nodes without predefined storage and retrieval patterns, thereby supporting long-term interactions.
A*-Thought: Efficient Reasoning via Bidirectional Compression for Low-Resource Settings
Xiaoang Xu (Beijing University of Posts and Telecommunications), Zhaofeng He (Beijing University of Posts and Telecommunications)
CompressionOptimizationComputational EfficiencyTransformerTextChain-of-Thought
🎯 What it does: This paper proposes a chain-of-thought compression method based on A* search, called A*-Thought, which transforms lengthy reasoning into concise and efficient inference by first evaluating the bidirectional importance of each step and then searching for thinking paths with high information density and low cost on the search tree.
A$^3$E: Towards Compositional Model Editing
Hongming Piao (City University of Hong Kong), Ying Wei (Zhejiang University)
Large Language ModelTextBiomedical Data
🎯 What it does: The A³E model editing framework is proposed to address the issues of knowledge loss, pre-existing errors, and knowledge drowning in large language models under multiple editing scenarios.
AANet: Virtual Screening under Structural Uncertainty via Alignment and Aggregation
Wenyu Zhu (Tsinghua University), Yanyan Lan (Tsinghua University)
Drug DiscoveryContrastive LearningMultimodalityBiomedical Data
🎯 What it does: This paper proposes AANet, which addresses the issue of structure-based virtual screening under structural uncertainty (especially in the absence of experimental or predicted active binding pockets) by constructing an alignment-aggregation framework, achieving efficient virtual screening on apo and AlphaFold2 predicted structures.
Absolute Zero: Reinforced Self-play Reasoning with Zero Data
Andrew Zhao (Tsinghua University), Gao Huang (Tsinghua University)
Large Language ModelReinforcement Learning
🎯 What it does: The paper proposes the 'Absolute Zero' paradigm and constructs a self-evolving reasoning model called the Absolute Zero Reasoner (AZR), which can self-generate tasks and solve them through a code executor.
Absorb and Converge: Provable Convergence Guarantee for Absorbing Discrete Diffusion Models
Yuchen Liang (Ohio State University), Yingbin Liang (Ohio State University)
GenerationData SynthesisDiffusion modelImageText
🎯 What it does: This paper presents for the first time the finite-time error bounds and convergence rate analysis of the absorption rate matrix in discrete diffusion models, proving that the sampler under this rate matrix can converge at a linear order.
Abstain Mask Retain Core: Time Series Prediction by Adaptive Masking Loss with Representation Consistency
Renzhao Liang (Beihang University), Takahiro Yabe (New York University)
TransformerTime Series
🎯 What it does: The AMRC method is proposed, which suppresses redundant features in time series prediction through adaptive masking loss and representation consistency constraints, thereby improving prediction accuracy.
Abstract Counterfactuals for Language Model Agents
Edoardo Pona (King's College London), Nicola Paoletti (King's College London)
TransformerLarge Language ModelAgentic AIText
🎯 What it does: An abstract counterfactual reasoning framework (Abstract Counterfactuals, ACF) aimed at language model agents is proposed, addressing the shortcomings of traditional token-based counterfactual methods in open action spaces by reasoning at a high-level semantic abstraction of actions.
Abstract Rendering: Certified Rendering Under 3D Semantic Uncertainty
Chenxi Ji (University of Illinois Urbana-Champaign), Sayan Mitra (University of Illinois Urbana-Champaign)
ClassificationObject DetectionPose EstimationNeural Radiance FieldGaussian SplattingImage
🎯 What it does: An abstract rendering framework is proposed that can generate verifiable image sets for 3D scenes (Gaussian Splats and NeRF) under continuous changes in camera pose or scene parameters, and provides formal safety guarantees for downstream visual models (classification, detection, pose estimation) through this abstract image.
AC-DiT: Adaptive Coordination Diffusion Transformer for Mobile Manipulation
Sixiang Chen (Peking University), Shanghang Zhang (Peking University)
Robotic IntelligenceTransformerDiffusion modelImagePoint Cloud
🎯 What it does: An end-to-end Diffusion Transformer model AC-DiT suitable for mobile robots (chassis + robotic arm) is proposed for language-conditioned mobile manipulation.
AC-LoRA: (Almost) Training-Free Access Control Aware Multi-Modal LLMs
Lara Magdalena Lazier (Huawei Technologies Switzerland AG), Lukas Cavigelli (EPFL)
RetrievalSafty and PrivacyTransformerLarge Language ModelPrompt EngineeringTextMultimodalityRetrieval-Augmented Generation
🎯 What it does: An end-to-end multimodal access control-aware LLM inference system AC-LORA is proposed and implemented, utilizing independent LoRA to achieve data isolation across different permission domains, and dynamically retrieving and mixing relevant LoRAs by similarity during inference, providing strict permission checks and prompt mechanisms.
Accelerated Distance-adaptive Methods for Hölder Smooth and Convex Optimization
Yijin Ren (Shanghai University of Finance and Economics), Qi Deng (Shanghai Jiao Tong University)
OptimizationTabularBiomedical Data
🎯 What it does: A new non-parametric accelerated first-order method is proposed to solve convex optimization problems with Hölder smoothness, aiming to improve convergence speed and eliminate the need for prior knowledge of the smoothness parameter.
Accelerated Evolving Set Processes for Local PageRank Computation
BinbinHuang, Baojian Zhou (Fudan University)
OptimizationComputational EfficiencyGraph Neural NetworkGraph
🎯 What it does: A local personalized PageRank (PPR) computation framework based on the nested evolution set process (AESP) is proposed, utilizing approximate accelerated proximal point iteration for faster solving.
Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking
Heli Ben-Hamu (FAIR Meta AI), Brian Karrer (FAIR Meta AI)
GenerationComputational EfficiencyTransformerDiffusion modelText
🎯 What it does: A pluggable adaptive multi-word decoding method called EB-Sampler is proposed, which dynamically determines how many words to decode at once in MDM sampling based on entropy.
Accelerated Vertical Federated Adversarial Learning through Decoupling Layer-Wise Dependencies
Tianxing Man (Jilin University), Yi Chang (Jilin University)
Federated LearningComputational EfficiencyAdversarial AttackGenerative Adversarial NetworkImage
🎯 What it does: The DecVFAL framework is proposed, which significantly accelerates vertical federated adversarial training (VFAL) through a dual-layer decoupling (lazy sequential backpropagation and decoupled parallel backpropagation) while maintaining model robustness.
Accelerating 3D Molecule Generative Models with Trajectory Diagnosis
Zhilong Zhang (Tsinghua University), Wei-Ying Ma (Tsinghua University)
GenerationComputational EfficiencyDrug DiscoveryGraph Neural NetworkFlow-based ModelGraphBiomedical Data
🎯 What it does: A method is proposed that divides the 3D molecular generation process into a rearrangement phase and a feature adjustment phase, with acceleration strategies designed for each phase, significantly improving generation efficiency.
Accelerating Block Coordinate Descent for LLM Finetuning via Landscape Expansion
Qijun Luo (Chinese University of Hong Kong), Xiao Li (Chinese University of Hong Kong)
OptimizationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: The BREAD method is proposed by combining Block Coordinate Descent (BCD) with Landscape Expansion (LoRA/SGD), significantly accelerating LLM fine-tuning while maintaining low memory consumption.
Accelerating data-driven algorithm selection for combinatorial partitioning problems
Vaggos Chatziafratis (University of California Santa Cruz), Ellen Vitercik (Stanford University)
OptimizationGraphStochastic Differential Equation
🎯 What it does: This study investigates the concept of 'size generalization' in data-driven algorithm selection, providing methods for estimating algorithm performance on large instances using small samples in two types of combinatorial optimization problems: clustering and maximum cut, along with theoretical guarantees and experimental validation.
Accelerating Diffusion LLMs via Adaptive Parallel Decoding
Daniel Mingyi Israel (University of California), Aditya Grover (University of California)
GenerationComputational EfficiencyTransformerLarge Language ModelMixture of ExpertsDiffusion modelText
🎯 What it does: This paper proposes the Adaptive Parallel Decoding (APD) method, which utilizes parallel sampling of the discrete diffusion language model (dLLM) and joint judgment with a small autoregressive model to dynamically determine how many tokens to sample at each step, significantly improving text generation speed.
Accelerating Feature Conformal Prediction via Taylor Approximation
Zihao Tang (Shanghai University of Finance and Economics), Jiaye Teng (Shanghai University of Finance and Economics)
ClassificationSegmentationOptimizationComputational EfficiencyImageTabular
🎯 What it does: A fast feature consistency prediction (FFCP) based on gradients is proposed, which accelerates traditional feature consistency prediction (FCP) through first-order Taylor expansion, achieving faster confidence interval construction.
Accelerating Model-Free Optimization via Averaging of Cost Samples
Guido Carnevale (Alma Mater Studiorum Universita di Bologna), Giuseppe Notarstefano (Alma Mater Studiorum Universita di Bologna)
OptimizationTabular
🎯 What it does: This paper proposes a model-independent optimization algorithm based on a memory mechanism, which approximates the gradient by caching the most recent cost samples in each direction and performing a weighted average across all directions, thereby accelerating convergence without increasing the number of function evaluations.
Accelerating Multimodal Large Language Models via Dynamic Visual-Token Exit and the Empirical Findings
Qiong Wu (Xiamen University), Rongrong Ji (Xiamen University)
Computational EfficiencyTransformerLarge Language ModelVision Language ModelTextMultimodality
🎯 What it does: To address the issue of visual token redundancy in multimodal large language models, this paper proposes and validates the Dynamic Visual Token Exit (DyVTE) mechanism.
Accelerating Optimization via Differentiable Stopping Time
Zhonglin Xie (Peking University), Zaiwen Wen (Peking University)
OptimizationTabularOrdinary Differential Equation
🎯 What it does: Proposes a concept of differentiable discrete stopping time, allowing for gradient optimization of the stopping steps in iterative optimization algorithms;
Accelerating Parallel Diffusion Model Serving with Residual Compression
Jiajun Luo (Tsinghua University), Zhi Wang (Tsinghua University)
GenerationCompressionComputational EfficiencyDiffusion modelImageVideoText
🎯 What it does: Designed and implemented CompactFusion—a parallel diffusion model inference acceleration framework based on residual compression, which reduces communication volume while maintaining high generation quality.
Accelerating RL for LLM Reasoning with Optimal Advantage Regression
Kianté Brantley (Harvard University), Xuezhou Zhang (Princeton University)
Reinforcement LearningTextBenchmark
🎯 What it does: A new reinforcement learning framework A⋆-PO is proposed, which directly approximates the optimal advantage function through a two-stage process, eliminating the need for explicit value networks and the overhead of multiple generations;
Accelerating Visual-Policy Learning through Parallel Differentiable Simulation
Haoxiang You (Yale University), Ian Abraham (Yale University)
Computational EfficiencyKnowledge DistillationRobotic IntelligenceConvolutional Neural NetworkReinforcement LearningAgentic AIImage
🎯 What it does: A visual strategy learning algorithm based on differentiable simulation and first-order analytical gradients (D.Va) is proposed, achieving efficient training by decoupling the rendering process from the computation graph.
Acceleration via silver step-size on Riemannian manifolds with applications to Wasserstein space
Jiyoung Park (Texas A&M University), Anirban Bhattacharya (Texas A&M University)
Optimization
🎯 What it does: This study investigates the use of silver step size accelerated optimization algorithms on Riemannian manifolds, particularly for applications in Wasserstein spaces.
Accident Anticipation via Temporal Occurrence Prediction
Tianhao Zhao (Wuhan University), Yutian Lin (Wuhan University)
Anomaly DetectionAutonomous DrivingConvolutional Neural NetworkTransformerVideo
🎯 What it does: This paper proposes a new accident prediction paradigm: switching from traditional per-frame risk scoring to predicting the probability of accidents occurring at different future time steps (0.1s to 2.0s), using precise accident timestamps for supervision, and designing an end-to-end network based on segment encoders for spatiotemporal feature extraction and a Transformer temporal decoder to achieve online frame-level prediction.
ACCO: Accumulate While You Communicate for Communication-Overlapped Sharded LLM Training
Adel Nabli (Sorbonne Université), Edouard Oyallon (Sorbonne Université)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: A new distributed LLM training algorithm called ACCO is proposed, which achieves overlapping of gradient communication and computation without increasing additional memory overhead, thereby reducing GPU idle time and supporting heterogeneous hardware.
AccuQuant: Simulating Multiple Denoising Steps for Quantizing Diffusion Models
Seunghoon Lee (Yonsei University), Bumsub Ham (Yonsei University)
GenerationData SynthesisOptimizationComputational EfficiencyDiffusion modelImage
🎯 What it does: A post-training quantization method called AccuQuant is proposed, which uses multi-step denoising simulation to reduce the cumulative quantization error of diffusion models.
Accurate and Efficient Low-Rank Model Merging in Core Space
Aniello Panariello (University of Modena and Reggio Emilia), Joost van de Weijer (Universitat Autònoma de Barcelona)
OptimizationComputational EfficiencyTransformerLarge Language ModelImageText
🎯 What it does: This paper proposes a Core Space framework for efficient fusion between low-rank adaptation (LoRA) models.
Accurate KV Cache Eviction via Anchor Direction Projection for Efficient LLM Inference
Zijie Geng (University of Science and Technology of China), Feng Wu (University of Science and Technology of China)
OptimizationComputational EfficiencyTransformerLarge Language ModelTextBenchmark
🎯 What it does: A new KV cache eviction method called AnDPro is proposed, which introduces a projection-based scoring function to more accurately measure the importance of tokens, thereby optimizing memory usage and inference latency of large language models.
Accurately Predicting Protein Mutational Effects via a Hierarchical Many-Body Attention Network
Dahao Xu (Sun Yat-sen University), Yuedong Yang (Sun Yat-sen University)
Drug DiscoveryProtein Structure PredictionGraph Neural NetworkSupervised Fine-TuningBiomedical Data
🎯 What it does: H3-DDG is proposed, a hierarchical multi-body attention network based on hypergraphs, for accurate prediction of protein interaction ΔΔG.
AceReason-Nemotron: Advancing Math and Code Reasoning through Reinforcement Learning
Yang Chen (NVIDIA), Wei Ping (NVIDIA)
AI Code AssistantLarge Language ModelReinforcement LearningText
🎯 What it does: Using a staged reinforcement learning approach of first mathematical RL and then code RL on existing powerful SFT models significantly enhances the reasoning capabilities of small and medium LLMs.
AceSearcher: Bootstrapping Reasoning and Search for LLMs via Reinforced Self-Play
Ran Xu (Emory University), Carl Yang (Emory University)
RetrievalTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Proposes AceSearcher, a retrieval-augmented model that simultaneously performs question decomposition and solving through cooperative self-play training of a single LLM.
Achieving $\tilde{\mathcal{O}}(1/N)$ Optimality Gap in Restless Bandits through Gaussian Approximation
Chen YAN, Lei Ying (University of Michigan)
OptimizationReinforcement Learning from Human FeedbackReinforcement LearningTabularTime Series
🎯 What it does: For the RMAB problem with limited time slots, a Gaussian approximation-based stochastic programming (SP) strategy is proposed, and its approximate optimality as N approaches infinity is provided.
Achilles' Heel of Mamba: Essential difficulties of the Mamba architecture demonstrated by synthetic data
Tianyi Chen (Shanghai Jiao Tong University), Zhi-Qin John Xu (Shanghai Jiao Tong University)
RecognitionData SynthesisTransformerSequential
🎯 What it does: This paper conducts systematic experiments on the Mamba architecture using synthetic data by designing composite function tasks and reverse matching tasks, revealing a fundamental bias in its recognition of symmetric patterns.
ACT as Human: Multimodal Large Language Model Data Annotation with Critical Thinking
Lequan Lin (University of Sydney), Junbin Gao (ByteDance)
ClassificationTransformerLarge Language ModelPrompt EngineeringImageTextMultimodalityChain-of-Thought
🎯 What it does: Designed and validated the ACT (Annotation with Critical Thinking) data pipeline: first, a multimodal large language model (MLLM) automatically annotates most of the data, then another MLLM evaluates the error probability, selecting high-risk samples for manual review according to the labor budget, significantly reducing the cost of manual annotation.
Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rollouts
Haizhong Zheng (Carnegie Mellon University), Beidi Chen (Lawrence Livermore National Laboratory)
Computational EfficiencyTransformerLarge Language ModelReinforcement LearningPrompt EngineeringText
🎯 What it does: Designed and implemented GRESO, an online lightweight method for filtering non-informative prompts in RL training, significantly reducing roll-out computation.
Act to See, See to Act: Diffusion-Driven Perception-Action Interplay for Adaptive Policies
Jing Wang (University of Alberta), Li cheng
Robotic IntelligenceReinforcement Learning from Human FeedbackDiffusion modelContrastive LearningSequentialStochastic Differential Equation
🎯 What it does: A perception-action closed-loop strategy DP-AG based on diffusion models is proposed.
Actial: Activate Spatial Reasoning Ability of Multimodal Large Language Models
Xiaoyu Zhan (Nanjing University), Yanwen Guo (Nanjing University)
Large Language ModelSupervised Fine-TuningReinforcement LearningMultimodalityBenchmarkChain-of-Thought
🎯 What it does: This paper proposes a viewpoint learning task that activates the spatial reasoning capabilities of multimodal large language models through a two-stage fine-tuning process.
Activated LoRA: Fine-tuned LLMs for Intrinsics
Kristjan Greenewald (IBM Research), David Daniel Cox
Computational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Proposes the Activated LoRA (aLoRA) framework, which implements dynamic activation of LoRA-based weights in multi-turn interactions, allowing for direct reuse of the base model's KV cache, thereby improving inference efficiency; based on this, a set of 'intrinsics' task models (uncertainty quantification, response judgment, query rewriting, jailbreak detection, etc.) are designed and trained.
Activation Control for Efficiently Eliciting Long Chain-of-thought Ability of Language Models
Zekai Zhao (University of California San Diego), Biwei Huang (University of California San Diego)
Explainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelPrompt EngineeringTextBenchmarkChain-of-Thought
🎯 What it does: By amplifying activation values and inserting the 'wait' trigger word, the long chain reasoning (Long-CoT) capability of large language models is activated without any training;
Activation-Guided Consensus Merging for Large Language Models
Yuxuan Yao (City University of Hong Kong), Linqi Song (City University of Hong Kong)
OptimizationKnowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: The Activation-Guided Consensus Merging (ACM) method is proposed, which adaptively determines hierarchical weight coefficients by calculating the mutual information of activations at different layers, achieving a gradient-free and additional training-free merging of pre-trained models and multi-task fine-tuned models.
Activation-Informed Merging of Large Language Models
Amin Heyrani Nobari (Massachusetts Institute of Technology), Navid Azizan (Massachusetts Institute of Technology)
TransformerLarge Language ModelSupervised Fine-TuningTextMultimodality
🎯 What it does: Proposes the Activation-Informed Merging (AIM) method, which uses activation space information to assist in merging multiple specialized fine-tuned LLMs based on the same pre-trained model, achieving a more robust model merging.
Active Measurement: Efficient Estimation at Scale
Max Hamilton (University of Massachusetts), Daniel Sheldon (University of Massachusetts)
Object DetectionConvolutional Neural NetworkImage
🎯 What it does: This paper proposes an active measurement framework for human-computer interaction, which obtains unbiased and variance-estimable scientific measurement results through AI model prediction, importance sampling, and incremental label updates.
Active Seriation: Efficient Ordering Recovery with Statistical Guarantees
James Cheshire (Institut Polytechnique de Paris), Yann Issartel (Institut Polytechnique de Paris)
Biomedical Data
🎯 What it does: An active serialization algorithm is proposed, aimed at recovering the unknown item order from the pairwise similarities of noisy observations.
Active Target Discovery under Uninformative Priors: The Power of Permanent and Transient Memory
Anindya Sarkar (Washington University in St. Louis), Yevgeniy Vorobeychik (Washington University in St. Louis)
Object DetectionOptimizationReinforcement LearningDiffusion modelImage
🎯 What it does: A framework for active target discovery under no prior information, called EM-PTDM, is proposed, which gradually enhances the prior and guides sampling using permanent memory (pre-trained diffusion models) and instantaneous memory (Doob's h-transform).
Active Test-time Vision-Language Navigation
Heeju Ko (Korea University), Sangpil Kim (Korea University)
Domain AdaptationRobotic IntelligenceTransformerReinforcement LearningVision Language ModelMultimodality
🎯 What it does: ATENA proposes an online active testing adaptation framework that enhances the robustness of visual language navigation agents in unknown environments by utilizing human turn feedback and self-supervision.
ActiveVOO: Value of Observation Guided Active Knowledge Acquisition for Open-World Embodied Lifted Regression Planning
Xiaotian Liu (University of Toronto), Scott Sanner (University of Toronto)
Robotic IntelligenceLarge Language ModelVision Language ModelMultimodalityBenchmark
🎯 What it does: This paper proposes an active knowledge acquisition framework called ACTIVEVOO based on Object Perceived Value (VOO) for embodied planning in open worlds.
Activity Pruning for Efficient Spiking Neural Networks
Tong Bu (Peking University), Zhaofei Yu (Peking University)
Computational EfficiencySpiking Neural NetworkImage
🎯 What it does: A method for activity pruning based on adaptive thresholds is designed, utilizing AT-LIF neurons to dynamically adjust thresholds during training and output 0/θ signals, significantly reducing the firing rate of SNNs and computational load.
Actor-Free Continuous Control via Structurally Maximizable Q-Functions
Yigit Korkmaz (University of Southern California), Erdem Biyik
Reinforcement LearningSequential
🎯 What it does: A continuous control reinforcement learning algorithm without an actor, Q3C, is proposed, which utilizes learnable control points to achieve a structure that maximizes the Q function;
AcuRank: Uncertainty-Aware Adaptive Computation for Listwise Reranking
Soyoung Yoon (Seoul National University), seung-won hwang
RetrievalRecommendation SystemComputational EfficiencyTransformerLarge Language ModelTextBenchmark
🎯 What it does: This paper presents AcuRank, an uncertainty-driven adaptive computing framework for dynamically selecting documents that require further reasoning in list-based re-ranking assisted by large language models (LLMs), and iteratively updating their relevance estimates.
Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference
Yuan Feng (University of Science and Technology of China), S Kevin Zhou
OptimizationComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: This paper studies and proposes Ada-KV, which allocates budgets adaptively based on attention heads to improve the quality of KV cache eviction and enhance the efficiency of LLM long sequence inference.
Ada-R1: Hybrid-CoT via Bi-Level Adaptive Reasoning Optimization
Haotian Luo (Shenzhen Campus of Sun Yat-sen University), Li Shen (Shenzhen Campus of Sun Yat-sen University)
OptimizationComputational EfficiencyTransformerLarge Language ModelTextChain-of-Thought
🎯 What it does: A dual-stage adaptive reasoning framework called Ada-R1 is proposed, allowing large language models to dynamically select between long-chain or short-chain reasoning based on input difficulty, significantly reducing reasoning length and cost while maintaining accuracy.
AdaDetectGPT: Adaptive Detection of LLM-Generated Text with Statistical Guarantees
Hongyi Zhou (Tsinghua University), Chengchun Shi (London School of Economics)
ClassificationAnomaly DetectionTransformerLarge Language ModelText
🎯 What it does: An adaptive LLM text detector called AdaDetectGPT is designed, which learns and applies the witness function to enhance the discriminative ability of existing logits-based detectors, providing finite sample guarantees for statistical metrics such as FNR, TNR, FPR, and TPR.
AdaLRS: Loss-Guided Adaptive Learning Rate Search for Efficient Foundation Model Pretraining
Hongyuan Dong (ByteDance Inc), Ran Jiao
TransformerLarge Language ModelVision Language ModelTextMultimodality
🎯 What it does: Proposes AdaLRS, which uses online loss descent speed search to approximate the optimal learning rate in a single run and significantly accelerate LLM/VLM pre-training;
Adam Reduces a Unique Form of Sharpness: Theoretical Insights Near the Minimizer Manifold
Xinghan Li (Institute for Interdisciplinary Information Sciences Tsinghua University), Kaifeng Lyu (Institute for Interdisciplinary Information Sciences Tsinghua University)
OptimizationTabularStochastic Differential Equation
🎯 What it does: This paper studies the implicit bias of Adam when approaching the minimizer manifold through theoretical analysis and experimental validation, proving that it utilizes adaptive semi-gradient to reduce a specific sharpness (tr(Diag(H)^{1/2})) and distinguishes itself from SGD;
AdaMSS: Adaptive Multi-Subspace Approach for Parameter-Efficient Fine-Tuning
Jingjing Zheng (University of British Columbia), Zhouchen Lin (Peking University)
ClassificationOptimizationTransformerSupervised Fine-TuningImageText
🎯 What it does: Proposes AdaMSS, an adaptive multi-subspace parameter-efficient fine-tuning method.
Adaptable Safe Policy Learning from Multi-task Data with Constraint Prioritized Decision Transformer
Ruiqi Xue (Nanjing University), Yang Yu (Nanjing University)
Safty and PrivacyTransformerReinforcement LearningTabularBenchmark
🎯 What it does: A decision Transformer-based offline safe reinforcement learning framework called CoPDT is proposed, which can adaptively learn safe policies in multi-task, multi-constraint, and multi-budget environments.
AdaptDel: Adaptable Deletion Rate Randomized Smoothing for Certified Robustness
Zhuoqun Huang (University of Melbourne), Benjamin I. P. Rubinstein (University of Melbourne)
ClassificationAdversarial AttackTextSequential
🎯 What it does: For sequence classification tasks, a deletion rate adaptive random smoothing method (AdaptDel/AdaptDel+) is proposed, achieving provable robustness against edit distance attacks.