ICML 2025 Papers — Page 3
International Conference on Machine Learning · 3257 papers
Aligning with Logic: Measuring, Evaluating and Improving Logical Preference Consistency in Large Language Models
Yinhong Liu (University of Cambridge), Nigel Collier (University of Cambridge)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: A general framework is proposed to evaluate the logical preference consistency of LLMs (transitivity, symmetry, negation invariance), and the REPAIR method is developed to estimate and enhance consistency through ranking.
All-atom Diffusion Transformers: Unified generative modelling of molecules and materials
Chaitanya K. Joshi (Meta), Zachary Ward Ulissi
GenerationData SynthesisDrug DiscoveryTransformerDiffusion modelAuto EncoderTabular
🎯 What it does: A unified All-atom Diffusion Transformer (ADiT) is proposed, which simultaneously generates molecules and materials through a variational autoencoder and latent diffusion model.
All-atom inverse protein folding through discrete flow matching
Kai Yi (MRC Laboratory of Molecular Biology), Sjors HW Scheres
Protein Structure PredictionGraph Neural NetworkFlow-based ModelBiomedical Data
🎯 What it does: A discrete flow matching-based all-atom inverse folding model, ADFLIP, is proposed, which can generate structurally consistent amino acid sequences for protein complexes containing small molecules, nucleic acids, or metal ions.
All-Purpose Mean Estimation over R: Optimal Sub-Gaussianity with Outlier Robustness and Low Moments Performance
Jasper C.H. Lee (University of California), Paul Valiant (Purdue University)
🎯 What it does: This paper studies a mean estimation algorithm applicable to various scenarios in the one-dimensional real number domain, proving that it achieves optimal error under standard i.i.d., contamination, and low-order moment distributions.
Almost Optimal Fully Dynamic $k$-Center Clustering with Recourse
Sayan Bhattacharya (University of Warwick), Nikos Parotsidis (Google Research)
Optimization
🎯 What it does: An algorithm is proposed that can maintain a k-center approximate solution after point insertion/deletion operations in dynamic metric spaces.
ALMTokenizer: A Low-bitrate and Semantic-rich Audio Codec Tokenizer for Audio Language Modeling
Dongchao Yang (Chinese University of Hong Kong), Helen M. Meng
GenerationCompressionTransformerAuto EncoderGenerative Adversarial NetworkAudio
🎯 What it does: Proposed and implemented ALMTokenizer, a low-bitrate, semantically rich audio codec designed specifically for audio language models;
Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Search
Boyan Li (Hong Kong University of Science and Technology), Yuyu Luo (Hong Kong University of Science and Technology)
Large Language ModelReinforcement LearningTextChain-of-Thought
🎯 What it does: This paper proposes Alpha-SQL, a zero-shot text-to-SQL framework that utilizes Monte Carlo Tree Search (MCTS) to incrementally construct SQL in the search tree and employs a Large Language Model (LLM) as the action model, combined with a self-supervised reward function for search guidance.
AlphaDPO: Adaptive Reward Margin for Direct Preference Optimization
Junkang Wu (University of Science and Technology of China), Xiangnan He (University of Science and Technology of China)
Recommendation SystemOptimizationTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Proposes AlphaDPO, which achieves adaptive reward margins through dynamic reparameterization of the reference distribution, improving the alignment of large language models with human preferences.
AlphaPO: Reward Shape Matters for LLM Alignment
Aman Gupta (LinkedIn Corporation), Sathiya Keerthi
OptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningTextBenchmark
🎯 What it does: The AlphaPO method is proposed, which improves the alignment performance of direct preference optimization by adjusting the α parameter to change the shape of the reward function.
AlphaQCM: Alpha Discovery in Finance with Distributional Reinforcement Learning
Zhoufan Zhu (Xiamen University), Ke Zhu (University of Hong Kong)
Recommendation SystemOptimizationRecurrent Neural NetworkReinforcement LearningTabularTime SeriesFinance Related
🎯 What it does: A framework called AlphaQCM, centered on distributed reinforcement learning, has been constructed. It models the process of finding collaborative formulaic Alpha as a non-stationary and reward-sparse Markov decision process, utilizing distributed Q-values and quantile networks to search for the optimal Alpha pool.
AlphaVerus: Bootstrapping Formally Verified Code Generation through Self-Improving Translation and Treefinement
Pranjal Aggarwal (Carnegie Mellon University), Sean Welleck (Carnegie Mellon University)
GenerationAI Code AssistantTransformerLarge Language ModelReinforcement LearningPrompt EngineeringText
🎯 What it does: AlphaVerus is a self-improvement framework that generates Rust code satisfying formal verification through LLM in the absence of training data.
am-ELO: A Stable Framework for Arena-based LLM Evaluation
Zirui Liu (University of Science and Technology of China), Shijin Wang (iFLYTEK Co., Ltd)
Large Language ModelReinforcement LearningText
🎯 What it does: A stable arena evaluation framework am-ELO based on MLE is proposed for evaluating LLMs.
AMPO: Active Multi Preference Optimization for Self-play Preference Selection
Taneesh Gupta (Microsoft), Saravan Rajmohan (Microsoft)
OptimizationReinforcement Learning from Human FeedbackLarge Language ModelReinforcement LearningContrastive LearningText
🎯 What it does: Proposes the Active Multi-Preference Optimization (AMPO) framework, which combines on-policy generation, group-contrastive REFA loss, and active subset selection for more efficient and diverse alignment of large language models in self-play.
An Adaptive Orthogonal Convolution Scheme for Efficient and Flexible CNN Architectures
Thibaut Boissin (Institut de Recherche Technologique Saint-Exupery), Mathieu Serrurier (Institut de Recherche Technologique Saint-Exupery)
Convolutional Neural NetworkImage
🎯 What it does: Proposes the AOC (Adaptive Orthogonal Convolution) method, which constructs convolutional layers that support stride, transposed, grouped, and dilated convolutions while maintaining strict orthogonality;
An All-Atom Generative Model for Designing Protein Complexes
Ruizhe Chen (Hunan University), Quanquan Gu (ByteDance Seed)
GenerationProtein Structure PredictionTransformerFlow-based ModelBiomedical Data
🎯 What it does: A generative model APM based on atomic-level design has been developed to generate multi-chain protein complexes from scratch, capable of performing folding, unfolding, and specific functional protein (antibody, peptide) design tasks.
An Analysis for Reasoning Bias of Language Models with Small Initialization
Junjie Yao (Shanghai Jiao Tong University), Zhi-Qin John Xu (Shanghai Jiao Tong University)
TransformerLarge Language ModelText
🎯 What it does: This study investigates how the small parameter initialization scale affects the learning preferences of large language models in reasoning and memory tasks, providing a theoretical explanation.
An analytic theory of creativity in convolutional diffusion models
Mason Kamb (Stanford University), Surya Ganguli (Stanford University)
GenerationConvolutional Neural NetworkDiffusion modelImage
🎯 What it does: An analytical and interpretable creative theory for convolutional diffusion models is proposed, explaining how it generates novel images that are related to the training data and can predict generation results on a per-sample basis.
An Architecture Search Framework for Inference-Time Techniques
Jon Saad-Falcon (Stanford University), Azalia Mirhoseini (Stanford University)
Neural Architecture SearchTransformerLarge Language ModelTextBenchmark
🎯 What it does: The ARCHON framework is proposed, which can automatically search and combine various inference techniques with different LLMs to form an efficient inference architecture.
An Asymptotically Optimal Approximation Algorithm for Multiobjective Submodular Maximization at Scale
Fabian Christian Spaeh (Boston University), Atsushi Miyauchi (CENTAI Institute)
OptimizationGraph
🎯 What it does: A scalable multi-objective submodular maximization algorithm is proposed and applied to the problem of maximizing fair centrality.
An Augmentation-Aware Theory for Self-Supervised Contrastive Learning
Jingyi Cui (Peking University), Yisen Wang (Peking University)
ClassificationRepresentation LearningContrastive LearningImage
🎯 What it does: A theory of augmented perception and an error upper bound are proposed, analyzing the impact of data augmentation on self-supervised contrastive learning, and validating its effectiveness at both pixel and representation levels.
An Effective and Secure Federated Multi-View Clustering Method with Information-Theoretic Perspective
Xinyue Chen (University of Electronic Science and Technology of China), Yazhou Ren (University of Electronic Science and Technology of China)
Federated LearningSafty and PrivacyAuto EncoderMultimodality
🎯 What it does: This paper proposes a method for splitting features in federated multi-view clustering from an information-theoretic perspective, sharing only the features relevant to clustering to enhance performance and reduce the risk of privacy leakage.
An Efficient Matrix Multiplication Algorithm for Accelerating Inference in Binary and Ternary Neural Networks
Mohsen Dehghankar (University of Illinois Chicago), Abolfazl Asudeh (University of Illinois Chicago)
OptimizationComputational EfficiencyLarge Language ModelText
🎯 What it does: A matrix multiplication acceleration algorithm for binary and ternary weight neural networks is proposed, which constructs an index by preprocessing the weight matrix to achieve inference acceleration and memory compression.
An Efficient Private GPT Never Autoregressively Decodes
Zhengyi Li (Shanghai Jiao Tong University), Minyi Guo (Shanghai Jiao Tong University)
GenerationSafty and PrivacyComputational EfficiencyKnowledge DistillationTransformerLarge Language ModelTextFinance Related
🎯 What it does: Proposes the POST (Public Decoding and Secure Verification) scheme, which utilizes publicly generated draft tokens from GPT and securely verifies them through a private model, achieving one-time multi-token generation.
An Efficient Pruner for Large Language Model with Theoretical Guarantee
Canhong Wen (University of Science and Technology of China), Wenliang Pan (Chinese Academy of Sciences)
OptimizationComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: A single-layer-wise pruning method based on ℓ0 penalty, mAIHT, is proposed, which combines theoretical guarantees with efficiency.
An Efficient Search-and-Score Algorithm for Ancestral Graphs using Multivariate Information Scores for Complex Non-linear and Categorical Data
Nikita Lagrange (Institut Curie), Herve Isambert
GraphTabular
🎯 What it does: A greedy search-scoring algorithm for ancestral graphs (containing directed and bidirectional edges) is proposed, utilizing multivariate cross-information likelihood and achieving efficient structure learning through local node/edge information scores.
An Empirical Study on Configuring In-Context Learning Demonstrations for Unleashing MLLMs' Sentimental Perception Capability
Daiqing Wu (Institute of Information Engineering Chinese Academy of Sciences), Yu Zhou (Nankai University)
ClassificationRecognitionTransformerLarge Language ModelPrompt EngineeringTextMultimodality
🎯 What it does: By configuring context learning examples in multimodal large language models, the emotional perception ability in multimodal sentiment analysis tasks has been enhanced.
An End-to-End Model for Logits-Based Large Language Models Watermarking
KA HIM WONG, Yain-Whar Si (University of Macau)
GenerationOptimizationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: An end-to-end logits perturbation watermarking model is proposed, capable of embedding detectable watermarks in LLM-generated text while maintaining text quality.
An Error Analysis of Flow Matching for Deep Generative Modeling
Zhengyu Zhou (Wuhan University), Weiwei Liu (Wuhan University)
GenerationData SynthesisFlow-based Model
🎯 What it does: This paper presents the first complete end-to-end error analysis of Continuous Normalizing Flows (CNFs) under the Flow Matching method, proving that the generated distribution converges to the target distribution in Wasserstein-2 distance.
An Expressive and Self-Adaptive Dynamical System for Efficient Function Learning
Chuan Liu (University of Rochester), Tong Geng
Time SeriesSequential
🎯 What it does: EADS is proposed, an adaptive dynamic framework that can achieve efficient function learning on electronic power systems.
An Improved Clique-Picking Algorithm for Counting Markov Equivalent DAGs via Super Cliques Transfer
Lifu Liu (Northeast Normal University), Jianhua Guo (Beijing Technology and Business University)
OptimizationComputational EfficiencyGraph Neural NetworkGraph
🎯 What it does: The Clique-Picking algorithm has been improved by introducing the concepts of super cliques and super residuals, and designing the Super Cliques Transfer algorithm, which allows for efficient reuse of structures between different root cliques, significantly reducing the computational complexity of the counting steps.
An in depth look at the Procrustes-Wasserstein distance: properties and barycenters
Davide Adamo (Universite Cote d'Azur), Emmanuelle Vila (Universite Lumiere Lyon II)
OptimizationImagePoint Cloud
🎯 What it does: Defines the Procrustes-Wasserstein (PW) distance properties on discrete probability measures and proposes an algorithm for solving the PW centroid; also presents and evaluates various PW initialization strategies.
An Instrumental Value for Data Production and its Application to Data Pricing
Rui Ai (Massachusetts Institute of Technology), Haifeng Xu (University of Chicago)
🎯 What it does: This paper proposes a framework for measuring the value of instruments in the data production process (DPP) and applies it to the mechanism design of data pricing, analyzing the optimal pricing strategies in both fully customizable and limited customizable scenarios.
An Interpretable N-gram Perplexity Threat Model for Large Language Model Jailbreaks
Valentyn Boreiko (University of Tübing), Jonas Geiping (Max Planck Institute for Intelligent Systems)
Explainability and InterpretabilityComputational EfficiencyAdversarial AttackTransformerLarge Language ModelText
🎯 What it does: Designed an explainable threat model based on n-gram perplexity, and adapted mainstream jailbreak attacks (PRS, GCG, AutoDan, BEAST, PAIR) to conform to this model for a systematic evaluation of security-tuned LLMs.
An Online Adaptive Sampling Algorithm for Stochastic Difference-of-convex Optimization with Time-varying Distributions
Yuhan Ye (Peking University), Jingyi Wang (Lawrence Livermore National Laboratory)
OptimizationTime Series
🎯 What it does: This paper proposes an online adaptive sampling DCA framework for stochastic nonsmooth concave optimization problems under time-varying distribution.
An Online Statistical Framework for Out-of-Distribution Detection
Xinsong Ma (Wuhan University), Weiwei Liu (Wuhan University)
Anomaly DetectionImage
🎯 What it does: Modeling OOD detection as an online multiple hypothesis testing problem, and proposing the g-LOND algorithm to achieve FDR control, further proving that under certain distributions, FPR can approach 0;
An Optimistic Algorithm for online CMDPS with Anytime Adversarial Constraints
Jiahui Zhu (Washington State University), Honghao Wei
OptimizationReinforcement Learning
🎯 What it does: The OMDPD algorithm is proposed to address the adversarial constraint problem in online CMDP at any given time.
AnalogGenie-Lite: Enhancing Scalability and Precision in Circuit Topology Discovery through Lightweight Graph Modeling
Jian Gao (Northeastern University), Xuan Zhang (Northeastern University)
GenerationOptimizationTransformerReinforcement LearningGraph
🎯 What it does: This paper presents AnalogGenie-Lite, a generative model based on a decoder-only transformer that automatically discovers new analog IC topologies using a lightweight graph modeling approach.
Analytical Construction on Geometric Architectures: Transitioning from Static to Temporal Link Prediction
Yadong Sun (Jilin University), Heng Tao Shen (University of Electronic Science and Technology of China)
Recommendation SystemOptimizationComputational EfficiencyGraph Neural NetworkGraphTime Series
🎯 What it does: A multi-geometric dynamic graph learning framework is proposed, which automatically selects appropriate geometric representations for each k-hop local subgraph using a combination of Euclidean and hyperbolic spaces. By integrating a temporal state aggregation layer and an evolution-driven optimization objective, it achieves efficient temporal link prediction.
Analytical Lyapunov Function Discovery: An RL-based Generative Approach
Haohan Zou (University of California San Diego), Yuanyuan Shi (University of California San Diego)
OptimizationTransformerReinforcement LearningTime Series
🎯 What it does: A symbolic Transformer framework based on reinforcement learning is proposed, which can automatically generate local analytical Lyapunov functions from a given nonlinear dynamic model.
Analyze Feature Flow to Enhance Interpretation and Steering in Language Models
Daniil Laptev (Moscow Institute of Physics and Technology), Daniil Gavrilov (Moscow Institute of Physics and Technology)
OptimizationExplainability and InterpretabilityTransformerLarge Language ModelAuto EncoderText
🎯 What it does: Using Sparse Autoencoders (SAE) to train feature vectors in different modules (MLP, attention, residual) of various layers in large language models, cross-layer matching of features is achieved through zero-data cosine similarity, constructing a flow graph, and utilizing these graphs to implement multi-level model behavior control (activation/suppression of topic generation).
Angle Domain Guidance: Latent Diffusion Requires Rotation Rather Than Extrapolation
Cheng Jin (Tsinghua University), Yuantao Gu (Tsinghua University)
GenerationDiffusion modelImageText
🎯 What it does: This paper proposes an Angle-Domain Guidance (ADG) method to replace the traditional Classifier-Free Guidance (CFG) in text-to-image latent diffusion models, generating higher quality images by limiting amplitude amplification and focusing on angle rotation.
Annealing Flow Generative Models Towards Sampling High-Dimensional and Multi-Modal Distributions
Dongze Wu (Georgia Institute of Technology), Yao Xie (Georgia Institute of Technology)
GenerationData SynthesisOptimizationFlow-based ModelTabularBiomedical DataOrdinary Differential Equation
🎯 What it does: Proposes a high-dimensional multimodal distribution sampling method based on Annealing Flow.
Antidote: Post-fine-tuning Safety Alignment for Large Language Models against Harmful Fine-tuning Attack
Tiansheng Huang (Georgia Institute of Technology), Ling Liu (Georgia Institute of Technology)
Safty and PrivacyAdversarial AttackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
🎯 What it does: This paper proposes a post-fine-tuning safety alignment defense method called Antidote, which uses a round of pruning to remove key parameters that cause the model to lose safety alignment during the fine-tuning process, thereby restoring the model's safety.
any4: Learned 4-bit Numeric Representation for LLMs
Mostafa Elhoushi (Meta), Jeff Johnson (Meta)
TransformerLarge Language ModelText
🎯 What it does: This paper proposes an arbitrary 4-bit (any4) numerical representation method for unprocessed low-bit quantization of large language model (LLM) weights, and provides a corresponding GPU-efficient matrix multiplication library called tinygemm.
AnyEdit: Edit Any Knowledge Encoded in Language Models
Houcheng Jiang (University of Science and Technology of China), Tat-Seng Chua (National University of Singapore)
Large Language ModelTextMultimodalityBenchmark
🎯 What it does: Proposed the AnyEdit autoregressive editing framework for efficiently and accurately updating long texts and multi-format knowledge in large language models;
Anytime-Constrained Equilibria in Polynomial Time
Jeremy McMahan (University of Wisconsin Madison)
Optimization
🎯 What it does: The study introduces anytime constraints in finite-horizon Markov games and proposes a corresponding equilibrium concept (Anytime-Constrained Equilibrium, ACE), along with a method for calculating feasible strategies.
Approximate Differential Privacy of the $\ell_2$ Mechanism
Matthew Joseph (Google Research), Alexander Yu (Google Research)
Safty and Privacy
🎯 What it does: This paper analyzes the glyph[lscript]2 mechanism under approximate differential privacy, proposing an algorithm to obtain the minimum σ in arbitrary dimensions and providing strict (ε,δ)-DP guarantees, along with a parallel sampling method.
Approximate Forest Completion and Learning-Augmented Algorithms for Metric Minimum Spanning Trees
Nate Veldt (Texas A&M University), Geoffrey Sanders (Lawrence Livermore National Laboratory)
Recommendation SystemOptimizationTabular
🎯 What it does: A method is proposed to solve the minimum spanning tree (MST) problem in arbitrary metric spaces by first constructing an observable forest (initial forest) and completing it with an approximate algorithm.
Approximately Correct Label Distribution Learning
Weiwei Li (Nanjing University of Aeronautics and Astronautics), Xiuyi Jia (Nanjing University of Science and Technology)
OptimizationSupervised Fine-TuningTabular
🎯 What it does: This study proposes DeltaLDL—a performance evaluation metric and learning objective based on 'approximately correct' label distributions to improve the performance assessment and training of label distribution learning.
Approximating Latent Manifolds in Neural Networks via Vanishing Ideals
Nico Pelleriti (Zuse Institute Berlin), Sebastian Pokutta (Zuse Institute Berlin)
ClassificationOptimizationComputational EfficiencyConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: After truncating the intermediate layers of the pre-trained deep network, the vanishing ideal algorithm is used to generate sparse polynomials for each category, forming a single-layer polynomial transformation that maps the latent space of the truncated network to a linearly separable space, and final classification is completed through a linear classifier.
Approximation to Smooth Functions by Low-Rank Swish Networks
Zimeng Li (Beihang University), Ke Tang (Tsinghua University)
CompressionTabular
🎯 What it does: This paper proves that low-rank Swish networks can approximate any Hölder smooth function with arbitrary precision, providing a constructive proof and an upper bound on network size;
Arbitrarily-Conditioned Multi-Functional Diffusion for Multi-Physics Emulation
Da Long (University of Utah), Shandian Zhe (University of Utah)
GenerationData SynthesisOptimizationDiffusion modelTabularPhysics Related
🎯 What it does: A probabilistic surrogate model based on diffusion models, ACM-FD, has been developed to perform various tasks such as forward prediction, inverse problem solving, function completion, and complete system simulation in multiphysics systems.
Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Models
Thomas Fel (Harvard University), Talia Konkle (Harvard University)
Explainability and InterpretabilityRepresentation LearningConvolutional Neural NetworkTransformerAuto EncoderImage
🎯 What it does: This paper addresses the instability exhibited by existing Sparse Autoencoders (SAE) in concept extraction, proposing and implementing two new methods—Archetypal SAE (A-SAE) and its relaxed version RA-SAE—to stabilize and enhance the interpretability of concepts by constraining dictionary atoms within the convex hull of the data.
Are High-Quality AI-Generated Images More Difficult for Models to Detect?
Yao Xiao (Sun Yat-sen University), Pengxu Wei (Xi'an Jiaotong University)
Object DetectionGenerationDiffusion modelImage
🎯 What it does: This study investigates the impact of high-quality AI-generated images on detection models and systematically evaluates the relationship between different generators, text prompts, image quality, and detection difficulty.
Are Large Brainwave Foundation Models Capable Yet ? Insights from Fine-Tuning
Na Lee (Imperial College London), Stefanos Zafeiriou (Imperial College London)
ClassificationTransformerSupervised Fine-TuningTime SeriesBiomedical Data
🎯 What it does: This paper evaluates the performance of two large-scale EEG pre-trained models (LaBraM and NeuroGPT) in actual EEG signal classification by fine-tuning them on five standard BCI benchmark tasks.
Are Large Language Models Ready for Multi-Turn Tabular Data Analysis?
Jinyang Li (University of Hong Kong), Reynold Cheng (University of Hong Kong)
TransformerLarge Language ModelAgentic AITabularBenchmarkChain-of-Thought
🎯 What it does: This paper proposes a new benchmark, COTA, for evaluating large language models (LLMs) in multi-turn conversational table data analysis tasks, and constructs a multi-agent sandbox environment, DECISION COMPANY, for efficiently generating high-quality conversational data. It also introduces an Adaptive Conversation Reflection (ACR) strategy to enhance the performance of LLMs as data analysis agents.
Are LLMs Prescient? A Continuous Evaluation using Daily News as the Oracle
Hui Dai (New York University), Mengye Ren (New York University)
TransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: A daily automatically generated prediction question-answer dataset called Daily Oracle has been constructed to continuously evaluate the predictive capabilities of large language models regarding future events.
Are Sparse Autoencoders Useful? A Case Study in Sparse Probing
Subhash Kantamneni (Massachusetts Institute of Technology), Neel Nanda (Massachusetts Institute of Technology)
ClassificationExplainability and InterpretabilityLarge Language ModelAuto EncoderText
🎯 What it does: This paper systematically evaluates the practicality of Sparse Autoencoders (SAE) in the activation detection of Large Language Models (LLM), constructing 113 binary classification detection datasets and conducting experiments in four challenging scenarios: data scarcity, class imbalance, label noise, and covariate shift.
Armijo Line-search Can Make (Stochastic) Gradient Descent Provably Faster
Sharan Vaswani (Simon Fraser University), Reza Babanezhad Harikandeh
OptimizationTabular
🎯 What it does: This paper provides a theoretical analysis of the optimization method that combines Armijo line search with gradient descent (GD) and stochastic gradient descent (SGD). It proves that for non-uniformly smooth functions (such as logistic regression, multi-class cross-entropy, GLM, softmax policy gradient, etc.), Armijo line search can adapt to local smoothness, thereby achieving faster convergence rates. It further demonstrates that in the linearly separable logistic regression problem, the convergence rate can reach linear (logarithmic level) rather than the traditional O(1/ε) rate.
ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior
Zhongweiyang Xu (University of Illinois Urbana-Champaign), Romit Roy Choudhury (University of Illinois Urbana-Champaign)
GenerationData SynthesisDiffusion modelAudio
🎯 What it does: This paper proposes ArrayDPS, an unsupervised, array-independent, generative blind source separation method;
Arrow: Accelerator for Time Series Causal Discovery with Time Weaving
Yuanyuan Yao (Zhejiang University), TIANYI LI
OptimizationComputational EfficiencyTime Series
🎯 What it does: Developed the ARROW (Accelerator for Time Series Causal Discovery with Time Weaving) accelerator, aimed at significantly improving the operational efficiency and accuracy of existing time series causal discovery algorithms.
ARS: Adaptive Reward Scaling for Multi-Task Reinforcement Learning
Myungsik Cho (Korea Advanced Institute of Science and Technology), Youngchul Sung (Korea Advanced Institute of Science and Technology)
Robotic IntelligenceReinforcement LearningSequential
🎯 What it does: Proposes the Adaptive Reward Scaling (ARS) framework to address the issue of uneven reward distribution in multi-task reinforcement learning through adaptive reward scaling and periodic network resets.
Assessing Safety Risks and Quantization-aware Safety Patching for Quantized Large Language Models
Kejia Chen (Zhejiang University), Mingli Song (Zhejiang University)
Safty and PrivacyLarge Language ModelText
🎯 What it does: Conduct a systematic security risk assessment of quantized large language models (LLMs) and propose a quantization-aware security patching framework Q-resafe to restore the model's security performance while maintaining utility.
AssistanceZero: Scalably Solving Assistance Games
Cassidy Laidlaw (University of California), Anca Dragan (University of California)
Robotic IntelligenceSupervised Fine-TuningReinforcement LearningSequential
🎯 What it does: A scalable AlphaZero-based method called AssistanceZero is proposed to solve assistance games and train AI assistants in large-scale Minecraft building assistance games.
Asymmetric Decision-Making in Online Knowledge Distillation: Unifying Consensus and Divergence
Zhaowei Chen (JIIOV Technology), Jiajun Liang
ClassificationSegmentationKnowledge DistillationConvolutional Neural NetworkImage
🎯 What it does: An asynchronous decision mechanism (ADM) is proposed, allowing students to focus on the foreground features consistent with the teacher during online knowledge distillation, while the teacher focuses on the foreground features that are inconsistent with the student, thereby enhancing the distillation effect.
AsymRnR: Video Diffusion Transformers Acceleration with Asymmetric Reduction and Restoration
Wenhao Sun (Nanyang Technological University), Dacheng Tao (Nanyang Technological University)
GenerationComputational EfficiencyTransformerDiffusion modelVideo
🎯 What it does: This paper proposes a training-independent, pluggable AsymRnR method that accelerates the sampling process of video diffusion Transformers by asymmetrically reducing and restoring tokens for Q and KV in self-attention.
ATA: Adaptive Task Allocation for Efficient Resource Management in Distributed Machine Learning
Arto Maranjyan, Francesco Orabona (King Abdullah University of Science and Technology)
OptimizationComputational EfficiencyConvolutional Neural NetworkImage
🎯 What it does: An Adaptive Task Allocation (ATA) algorithm is proposed to address heterogeneous and random computation times in distributed machine learning, significantly reducing resource waste while completing a predetermined number of B tasks in each round.
AtlasD: Automatic Local Symmetry Discovery
Manu Bhat (University of California San Diego), Rose Yu (University of California San Diego)
Convolutional Neural NetworkGraph Neural NetworkTabularTime SeriesPhysics Related
🎯 What it does: This paper proposes the AtlasD framework, which can automatically discover local symmetries (i.e., atlas equivariance) from data and express them as Lie group bases and discrete cosets;
Attention Mechanisms Perspective: Exploring LLM Processing of Graph-Structured Data
Zhong Guan (Tianjin University), Jianping Fan (Lenovo Research)
TransformerLarge Language ModelGraph
🎯 What it does: This paper systematically explores the behavior of the attention mechanism of large language models (LLMs) when processing graph-structured data, and analyzes the matching of attention distribution with graph topology through experimental analysis.
Attention-Level Speculation
Jack Cai (University of Toronto), Mark C. Jeffrey (University of Toronto)
OptimizationComputational EfficiencyTransformerLarge Language ModelTextBenchmarkChain-of-Thought
🎯 What it does: Proposes Attention Layered Speculative Parallel (ALSpec), which predicts self-attention output during inference and executes precise verification in parallel, thereby overlapping attention with subsequent computations and significantly reducing inference latency.
Attention-Only Transformers via Unrolled Subspace Denoising
Peng Wang (University of Michigan), Yi Ma (University of Hong Kong)
Representation LearningTransformerImageText
🎯 What it does: A Transformer model is proposed that uses only attention layers, adopting a subspace denoising perspective, treating each layer as a denoising iteration step to achieve feature compression and representation learning.
Attributes Shape the Embedding Space of Face Recognition Models
Pierrick Leroy (Politecnico di Torino), Francesco Vaccarino (Politecnico di Torino)
RecognitionExplainability and InterpretabilityGenerative Adversarial NetworkImage
🎯 What it does: This paper studies the geometric structure of the embedding space of facial recognition models, exploring how interpretable attributes (such as hair color, age, pose, etc.) shape this space on both macro scales (relationships between identity clouds) and micro scales (internal structure of a single identity), and proposes corresponding quantification methods.
Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilities
Sreyan Ghosh (University of Maryland), Bryan Catanzaro (NVIDIA)
RecognitionGenerationData-Centric LearningTransformerLarge Language ModelContrastive LearningMultimodalityAudio
🎯 What it does: Introducing Audio Flamingo 2 (AF2), an audio-language model capable of processing audio up to 5 minutes long with expert-level reasoning abilities;
Auditing $f$-differential privacy in one run
Saeed Mahloujifar (Meta), Kamalika Chaudhuri (Meta)
Safty and PrivacyImage
🎯 What it does: A method for auditing mechanisms with a single training session, called f-DP, is proposed, along with a corresponding upper bound analysis.
Auditing Prompt Caching in Language Model APIs
Chenchen Gu (Stanford University), Tatsunori Hashimoto (Stanford University)
Safty and PrivacyTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper conducts a systematic audit of the prompt caching mechanism in existing large language model (LLM) APIs, constructing a statistical hypothesis testing framework to detect the existence of caches and the level of cache sharing (user-level, organization-level, global-level).
AuPair: Golden Example Pairs for Code Repair
Aditi Mavalankar (Google DeepMind), Tom Schaul (Google DeepMind)
OptimizationAI Code AssistantTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: An algorithm named AuPair is proposed, which enhances code repair performance by selecting ordered golden example pairs during inference.
Auto-reconfiguration for Latency Minimization in CPU-based DNN Serving
Ankit Bhardwaj (Massachusetts Institute of Technology), Ryan Stutsman (University of Utah)
OptimizationComputational EfficiencyTransformerLarge Language ModelImageText
🎯 What it does: Packrat, a CPU-oriented DNN server system, has been designed and implemented to minimize inference latency by dynamically partitioning model instances, threads, and batch sizes, and supports online reconfiguration without downtime.
AutoAdvExBench: Benchmarking Autonomous Exploitation of Adversarial Example Defenses
Nicholas Carlini (Google DeepMind), Florian Tramèr (ETH Zurich)
Adversarial AttackTransformerLarge Language ModelImageBenchmark
🎯 What it does: A new benchmark called AutoAdvExBench has been constructed, providing 75 real or CTF-style implementations of adversarial sample defenses, allowing large language models (LLMs) to automatically generate attack samples and evaluate their success rates given defense papers and code.
AutoAL: Automated Active Learning with Differentiable Query Strategy Search
Yifeng Wang (Carnegie Mellon University), Siyu Huang (Clemson University)
ClassificationOptimizationConvolutional Neural NetworkReinforcement LearningImageBiomedical Data
🎯 What it does: Proposes AutoAL, a method for automatically searching for optimal active learning query strategies through a bi-level differentiable optimization framework;
AutoCATE: End-to-End, Automated Treatment Effect Estimation
Toon Vanderschueren (KU Leuven), Wouter Verbeke (KU Leuven)
OptimizationHyperparameter SearchTabular
🎯 What it does: Developed and validated AutoCATE, a fully automated, end-to-end causal effect estimation framework that can automatically search for and construct the optimal CATE estimation pipeline in observational data.
AUTOCIRCUIT-RL: Reinforcement Learning-Driven LLM for Automated Circuit Topology Generation
Prashanth Vijayaraghavan (IBM Almaden Research Center), Xin Zhang (IBM Thomas J. Watson Research Center)
OptimizationTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTabular
🎯 What it does: Proposes the AUTOCIRCUIT-RL framework, which combines large language models and reinforcement learning to achieve automatic simulation circuit topology generation.
AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling
Alexander Capstick (Imperial College London), Payam Barnaghi (Imperial College London)
OptimizationExplainability and InterpretabilityDrug DiscoveryLarge Language ModelPrompt EngineeringTabularBiomedical DataAlzheimer's Disease
🎯 What it does: Utilizing large language models (LLM) to automatically extract expert prior distributions for constructing interpretable linear predictive models, thereby improving predictive performance in scenarios with insufficient samples.
Autoencoder-Based Hybrid Replay for Class-Incremental Learning
Milad Khademi Nori (Toronto Metropolitan University), Guanghui Wang (Toronto Metropolitan University)
ClassificationOptimizationKnowledge DistillationRepresentation LearningAuto EncoderImage
🎯 What it does: A hybrid replay (AHR) strategy based on autoencoders is proposed for class-incremental learning (CIL), which compresses examples stored in latent space, merging the advantages of explicit replay and generative replay.
AutoEval Done Right: Using Synthetic Data for Model Evaluation
Pierre Boyeau (University of California), Michael I. Jordan (University of California)
Data SynthesisRecommendation SystemImageTabular
🎯 What it does: A statistically unbiased and lower variance model evaluation method (AutoEval) is proposed by combining synthetic labels generated by artificial intelligence with a small amount of manually labeled data.
Autoformulation of Mathematical Optimization Models Using LLMs
Nicolás Astorga (University of Cambridge), Mihaela van der Schaar (University of Cambridge)
OptimizationTransformerLarge Language ModelText
🎯 What it does: This paper proposes an automatic modeling method based on large language models (LLM) that can automatically convert natural language problem descriptions into solvable mathematical optimization models.
AutoGFM: Automated Graph Foundation Model with Adaptive Architecture Customization
Haibo Chen (Tsinghua University), Wenwu Zhu (Tsinghua University)
Graph Neural NetworkContrastive LearningGraph
🎯 What it does: This paper proposes AutoGFM, a foundational model for graph neural networks (GNNs) that can automatically customize GNN architectures for each graph in multi-task and multi-domain graph data.
Automated Benchmark Generation for Repository-Level Coding Tasks
Konstantinos Vergopoulos (LogicStar AI), Martin Vechev (ETH Zurich)
AI Code AssistantLarge Language ModelTextBenchmark
🎯 What it does: This paper presents SETUPAGENT, an automated tool driven by LLM for constructing historical dependencies, installation, and testing commands, and uses it to generate larger, more diverse, and frequently updated repository-level code generation benchmarks SWEE-Bench and SWA-Bench.
Automated Hypothesis Validation with Agentic Sequential Falsifications
Kexin Huang (Stanford University), Jure Leskovec (Stanford University)
Large Language ModelAgentic AIBiomedical Data
🎯 What it does: Constructed and implemented the POPPER framework, utilizing two LLM agent systems (experimental design and experimental execution) to automate the generation and execution of falsifiable experiments targeting natural language hypotheses, with strict Type-I error control through progressively aggregated e-values; validated hundreds of hypotheses across six fields including biology, economics, and sociology.
Automated Red Teaming with GOAT: the Generative Offensive Agent Tester
Maya Pavlova (Meta), Aaron Grattafiori (Meta)
Adversarial AttackTransformerLarge Language ModelTextChain-of-Thought
🎯 What it does: An automated multi-round red team attack system called GOAT is proposed, which utilizes a general LLM to dynamically combine different jailbreak techniques in conversations to induce the target model to produce inappropriate content.
Automatic Differentiation of Optimization Algorithms with Time-Varying Updates
Sheheryar Mehmood (Saarland University), Peter Ochs (Saarland University)
OptimizationImageTabular
🎯 What it does: This paper studies the automatic differentiation of optimization algorithms for time-varying updates and provides guarantees for the convergence and convergence rate of derivative iterations.
Automatic Reward Shaping from Confounded Offline Data
Mingxuan Li (Columbia University), Elias Bareinboim (Columbia University)
Reinforcement LearningSequential
🎯 What it does: This paper proposes a method for automatically generating potential functions using offline data affected by confounding, and applies it to potential function reward shaping (PBRS) to accelerate UCB-based online reinforcement learning.
Automatically Identify and Rectify: Robust Deep Contrastive Multi-view Clustering in Noisy Scenarios
Xihong Yang (National University of Defense Technology), Yueming Jin (National University of Singapore)
Anomaly DetectionAuto EncoderContrastive LearningImage
🎯 What it does: A deep contrast multi-view clustering framework AIRMVC is proposed, which can achieve robust clustering in noisy environments.
Automatically Interpreting Millions of Features in Large Language Models
Gonçalo Santos Paulo (EleutherAI), Nora Belrose (EleutherAI)
Explainability and InterpretabilityComputational EfficiencyLarge Language ModelAuto EncoderText
🎯 What it does: An automated process has been established to generate natural language explanations for millions of features of Sparse Autoencoders (SAE) using large language models, and five more efficient methods for evaluating explanation quality have been proposed.
AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoML
Patara Trirat (DeepAuto), Sung Ju Hwang (DeepAuto)
TransformerLarge Language ModelPrompt EngineeringImageTextTabularTime SeriesRetrieval-Augmented Generation
🎯 What it does: A multi-agent LLM framework called AutoML-Agent is constructed to achieve end-to-end AutoML from data retrieval to model deployment.
Autonomy-of-Experts Models
Ang Lv (Renmin University of China), Rui Yan (Renmin University of China)
OptimizationTransformerMixture of ExpertsText
🎯 What it does: A new Mixture-of-Experts (MoE) model is proposed—Autonomy-of-Experts (AoE), which eliminates the router and allows experts to self-select whether to process the input based on their own activation norms;
AutoStep: Locally adaptive involutive MCMC
Tiange Liu (University of British Columbia), Trevor Campbell (University of British Columbia)
OptimizationReinforcement LearningTabular
🎯 What it does: An adaptive step size involutive MCMC method called AutoStep is proposed, which can automatically select an appropriate step size at each step based on the local geometry of the target distribution, thereby achieving efficient sampling.
Average Certified Radius is a Poor Metric for Randomized Smoothing
Chenhao Sun (ETH Zurich), Martin Vechev (ETH Zurich)
ClassificationAdversarial AttackGaussian SplattingImage
🎯 What it does: This paper studies the effectiveness of the average certification radius (ACR) as a metric for evaluating the robustness of random smoothing (RS) and reveals its shortcomings in both theory and practice.
Average Sensitivity of Hierarchical $k$-Median Clustering
Shijie Li (University of Science and Technology of China), Pan Peng (University of Science and Technology of China)
OptimizationSafty and PrivacyTabular
🎯 What it does: A hierarchical k-median clustering algorithm based on the exponential mechanism is proposed, and its theoretical upper bounds for average sensitivity and clustering quality are provided; the high average sensitivity of single-linkage clustering and the CLNSS algorithm is also analyzed.
Avoiding Catastrophe in Online Learning by Asking for Help
Benjamin Plaut (University of California), Stuart Russell (University of California)
🎯 What it does: Proposes an online learning model that uses mentor assistance and local generalization to avoid irrecoverable catastrophic errors, and presents algorithms that achieve sub-constant regret and sub-linear queries.
Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
Mateo Espinosa Zarlenga (University of Cambridge), Mateja Jamnik (University of Cambridge)
ClassificationDomain AdaptationContrastive LearningImage
🎯 What it does: This study investigates the impact of Concept-Based Models (CBMs) on concept intervention under distribution shift (OOD) and proposes a new MixCEM model to avoid leakage poisoning.