ICML 2025 Papers with AI Summaries
International Conference on Machine Learning · 3257 papers
→ ICML 2025 papers with code (722)
Each paper below shows an AI-generated one-line summary. Get the full 6-part summary (innovation, method, data, results, limitations) and search all 3257 ICML 2025 papers by keyword, author or institution —
free trial on arXivSub.
"Who experiences large model decay and why?" A Hierarchical Framework for Diagnosing Heterogeneous Performance Drift
Harvineet Singh (University of California), Jean Feng (University of California)
Domain AdaptationAnomaly DetectionTextTabular
🎯 What it does: This paper presents SHIFT, a hierarchical hypothesis testing framework designed to detect and explain significant performance degradation of machine learning models across different subgroups during transfer or over time.
"Why Is There a Tumor?": Tell Me the Reason, Show Me the Evidence
Mengmeng Ma (University of Delaware), Xi Peng (University of Delaware)
Object DetectionSegmentationExplainability and InterpretabilityLarge Language ModelVision Language ModelImageBiomedical DataMagnetic Resonance Imaging
🎯 What it does: This paper proposes an interpretable medical imaging model that can perform tumor detection and segmentation while generating clinical terminology reasoning and specifying location information.
(How) Can Transformers Predict Pseudo-Random Numbers?
Tao Tao (University of Maryland), Maissam Barkeshli (University of Maryland)
TransformerSequential
🎯 What it does: This study investigates the use of Transformer in learning and predicting linear congruential generator (LCG) sequences, exploring its generalization ability for unknown moduli and parameters.
(How) Do Language Models Track State?
Belinda Z. Li (Massachusetts Institute of Technology), Jacob Andreas (Massachusetts Institute of Technology)
TransformerLarge Language ModelSupervised Fine-TuningTextSequential
🎯 What it does: This paper investigates how Transformer language models track hidden states using the Permutation Composition task, and identifies two algorithms implemented internally by the model—Associative Algorithm (AA) and Parity-Associative Algorithm (PAA)—through interpretability techniques such as activation patching, linear probing, and attention patterns.
$\infty$-Video: A Training-Free Approach to Long Video Understanding via Continuous-Time Memory Consolidation
Saul Santos (Instituto de Telecomunicacoes), Andre Martins
RecognitionRetrievalTransformerLarge Language ModelVideoMultimodality
🎯 What it does: A continuous time long-term memory mechanism (LTM) is proposed, which does not require additional training and can extend the existing short video Q-Former model into a long video understanding framework (∞-VIDEO) capable of handling videos of arbitrary length.
$\mathcal{V}ista\mathcal{DPO}$: Video Hierarchical Spatial-Temporal Direct Preference Optimization for Large Video Models
Haojian Huang (University of Hong Kong), Hao Fei (National University of Singapore)
OptimizationReinforcement LearningVideoText
🎯 What it does: A hierarchical space-time direct preference optimization framework (VistaDPO) is proposed, which aligns video-language preferences at a fine-grained level through three layers: instance, time, and perception.
$\texttt{I$^2$MoE}$: Interpretable Multimodal Interaction-aware Mixture-of-Experts
Jiayi Xin (University of Pennsylvania), Qi Long (University of Pennsylvania)
Explainability and InterpretabilityMixture of ExpertsMultimodalityBiomedical DataAlzheimer's DiseaseElectronic Health Records
🎯 What it does: This paper proposes an interpretable multimodal interactive perception mixture of experts framework, I MoE 2, specifically designed to capture different types of multimodal interactions (unique, mutual, redundant) and adaptively weight the prediction results.
$K^2$VAE: A Koopman-Kalman Enhanced Variational AutoEncoder for Probabilistic Time Series Forecasting
Xingjian Wu (East China Normal University), Chenjuan Guo (East China Normal University)
Auto EncoderTime SeriesBenchmark
🎯 What it does: This paper proposes a variational autoencoder framework called K<sup>2</sup>VAE for long-term probabilistic time series forecasting; it maps nonlinear time series to measurement space and linearizes it (KoopmanNet), then uses KalmanNet to adaptively correct the predictions and uncertainties of linear dynamics, ultimately outputting the predictive distribution in a single generation step;
$S^2$FGL: Spatial Spectral Federated Graph Learning
Zihan Tan (Wuhan University), Mang Ye (Wuhan University)
Federated LearningKnowledge DistillationGraph Neural NetworkGraph
🎯 What it does: In response to the client drift problem caused by label signal interruption and spectral heterogeneity in subgraph federated learning for molecular graph learning, this paper proposes the S2FGL framework, which addresses label loss and spectral drift using a global prototype library and spectral projection alignment.
3D Question Answering via only 2D Vision-Language Models
FENGYUN WANG, Qianru Sun (Singapore Management University)
TransformerVision Language ModelMultimodality
🎯 What it does: Utilizing a pre-trained 2D visual language model (LLAVA-OV) to perform 3D scene question answering tasks in a zero-shot manner, the core method involves the automatic selection of key and diverse 2D views (cdViews).
3D-LMVIC: Learning-based Multi-View Image Compression with 3D Gaussian Geometric Priors
Yujun Huang (Shenzhen International Graduate School Tsinghua University), Shu-Tao Xia (Shenzhen International Graduate School Tsinghua University)
Depth EstimationCompressionGaussian SplattingImage
🎯 What it does: This paper proposes 3D-LMVIC, a learning-based multi-view image compression framework that utilizes 3D Gaussian Splatting as a geometric prior, capable of accurately estimating disparities across views and effectively merging reference view information.
A Bayesian Model Selection Criterion for Selecting Pretraining Checkpoints
Michael Munn (Google Research), Susan Wei (Monash University)
Domain AdaptationOptimizationConvolutional Neural NetworkImageStochastic Differential Equation
🎯 What it does: This paper proposes a pre-training checkpoint selection criterion based on Bayesian free energy, aimed at predicting the adaptability of models to unseen downstream tasks.
A Bregman Proximal Viewpoint on Neural Operators
Abdel-Rahim Mezidi (Universite Jean Monnet), Marc Sebban (Universite Jean Monnet)
OptimizationTabularBenchmark
🎯 What it does: A framework is proposed that views neural operator layers as a minimization of Bregman regularization optimization problems, and based on this, a new Bregman neural operator is designed, unifying and extending existing neural operator models.
A Causal World Model Underlying Next Token Prediction: Exploring GPT in a Controlled Environment
Raanan Yehezkel Rohekar (Intel Labs), Vasudev Lal (Intel Labs)
TransformerLarge Language ModelWorld ModelText
🎯 What it does: This paper proposes a zero-shot causal structure learning method by associating the masking attention mechanism of GPT with structural causal models (SCM), defines a confidence score based on p-value entropy difference, and subsequently validates the method in two controlled game environments, Othello and Chess, demonstrating that GPT implicitly learns an interpretable causal world model.
A Certified Unlearning Approach without Access to Source Data
Umit Yigit Basaran (University of California), Basak Guler (University of California)
Data-Centric LearningImage
🎯 What it does: A framework for achieving certified model forgetting in the absence of available passive data is proposed, utilizing a proxy dataset to approximate the original distribution and ensuring the forgetting effect through noise calibration.
A Chaotic Dynamics Framework Inspired by Dorsal Stream for Event Signal Processing
Yu Chen (Lanzhou University), Gang Wang (Beijing Institute of Basic Medical Sciences)
ClassificationObject DetectionConvolutional Neural NetworkImageVideoOrdinary Differential Equation
🎯 What it does: A chaotic dynamics framework inspired by the dorsal visual pathway is proposed, which encodes the event streams generated by event cameras using Continuous Coupled Neural Networks (CCNN) and extracts higher-order mappings through continuous wavelet transform, thereby generating stable and general event representations, which are combined with traditional CNNs for object classification.
A Checks-and-Balances Framework for Context-Aware Ethical AI Alignment
Edward Y Chang
ClassificationGenerationExplainability and InterpretabilityReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: A check-and-balance framework based on a tripartite system (executive, legislative, judicial) is proposed, separating LLM knowledge generation from ethical oversight, and achieving global ethical alignment through emotional regulation and adversarial review.
A Classification View on Meta Learning Bandits
Mirco Mutti (Technion Israel Institute of Technology), Aviv Tamar (Technion Israel Institute of Technology)
Explainability and InterpretabilityMeta LearningTabular
🎯 What it does: This paper proposes a classification perspective to study the bandwidth problem in meta-learning, aiming to design interpretable and fast exploration plans for a fixed set of bandwidths.
A Closer Look at Backdoor Attacks on CLIP
Shuo He (Nanyang Technological University), Lei Feng (Southeast University)
Representation LearningAdversarial AttackTransformerImage
🎯 What it does: This paper systematically analyzes the infection characteristics of different types of backdoor attacks on the model's attention heads (AH) and multi-layer perceptron (MLP) layers through the decomposition and mean ablation of the image representations of the CLIP model. Based on these findings, two backdoor defense schemes are designed: one is representation repair for infected AH/MLP (Decomp-Rep), and the other is detection of backdoor samples using the number of infected AHs (Decomp-Det);
A Closer Look at Generalized BH Algorithm for Out-of-Distribution Detection
Xinsong Ma (Wuhan University), Weiwei Liu (Wuhan University)
Anomaly DetectionConvolutional Neural NetworkImage
🎯 What it does: This paper studies the impact of calibration set size on detection performance when using the generalized BH (g-BH) algorithm in OOD detection. It theoretically analyzes the conditional expected distribution of TPR for g-BH and proposes an ensemble g-BH (eg-BH) algorithm to address the small calibration set issue.
A Closer Look at Multimodal Representation Collapse
Abhra Chaudhuri (Fujitsu Research of Europe), Serban Georgescu (Fujitsu Research of Europe)
Anomaly DetectionKnowledge DistillationRepresentation LearningContrastive LearningMultimodalityBiomedical DataElectronic Health Records
🎯 What it does: This paper conducts an in-depth analysis of the modal collapse phenomenon that occurs in multimodal fusion models and proposes methods to suppress this phenomenon through cross-modal knowledge distillation and explicit basis redistribution (EBR).
A Closer Look at Transformers for Time Series Forecasting: Understanding Why They Work and Where They Struggle
Yu Chen (Imperial College London), Payam Barnaghi (Imperial College London)
TransformerTime SeriesBiomedical Data
🎯 What it does: This paper systematically compares the performance of different tokenization methods (point, patch, variate) and attention mechanisms of Transformers in time series forecasting through mutual information metrics and synthetic experiments, and validates the effectiveness of simple Transformers with Z-score normalization on benchmark datasets.
A Cognac Shot To Forget Bad Memories: Corrective Unlearning for Graph Neural Networks
Varshita Kolipaka (International Institute of Information Technology Hyderabad), Ponnurangam Kumaraguru (International Institute of Information Technology Hyderabad)
Graph Neural NetworkContrastive LearningGraph
🎯 What it does: This paper studies the issue of correcting unlearned problems in graph neural networks and proposes the Cognac method to eliminate the impact of manipulated samples on the model with only 5% of the manipulated samples known.
A Comprehensive Framework for Analyzing the Convergence of Adam: Bridging the Gap with SGD
Ruinan Jin (Hong Kong Institute of Science and Innovation Chinese Academy of Sciences), Baoxiang Wang (Chinese University of Hong Kong)
Optimization
🎯 What it does: A general framework is proposed to prove the asymptotic convergence, L1 convergence, and non-asymptotic sample complexity O(ln T/√T) of Adam under the weak assumptions of L-smoothness and the ABC inequality (the same weak assumptions as SGD);
A Computationally Efficient Algorithm for Infinite-Horizon Average-Reward Linear MDPs
Kihyuk Hong (University of Michigan), Ambuj Tewari (University of Michigan)
Computational EfficiencyReinforcement Learning
🎯 What it does: A linear MDP reinforcement learning algorithm for infinite periodic average reward that is independent of computational complexity and state space (γ‑DC‑LSCVI‑UCB) is proposed.
A Cross Modal Knowledge Distillation & Data Augmentation Recipe for Improving Transcriptomics Representations through Morphological Features
Ihab Bendidi (Valence Labs), Emmanuel Noutahi (Valence Labs)
Knowledge DistillationRepresentation LearningContrastive LearningMultimodalityBiomedical Data
🎯 What it does: A framework has been constructed that utilizes weakly paired microscopic images and transcriptomic data for cross-modal knowledge distillation and data augmentation, enhancing the predictive power and interpretability of single transcriptomic representations.
A Dynamical Systems-Inspired Pruning Strategy for Addressing Oversmoothing in Graph Attention Networks
Biswadeep Chakraborty (Georgia Institute of Technology), Saibal Mukhopadhyay (Georgia Institute of Technology)
OptimizationComputational EfficiencyGraph Neural NetworkGraph
🎯 What it does: A dynamic pruning strategy called DYNAMO-GAT based on dynamical systems theory is proposed to address the over-smoothing problem in deep graph attention networks.
A First-order Generative Bilevel Optimization Framework for Diffusion Models
Quan Xiao (Rensselaer Polytechnic Institute), Tianyi Chen (Cornell University)
GenerationOptimizationHyperparameter SearchDiffusion modelImage
🎯 What it does: A first-order gradient-based generative bi-level optimization framework is proposed to adjust the hyperparameters of diffusion models, addressing two types of bi-level problems: fine-tuning (reward alignment) and noise scheduling (noise timetable during training).
A Forget-and-Grow Strategy for Deep Reinforcement Learning Scaling in Continuous Control
Zilin Kang (Tsinghua University), Huazhe Xu (Tsinghua University)
Reinforcement LearningSequential
🎯 What it does: A deep reinforcement learning framework called FoG is proposed, which combines forgetting and growth to address the issue of prior bias in continuous control tasks.
A General Framework for Inference-time Scaling and Steering of Diffusion Models
Raghav Singhal (New York University), Rajesh Ranganath
GenerationData SynthesisOptimizationReinforcement LearningDiffusion modelImageText
🎯 What it does: A control framework for inference time diffusion models based on Feynman-Kac interactive particle systems (FK steering) is proposed, which can resample the diffusion process using any reward function to obtain samples that better align with user preferences.
A General Graph Spectral Wavelet Convolution via Chebyshev Order Decomposition
Nian Liu (National University of Singapore), Xavier Bresson (National University of Singapore)
Graph Neural NetworkGraph
🎯 What it does: This paper proposes a spectral graph convolutional network called WaveGC based on graph wave functions, which can capture both short-range and long-range information simultaneously.
A General Representation-Based Approach to Multi-Source Domain Adaptation
Ignavier Ng (Carnegie Mellon University), Kun Zhang (Mohamed bin Zayed University of Artificial Intelligence)
Domain AdaptationConvolutional Neural NetworkAuto EncoderImage
🎯 What it does: This paper proposes a multi-source domain adaptation framework called GAMA based on latent representations. It first partitions the Markov blanket of labels into three subspaces: parents, children, and spouses, and learns identifiable latent representations. Then, it utilizes these representations to achieve unsupervised adaptation in the target domain.
A Generalizable Physics-Enhanced State Space Model for Long-Term Dynamics Forecasting in Complex Environments
Yuchen Wang (William and Mary), Huajie Shao (William and Mary)
Auto EncoderTime SeriesPhysics RelatedOrdinary Differential Equation
🎯 What it does: A general physics-enhanced state space model named Phy-SSM is proposed for long-term dynamic prediction in complex environments with noise and irregular sampling.
A Generalization Result for Convergence in Learning-to-Optimize
Michael Sucker (University of Tübingen), Peter Ochs (Saarland University)
Optimization
🎯 What it does: A probabilistic framework is proposed that combines PAC-Bayes generalization with the Kurdyka-Łojasiewicz inequality, providing high-probability convergence guarantees for learning optimization algorithms, and its effectiveness is validated through experiments in quadratic optimization and neural network training.
A Generalization Theory for Zero-Shot Prediction
Ronak Mehta (University of Washington), Zaid Harchaoui (University of Washington)
ClassificationRecognitionPrompt EngineeringContrastive LearningImage
🎯 What it does: This paper studies the theoretical foundation of Zero-Shot Prediction (ZSP) and proposes a three-part decomposition of prediction error (prompt bias, residual dependence, estimation error). It provides generalization bounds and experimental validation for two types of learning methods (conditional mean and information density).
A Generic Family of Graphical Models: Diversity, Efficiency, and Heterogeneity
Yufei Huang (Peking University), Ruibin Xi (Peking University)
TabularBiomedical Data
🎯 What it does: A 'marginally recoverable' family of distributions is proposed to construct graphical models compatible with various data types (continuous, count, binary, etc.), and to avoid high-dimensional integration through maximum marginal likelihood estimation (MMLE) in high dimensions, further extending to mixed models for heterogeneous structure inference.
A Geometric Approach to Personalized Recommendation with Set-Theoretic Constraints Using Box Embeddings
Shib Sankar Dasgupta (University of Massachusetts Amherst), Andrew McCallum (University of Massachusetts Amherst)
Recommendation SystemTabular
🎯 What it does: This paper proposes a geometric method based on box embeddings to address personalized recommendation problems with set-theoretic constraints.
A Hitchhiker's Guide to Scaling Law Estimation
Leshem Choshen (Massachusetts Institute of Technology), Jacob Andreas (Massachusetts Institute of Technology)
TransformerLarge Language ModelText
🎯 What it does: This paper studies the estimation methods for the scaling laws of large language models, constructs and publicly releases a scaling law dataset containing 485 pre-trained models, and systematically evaluates the sources of estimation errors based on this dataset, proposing a set of practical best practices.
A Large Recurrent Action Model: xLSTM enables Fast Inference for Robotics Tasks
Thomas Schmied (Johannes Kepler University Linz), Sepp Hochreiter (Johannes Kepler University Linz)
Robotic IntelligenceRecurrent Neural NetworkReinforcement LearningMultimodalitySequential
🎯 What it does: A Large Recursive Action Model (LRAM) is proposed, centered around xLSTM, to achieve rapid inference in robotic reinforcement learning tasks.
A Lens into Interpretable Transformer Mistakes via Semantic Dependency
Ruo-Jing Dong (University of Sydney), Tongliang Liu (University of Sydney)
Explainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: This paper studies the role of semantic dependencies in transformer models for question answering, revealing the reasons for errors in the model's information generation by analyzing the impact of semantic changes on token values.
A Likelihood Based Approach to Distribution Regression Using Conditional Deep Generative Models
Shivam Kumar (University of Notre Dame), Lizhen Lin (University of Maryland)
Auto EncoderTabular
🎯 What it does: A likelihood-based conditional deep generative model for distribution regression is proposed, along with its convergence rate theory under Hellinger and Wasserstein distances.
A Machine Learning Approach to Duality in Statistical Physics
Prateek Gupta (Max Planck Institute for Human Development), Nabil Iqbal (Amsterdam Machine Learning Laboratory)
OptimizationRepresentation LearningData-Centric LearningTabularPhysics Related
🎯 What it does: Using neural networks and loss functions to automatically discover dual relationships in statistical physics models, we reproduced the Kramers-Wannier duality of the two-dimensional Ising model and explored equivalent forms with four-body interactions.
A Manifold Perspective on the Statistical Generalization of Graph Neural Networks
Zhiyang Wang (University of Pennsylvania), Alejandro Ribeiro (University of Pennsylvania)
Graph Neural NetworkGraph
🎯 What it does: This paper derives the statistical generalization theory of graph neural networks implemented on graphs obtained from manifold sampling from a manifold perspective, and provides upper bounds on generalization error related to the number of nodes, manifold dimension, and filter smoothness.
A Market for Accuracy: Classification Under Competition
Ohad Einav (Technion Israel Institute of Technology), Nir Rosenfeld (Technion Israel Institute of Technology)
ClassificationOptimizationTabular
🎯 What it does: This paper studies classification tasks in a competitive environment, proposing to transform the maximization of market share into a weighted accuracy objective, and analyzes the best response dynamics and their convergence;
A Mathematical Framework for AI-Human Integration in Work
L. Elisa Celis (Yale University), Nisheeth K. Vishnoi (Nanjing University)
Large Language ModelText
🎯 What it does: A mathematical framework has been proposed and implemented to evaluate the collaborative effectiveness of humans and generative artificial intelligence (GenAI). The framework decomposes work, workers, and their skills into decision-level and execution-level sub-skills, and defines the probability of work success through this framework. It further clarifies the theoretical mechanisms of skill enhancement, worker merging, and the productivity compression effect of generative AI.
A Memory Efficient Randomized Subspace Optimization Method for Training Large Language Models
Yiming Chen (Peking University), Zaiwen Wen (Peking University)
OptimizationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: A Random Subspace Optimization (RSO) framework is proposed for the pre-training and fine-tuning of LLMs, significantly reducing the memory usage of the optimizer's state and activations by decomposing high-dimensional training problems into low-dimensional subproblems.
A Meta-learner for Heterogeneous Effects in Difference-in-Differences
Hui Lan (Stanford University), Vasilis Syrgkanis (Stanford University)
Meta LearningTabular
🎯 What it does: An algorithm based on a double robust Meta-learner is proposed to estimate heterogeneous treatment effects (CATT) within the difference-in-differences (DiD) framework.
A Mixed-Curvature based Pre-training Paradigm for Multi-Task Vehicle Routing Solver
Suyu Liu (Nanyang Technological University), Yew-Soon Ong (Agency for Science Technology and Research)
OptimizationReinforcement LearningTabular
🎯 What it does: A multi-task vehicle routing solver pre-trained in mixed curvature space is proposed, utilizing hyperplanes, Euclidean, and planar curvature to extract the geometric structures of different tasks.
A Mixture-Based Framework for Guiding Diffusion Models
Yazid Janati (Ecole polytechnique), Jimmy Olsson (KTH Royal Institute of Technology)
RestorationGenerationData SynthesisDiffusion modelImageAudio
🎯 What it does: A framework based on mixed distributions is proposed, which approximates the intermediate posterior distribution in inference problems through mixing and uses Gibbs sampling for sampling.
A Model of Place Field Reorganization During Reward Maximization
M Ganesh Kumar (Harvard University), Cengiz Pehlevan (Harvard University)
OptimizationReinforcement LearningSequential
🎯 What it does: This paper constructs a normalized framework aimed at maximizing rewards, directly connecting adjustable Gaussian radial basis function (RBF) position fields to an actor-critic network. It utilizes TD error to online update the amplitude, center, and width of the position fields, achieving adaptive reorganization of spatial representations and explaining experimental phenomena such as high density at rewards, field stretching, and drift.
A Multi-Region Brain Model to Elucidate the Role of Hippocampus in Spatially Embedded Decision-Making
Yi Xie (Princeton University), Ila R Fiete
Recurrent Neural NetworkReinforcement Learning
🎯 What it does: This paper proposes a multi-region brain model that combines the entorhinal-hippocampal structure (Vector-HaSH+) with a cortical RNN to simulate the spatial decision-making process of mice in an accumulation tower task.
A Near Linear Query Lower Bound for Submodular Maximization
Binghui Peng (Stanford University), Aviad Rubinstein (Stanford University)
Optimization
🎯 What it does: This paper re-examines the problem of selecting k elements to optimize a target function and explores whether this problem can be approximately solved under sublinear query complexity.
A Near-Optimal Single-Loop Stochastic Algorithm for Convex Finite-Sum Coupled Compositional Optimization
Bokun Wang (Texas A&M University), Tianbao Yang (Texas A&M University)
OptimizationTabular
🎯 What it does: A single-loop, block coordinate-based stochastic primal-dual algorithm ALEXR is proposed to solve the convex finite and coupled composite optimization (cFCCO) problem.
A New Approach to Backtracking Counterfactual Explanations: A Unified Causal Framework for Efficient Model Interpretability
Pouria Fatemi (Technical University of Munich), Mohammad Hossein Yassaee (Sharif University of Technology)
Explainability and InterpretabilityComputational EfficiencyTabularFinance Related
🎯 What it does: The BRACE method is proposed, which utilizes backward counterfactual reasoning combined with interpretability optimization to generate counterfactual explanations that are both consistent with causal structures and executable.
A New Concentration Inequality for Sampling Without Replacement and Its Application for Transductive Learning
Yingzhen Yang (Arizona State University)
🎯 What it does: This paper proposes and utilizes the Transductive Local Complexity (TLC) tool to derive concentration inequalities for the test-training process of sampling without replacement, and applies it to the upper bound of generalization error in transfer learning.
A Non-Asymptotic Convergent Analysis for Scored-Based Graph Generative Model via a System of Stochastic Differential Equations
Junwei Su (University of Hong Kong), Chuan Wu (University of Hong Kong)
GenerationData SynthesisGraph Neural NetworkScore-based ModelGraphStochastic Differential Equation
🎯 What it does: This paper presents a non-asymptotic convergence analysis for the Score Generative Graph Model (SGGM), providing theoretical convergence upper bounds for three scenarios: node feature generation, graph structure generation, and joint generation.
A Non-isotropic Time Series Diffusion Model with Moving Average Transitions
Chenxi Wang (University of Hong Kong), Yi Wang (University of Hong Kong)
GenerationSuper ResolutionDiffusion modelTime Series
🎯 What it does: A non-isotropic time series diffusion model MA-TSD is proposed, utilizing moving average filtering as a forward denoising process, while the backward process can be accelerated and directly serves as time series super-resolution.
A Novel Characterization of the Population Area Under the Risk Coverage Curve (AURC) and Rates of Finite Sample Estimators
Han Zhou (KU Leuven), Matthew B. Blaschko (KU Leuven)
ClassificationOptimizationConvolutional Neural NetworkTransformerImageText
🎯 What it does: This paper presents the overall (population) definition of the evaluation metric AURC for selective classifiers (SC) and derives an equivalent weighted risk function expression from a statistical perspective.
A Parameter-Free and Near-Optimal Zeroth-Order Algorithm for Stochastic Convex Optimization
Kunjie Ren (Fudan University), Luo Luo (Fudan University)
OptimizationTabular
🎯 What it does: This paper proposes a parameter-free stochastic zeroth-order optimization algorithm called POEM, which can adaptively adjust the step size and smoothing parameter in convex stochastic optimization, achieving approximately optimal SZO (stochastic zeroth-order oracle) complexity.
A Parametric Contextual Online Learning Theory of Brokerage
François Bachoc (Universite Paul Sabatier), Roberto Colomboni (Politecnico di Milano)
Recommendation SystemAnomaly DetectionOptimizationReinforcement Learning from Human FeedbackTabularFinance Related
🎯 What it does: This paper studies the role of contextual information in the broker problem of online learning among traders, designs corresponding algorithms, and proves optimal theoretical regret guarantees under various standard assumptions.
A Peer-review Look on Multi-modal Clustering: An Information Bottleneck Realization Method
Zhengzheng Lou (Zhengzhou University), Shizhe Hu (Zhengzhou University)
OptimizationMultimodalityReview/Survey Paper
🎯 What it does: This paper proposes a trustworthy information bottleneck method based on the idea of peer review for weighted multimodal clustering.
A Physics-Augmented Deep Learning Framework for Classifying Single Molecule Force Spectroscopy Data
Cailong Hua (University of Minnesota), Murti Salapaka
ClassificationOptimizationConvolutional Neural NetworkRecurrent Neural NetworkTime SeriesSequentialPhysics Related
🎯 What it does: A dual-branch physics-enhanced deep learning model (PemNN) is proposed to automatically classify single-molecule force spectroscopy (SMFS) curves into three categories: 'no molecule', 'single molecule', and 'multiple molecules', significantly reducing the time required for manual screening.
A Physics-Informed Machine Learning Framework for Safe and Optimal Control of Autonomous Systems
Manan Tayal (Indian Institute of Science), Somil Bansal (Stanford University)
Autonomous DrivingOptimizationSafty and PrivacyReinforcement LearningTime SeriesPhysics Related
🎯 What it does: A physical information-based machine learning framework is proposed, which unifies safety constraints and performance objectives into a state-constrained optimal control problem, and utilizes deep networks to approximate its Hamilton–Jacobi–Bellman equation, resulting in a control strategy that satisfies both safety and performance.
A Reasoning-Based Approach to Cryptic Crossword Clue Solving
Martin Andrews (Red Dragon AI), Sam Witteveen (Red Dragon AI)
Large Language ModelSupervised Fine-TuningText
🎯 What it does: A reasoning system based on large language models has been designed to automatically solve cryptic crosswords; the system first generates candidate answers, then produces multiple semantic explanations for each candidate, and subsequently transforms the explanations into Python DSL and verifies them through AST to achieve interpretable proofs.
A Recipe for Causal Graph Regression: Confounding Effects Revisited
Yujia Yin (Hong Kong Baptist University), Yifan Chen (Hong Kong Baptist University)
Graph Neural NetworkContrastive LearningGraph
🎯 What it does: A causal graph regression method for graph regression tasks is proposed, rethinking the treatment of confounding effects and introducing enhanced graph information bottlenecks and causal interventions through contrastive learning.
A Reduction Framework for Distributionally Robust Reinforcement Learning under Average Reward
Zachary Andrew Roch (University of Central Florida), Yue Wang (University of Central Florida)
Reinforcement Learning
🎯 What it does: The paper proposes a dimensionality reduction framework that transforms the robust average reward reinforcement learning problem into a robust discounted reward problem through a specific discount factor.
A Reductions Approach to Risk-Sensitive Reinforcement Learning with Optimized Certainty Equivalents
Kaiwen Wang (Cornell Tech), Wen Sun (Cornell Tech)
Reinforcement LearningTabular
🎯 What it does: Two types of meta-algorithms (optimistic and gradient) are proposed to transform static OCE risk reinforcement learning (RL) into conventional risk-neutral RL, solved in augmented MDPs, thus obtaining theoretical guarantees for global convergence, PAC, and local improvement, and experimentally validating the necessity of non-Markovian policies.
A Rescaling-Invariant Lipschitz Bound Based on Path-Metrics for Modern ReLU Network Parameterizations
Antoine Gonon (École Polytechnique Fédérale de Lausanne), Rémi Gribonval
OptimizationConvolutional Neural NetworkImage
🎯 What it does: A re-scalar invariant Lipschitz upper bound based on path metrics is proposed, which can analyze the robustness of the parameter space of modern ReLU DAG networks and design pruning criteria that are invariant to re-scaling.
A Sample Efficient Conditional Independence Test in the Presence of Discretization
Boyang Sun (Mohamed bin Zayed University of Artificial Intelligence), Kun Zhang (Carnegie Mellon University)
Recommendation SystemTabularFinance Related
🎯 What it does: A sample-efficient conditional independence test method for discretized data, DCT-GMM, is proposed.
A Selective Learning Method for Temporal Graph Continual Learning
Hanmo Liu (Hong Kong University of Science and Technology), Lei Chen (Hong Kong University of Science and Technology)
Graph Neural NetworkGraphTime Series
🎯 What it does: The paper proposes the Temporal Graph Continual Learning (TGCL) problem and designs the Learning Towards the Future (LTF) framework, which efficiently updates the temporal graph learning model without losing knowledge of old classes by utilizing subset selection and distribution alignment methods.
A Sharper Global Convergence Analysis for Average Reward Reinforcement Learning via an Actor-Critic Approach
Swetha Ganesh (Purdue University), Vaneet Aggarwal (Purdue University)
Reinforcement Learning
🎯 What it does: This paper proposes a Multi-Layer Monte Carlo based Natural Actor-Critic (MLMC-NAC) algorithm for the general policy parameterization problem of average reward infinite MDPs, and proves that its global convergence rate is ˜O(1/√T);
A Simple Model of Inference Scaling Laws
Noam Itzhak Levi (Ecole Polytechnique Federale de Lausanne)
Large Language ModelText
🎯 What it does: A memory-based statistical model is proposed to analyze the scaling law of large language models (LLMs) in multiple inference attempts (pass@k), and a power-law decay formula for the corresponding inference loss with respect to the number of trials or total computational cost is provided.
A Square Peg in a Square Hole: Meta-Expert for Long-Tailed Semi-Supervised Learning
Yaxin Hou (Southeast University), Yuheng Jia (Southeast University)
ClassificationMeta LearningMixture of ExpertsImage
🎯 What it does: This study focuses on long-tail semi-supervised learning and proposes the Meta-Expert algorithm, which achieves high-quality pseudo-labels and predictions through dynamic expert allocation and multi-depth feature fusion.
A Stronger Mixture of Low-Rank Experts for Fine-Tuning Foundation Models
Mengyang Sun (Tsinghua University), Jie Tang (Tsinghua University)
Supervised Fine-TuningMixture of ExpertsMultimodality
🎯 What it does: This paper proposes combining Riemannian preconditioners with Mixture-of-Experts Low-Rank Adaptation (MoE-LoRA) to improve gradient projection for enhanced training stability and performance.
A Sub-Problem Quantum Alternating Operator Ansatz for Correlation Clustering
Lucas Fabian Naumann (TU Dresden), Bjoern Andres (TU Dresden)
OptimizationGraph Neural NetworkGraph
🎯 What it does: Proposed and experimentally validated a variant of the Subproblem Quantum Alternating Operator (SQAOA) for the related clustering problem, improving solution quality and resource utilization efficiency through kernel sampling and subproblem decomposition.
A Tale of Two Structures: Do LLMs Capture the Fractal Complexity of Language?
Ibrahim Alabdulmohsin (Google Deepmind), Andreas Peter Steiner
TransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This study investigates whether large language models can replicate the fractal complexity of natural language and explores the impact of factors such as temperature settings and prompting methods.
A Theoretical Framework For Overfitting In Energy-based Modeling
Giovanni Catania (Universidad Complutense de Madrid), Beatriz Seoane (Universidad Complutense de Madrid)
Tabular
🎯 What it does: This paper constructs a theoretical framework for overfitting in energy-based models (EBM), particularly focusing on the training dynamics of high-dimensional Gaussian EBM and binary Ising-BM, revealing the time scale separation of covariance spectral components and the overfitting behavior caused by finite samples;
A Theoretical Justification for Asymmetric Actor-Critic Algorithms
Gaspard Lambrechts (University of Liège), Aditya Mahajan (McGill University)
Reinforcement Learning
🎯 What it does: This paper provides a finite-time convergence analysis for heterogeneous actor-critic algorithms in partially observable environments and demonstrates that it outperforms symmetric learning in the presence of agent state aliasing.
A Theoretical Study of (Hyper) Self-Attention through the Lens of Interactions: Representation, Training, Generalization
Muhammed Ustaomeroglu (Carnegie Mellon University), Guannan Qu (Carnegie Mellon University)
TransformerTextTime Series
🎯 What it does: The study investigates the representation, training, and generalization performance of self-attention in interaction models, and proposes two new modules: HyperFeatureAttention and HyperAttention.
A Theory for Conditional Generative Modeling on Multiple Data Sources
Rongzhen Wang (Renmin University of China), Guoqiang Wu (Shandong University)
GenerationData SynthesisDiffusion modelImage
🎯 What it does: The paper presents a theoretical analysis of multi-source conditional generative models, providing an upper bound on the average total variation (TV) error based on the upper bracketing number, and proving that under the conditions of similar source distributions and sufficiently expressive models, multi-source training can achieve better error bounds than single-source training.
A Trichotomy for List Transductive Online Learning
Steve Hanneke (Purdue University), Amirreza Shaeiri (Purdue University)
🎯 What it does: This paper studies the transferable online learning framework for lists, proving the tripartite limits of the error upper bounds (Θ(T), Θ(log T), Θ(1)) under realizable settings, and providing an upper bound on the reward ˜O(√T) under unrealizable settings.
A Two-Stage Learning-to-Defer Approach for Multi-Task Learning
Yannis Montreuil (National University of Singapore), Wei Tsang Ooi (National University of Singapore)
ClassificationObject DetectionBiomedical DataElectronic Health Records
🎯 What it does: A two-stage learning-delayed (L2D) framework is proposed for simultaneously handling multi-task learning for classification and regression.
A Unified Approach to Routing and Cascading for LLMs
Jasper Dekoninck (ETH Zurich), Martin Vechev (ETH Zurich)
OptimizationComputational EfficiencyTransformerLarge Language ModelMixture of ExpertsTextBenchmark
🎯 What it does: A unified model selection strategy is proposed, combining routing and cascading methods, referred to as cascading routing, aimed at improving the performance and cost-effectiveness of large language models (LLMs).
A Unified Comparative Study with Generalized Conformity Scores for Multi-Output Conformal Regression
Victor Dheur (University of Mons), Souhaib Ben Taieb (Mohamed bin Zayed University of Artificial Intelligence)
Flow-based ModelGenerative Adversarial NetworkImageTabular
🎯 What it does: The study investigates distribution-independent qualified prediction regions in multi-output regression, comparing nine methods and proposing two new types of consistency scores.
A Unified Framework for Entropy Search and Expected Improvement in Bayesian Optimization
Nuojin Cheng (University of Colorado Boulder), Luigi Nardi (Lund University)
OptimizationTabular
🎯 What it does: Proposes the Variational Entropy Search (VES) framework, unifying Expected Improvement (EI) with information-theoretic sampling functions (such as Max-Value Entropy Search, MES), and designs the VES-Gamma sampling function based on this framework;
A Unified Framework for Generalization Error Analysis of Learning with Arbitrary Discrete Weak Features
Kosuke Sugiyama (Waseda University), Masato Uchida (Waseda University)
OptimizationTabular
🎯 What it does: A unified framework for Discrete Weak Features Learning (WFL) is proposed, along with its theoretical form for risk minimization; under this framework, the LAC-dWFL learning algorithm class is defined, and a generalization error analysis is conducted, revealing the interaction between the feature estimation model g and the label prediction model f.
A Unified Theoretical Analysis of Private and Robust Offline Alignment: from RLHF to DPO
Xingyu Zhou (Wayne State University), Francesco Orabona (King Abdullah University of Science and Technology)
Safty and PrivacyReinforcement Learning from Human FeedbackReinforcement LearningTabular
🎯 What it does: This paper presents a unified theoretical analysis of offline alignment (RLHF and DPO) in the presence of both privacy protection and label corruption, proving the performance differences under different privacy-corruption orders (CTL and LTC), and providing corresponding sub-optimality upper bounds.
A Unified View on Learning Unnormalized Distributions via Noise-Contrastive Estimation
Jongha Jon Ryu (Massachusetts Institute of Technology), Gregory W. Wornell (Massachusetts Institute of Technology)
Contrastive Learning
🎯 What it does: This paper proposes a unified framework based on Noise Contrastive Estimation (NCE) for learning unnormalized (energy) distributions, further introducing two variants: α-centered NCE and f-conditioned NCE, and establishes their correspondence with existing methods (MLE, MC-MLE, GlobalGISO, pseudo-likelihood, etc.).
A Variational Framework for Improving Naturalness in Generative Spoken Language Models
Li-Wei Chen (Carnegie Mellon University), Alexander Rudnicky (Carnegie Mellon University)
GenerationDiffusion modelAuto EncoderAudio
🎯 What it does: This paper proposes an end-to-end framework that combines variational autoencoders and speech language models, using learned continuous features to supplement semantic discrete words, thereby enhancing the naturalness and coherence of speech generation.
A Variational Information Theoretic Approach to Out-of-Distribution Detection
Sudeepta Mondal (RTX Technology Research Center), Ganesh Sundaramoorthi (RTX Technology Research Center)
Anomaly DetectionTransformerImageOrdinary Differential Equation
🎯 What it does: A framework based on variational information theory is proposed, utilizing KL divergence and information bottleneck regularization to design random OOV features, and the optimal features are obtained through differential equation solving;
A Variational Perspective on Generative Protein Fitness Optimization
Lea Bogensperger (University of Zurich), Michael Krauthammer (University of Zurich)
OptimizationDrug DiscoveryFlow-based ModelAuto EncoderBiomedical Data
🎯 What it does: This paper proposes a variational inference-based protein adaptability optimization framework called VLGPO, which can perform posterior sampling of protein sequences in a continuous latent space to generate highly adaptive sequences.
A Versatile Influence Function for Data Attribution with Non-Decomposable Loss
Junwei Deng (University of Illinois Urbana-Champaign), Jiaqi W. Ma (University of Illinois Urbana-Champaign)
GraphTabular
🎯 What it does: A general influence function method is studied for training data attribution under non-decomposable loss.
A-PSRO: A Unified Strategy Learning Method with Advantage Metric for Normal-form Games
Yudong Hu (University of Chinese Academy of Sciences), Mingqiang Li (Information Science Academy, China Electronics Technology Group Corporation)
OptimizationReinforcement LearningTabular
🎯 What it does: This paper presents A-PSRO, a unified open-source learning framework, and introduces the advantage function as a metric for policy evaluation.
AAAR-1.0: Assessing AI’s Potential to Assist Research
Renze Lou (Pennsylvania State University), Wenpeng Yin
TransformerLarge Language ModelTextBenchmark
🎯 What it does: The AAAR-1.0 benchmark dataset is proposed to evaluate the performance of large language models on three core research tasks (equation reasoning, experimental design, and paper weakness discovery).
Ab Initio Nonparametric Variable Selection for Scalable Symbolic Regression with Large $p$
Shengbin Ye (Rice University), Meng Li (Northwestern University)
TabularPhysics Related
🎯 What it does: A PAN+SR framework is proposed, combining BART-based non-parametric variable selection with symbolic regression to achieve scalable symbolic regression on large feature sets.
ABKD: Pursuing a Proper Allocation of the Probability Mass in Knowledge Distillation via $\alpha$-$\beta$-Divergence
Guanghui Wang (University of Chinese Academy of Sciences), Qingming Huang (University of Chinese Academy of Sciences)
Knowledge DistillationImageText
🎯 What it does: A knowledge distillation framework based on α-β divergence, ABKD, is proposed, which can balance the probability mass distribution between forward KL and reverse KL.
ABNet: Adaptive explicit-Barrier Net for Safe and Scalable Robot Learning
Wei Xiao (Massachusetts Institute of Technology), Daniela Rus (Massachusetts Institute of Technology)
Autonomous DrivingRobotic IntelligenceReinforcement LearningMultimodality
🎯 What it does: This paper proposes the Adaptive explicit-Barrier Net (ABNet), which achieves safe integration and learning for robot control through multi-head explicit barrier modules.
Accelerated Diffusion Models via Speculative Sampling
Valentin De Bortoli (Google DeepMind), Arnaud Doucet (Google DeepMind)
GenerationRobotic IntelligenceDiffusion modelImage
🎯 What it does: A method is proposed to transfer Speculative Sampling technology to diffusion models (DDM) to accelerate the sampling of high-quality diffusion models.
Accelerating Large Language Model Reasoning via Speculative Search
Zhihai Wang (University of Science and Technology of China), Feng Wu (Huawei Technologies)
OptimizationComputational EfficiencyTransformerLarge Language ModelTextChain-of-Thought
🎯 What it does: Proposes the SpecSearch framework, which allows small models and large models to collaboratively generate thoughts at two levels: thought and token, significantly accelerating the tree reasoning of LLMs.
Accelerating Linear Recurrent Neural Networks for the Edge with Unstructured Sparsity
Alessandro Pierro (Intel Corporation), Sumit Bam Shrestha (Intel Corporation)
RestorationComputational EfficiencyRecurrent Neural NetworkAudio
🎯 What it does: This study investigates the use of unstructured sparsification and fixed-point quantization to accelerate linear recurrent neural networks (S5) on edge devices, and implements real-time audio denoising inference on the Intel Loihi 2 neuromorphic chip.