ICML 2025 Papers — Page 30
International Conference on Machine Learning · 3257 papers
The Global Convergence Time of Stochastic Gradient Descent in Non-Convex Landscapes: Sharp Estimates via Large Deviations
Waïss Azizian (Grenoble Alpes University), Panayotis Mertikopoulos (Grenoble Alpes University)
OptimizationStochastic Differential Equation
🎯 What it does: This paper provides precise upper and lower bounds on the time required for stochastic gradient descent (SGD) to reach global optimality on non-convex loss surfaces, using large deviation theory and stochastic perturbation dynamics.
The Harder Path: Last Iterate Convergence for Uncoupled Learning in Zero-Sum Games with Bandit Feedback
Côme Fiegel (ENSAE Paris - Centre de Recherche en Économie et Statistique), Vianney Perchet (Criteo AI Lab)
Optimization
🎯 What it does: This paper studies the use of uncoupled learning in zero-sum matrix games to achieve last-iterate convergence under bandit feedback, proving a tighter lower bound and proposing two algorithms that reach this lower bound.
The Hidden Dimensions of LLM Alignment: A Multi-Dimensional Analysis of Orthogonal Safety Directions
Wenbo Pan (City University of Hong Kong), Xiaohua Jia (City University of Hong Kong)
Safty and PrivacyAdversarial AttackLarge Language ModelText
🎯 What it does: The study investigates the multidimensional mechanisms of safety alignment in large language models, finding that safe behavior is jointly controlled by a dominant direction and multiple secondary directions, and reveals how secondary directions influence the dominant direction and escape attacks.
The Hidden Joules: Evaluating the Energy Consumption of Vision Backbones for Progress Towards More Efficient Model Inference
Zeyu Yang (University of Oxford), Wesley Armour (University of Oxford)
Computational EfficiencyConvolutional Neural NetworkImage
🎯 What it does: A systematic evaluation of the inference energy consumption of 1200 ImageNet classification models
The Hidden Life of Tokens: Reducing Hallucination of Large Vision-Language Models Via Visual Information Steering
Zhuowei Li (Rutgers University), Dimitris N. Metaxas (Rutgers University)
RecognitionGenerationTransformerVision Language ModelImageTextMultimodality
🎯 What it does: This study investigates the hallucination mechanisms in the generation process of large visual language models (LVLM) and proposes VISTA, which reduces hallucinations and enhances visual authenticity during the inference phase through two untrained intervention methods: Visual Signal Vector (VSV) and Self-Logit Augmentation (SLA).
The Illusion of Role Separation: Hidden Shortcuts in LLM Role Learning (and How to Fix Them)
Zihao Wang (University of Chicago), Heqing Huang (ByteDance Inc.)
GenerationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper studies the role separation problem of large language models (LLMs) under multi-role (system, user, tool, etc.) inputs, finding that models often utilize shortcut strategies such as task type and text starting position, making it difficult to truly distinguish roles.
The Impact of On-Policy Parallelized Data Collection on Deep Reinforcement Learning Networks
Walter Mayor (Independent Researcher), Pablo Samuel Castro (Google DeepMind)
Reinforcement LearningVideoBenchmark
🎯 What it does: The study investigates the effects of using more parallel environments (N envs) and shorter replay lengths (N RO) for data collection in on-policy reinforcement learning methods like PPO, exploring their impact on performance, representation learning, plasticity, and optimization stability.
The impact of uncertainty on regularized learning in games
Pierre-Louis Cauvin (Univ. Grenoble Alpes), Panayotis Mertikopoulos (Univ. Grenoble Alpes)
OptimizationStochastic Differential Equation
🎯 What it does: This paper studies the long-term behavior of Follow-The-Regularized-Leader (FTRL) learning dynamics in the presence of random noise and uncertainty, proving that noise drives players towards extreme pure strategies and leads to the limit being only pure Nash equilibria.
The Importance of Being Lazy: Scaling Limits of Continual Learning
Jacopo Graldi (ETH Zurich), Lorenzo Noci (ETH Zurich)
OptimizationRepresentation LearningImage
🎯 What it does: The system studies the impact of model scale and feature learning on catastrophic forgetting in continual learning, and conducts theoretical analysis using mean field theory in the infinite width limit.
The Jailbreak Tax: How Useful are Your Jailbreak Outputs?
Kristina Nikolić (ETH Zurich), Florian Tramèr (ETH Zurich)
Large Language ModelSupervised Fine-TuningPrompt EngineeringTextBiomedical DataBenchmark
🎯 What it does: The study quantifies the phenomenon of performance degradation in large language models after being jailbroken, introducing the concept of 'jailbreak tax' and constructing an assessable benchmark.
The Limits of Predicting Agents from Behaviour
Alexis Bellot (Google DeepMind), Tom Everitt (Google DeepMind)
World Model
🎯 What it does: This paper derives the theoretical limits and identifiable intervals of AI decision-making in new environments based on the assumption that AI behavior is driven by an unknown world model, providing precise boundaries for the predictability of AI's future behavior based on behavioral data.
The Limits of Tractable Marginalization
Oliver Broadrick (University of California), Markus Bläser
OptimizationComputational Efficiency
🎯 What it does: This paper analyzes the computational boundaries of marginalization problems, proving the existence of functions that are marginalizable but cannot be efficiently represented by known solvable models (UFMAC). It also explores stronger forms of marginalization such as Hamiltonian weights and virtual evidence marginalization, ultimately proving that UFMAC is a complete model for virtual evidence marginalization under the real RAM model.
The Lock-in Hypothesis: Stagnation by Algorithm
Tianyi Qiu, Max Kleiman-Weiner (University of Washington)
TransformerLarge Language ModelAgentic AIText
🎯 What it does: This study investigates and verifies the 'lock-in hypothesis' resulting from human interactions with large language models (LLMs), demonstrating through theoretical models, agent simulations, and analysis of real data from WildChat that LLM feedback loops lead to a decrease in viewpoint diversity.
The Logical Implication Steering Method for Conditional Interventions on Transformer Generation
Damjan Kalajdzievski (Salesforce)
GenerationExplainability and InterpretabilityTransformerPrompt EngineeringText
🎯 What it does: Proposes the Logical Implication Model Steering (LIMS), a method to embed logical implication circuits within pre-trained Transformers, enabling interpretable conditional control over generation behavior.
The Missing Alignment Link of In-context Learning on Sequences
Harshvardhan Agarwal (Indian Institute of Technology Bombay), Sunita Sarawagi (Indian Institute of Technology Bombay)
TransformerLarge Language ModelSupervised Fine-TuningSequential
🎯 What it does: This study investigates the contextual learning ability of large language models in sequence-to-sequence tasks and finds that the lack of input-output alignment leads to performance degradation; it proposes ICATune—a method for alignment learning that involves fine-tuning a small number of parameters in the attention layer.
The Noisy Laplacian: a Threshold Phenomenon for Non-Linear Dimension Reduction
Alex Kokot (University of Washington), Marina Meila (University of Washington)
Diffusion modelSequentialPhysics Related
🎯 What it does: This paper studies the impact of noise on spectral dimensionality reduction methods such as Diffusion Maps, and proves that under a fixed noise amplitude, the low-frequency spectrum can still reliably capture the underlying manifold geometry, forming a threshold phenomenon.
The Number of Trials Matters in Infinite-Horizon General-Utility Markov Decision Processes
Pedro Pinto Santos, Francisco S. Melo
Reinforcement Learning
🎯 What it does: This paper studies the impact of the number of trials on performance in the infinite-horizon General Utility Markov Decision Process (GUMDP), proving that the number of trials does not match the policy evaluation results and providing upper and lower bounds;
The Panaceas for Improving Low-Rank Decomposition in Communication-Efficient Federated Learning
Shiwei Li (Huazhong University of Science and Technology), Ruixuan Li (Huazhong University of Science and Technology)
OptimizationFederated LearningImage
🎯 What it does: An improved method for low-rank decomposition in communication-efficient federated learning is proposed, aimed at reducing communication overhead and enhancing model convergence speed and accuracy.
The Perils of Optimizing Learned Reward Functions: Low Training Error Does Not Guarantee Low Regret
Lukas Fluri (ETH Zurich), Joar Max Viktor Skalse (Oxford University)
OptimizationReinforcement Learning
🎯 What it does: This paper conducts a rigorous theoretical analysis of the relationship between training error during the reward function learning process and the final policy regret, revealing the existing error-regret mismatch phenomenon.
The Polynomial Stein Discrepancy for Assessing Moment Convergence
Narayan Srinivasan (Queensland University of Technology), Leah F South
🎯 What it does: A polynomial Stein discrepancy (PSD) with linear time complexity is proposed to assess the moment convergence of Bayesian sampling and conduct power tests.
The Power of Random Features and the Limits of Distribution-Free Gradient Descent
Ari Karchmer (Harvard University), Eran Malach (Harvard University)
Optimization
🎯 What it does: This paper studies the relationship between distribution-independent gradient descent and random feature learning, and proves that if a differentiable model can be learned through distribution-independent mini-batch stochastic gradient descent, then the objective function can almost certainly be approximated by a polynomial-sized linear combination of random features.
The Price of Freedom: Exploring Expressivity and Runtime Tradeoffs in Equivariant Tensor Products
YuQing Xie (Massachusetts Institute of Technology), Tess Smidt (Massachusetts Institute of Technology)
Mesh
🎯 What it does: This paper conducts a systematic analysis of the tensor product operation in E(3) equivariant networks, evaluating its expressiveness, interactivity, and runtime cost, and proposes a more concise and efficient spherical mesh implementation.
The Price of Linear Time: Error Analysis of Structured Kernel Interpolation
Alexander Moreno (MBZUAI), Jonathan Mei (IonQ)
Tabular
🎯 What it does: Error analysis of Structured Kernel Interpolation (SKI) is presented, providing bounds for the Gram matrix, hyperparameter estimation, and posterior inference errors, along with guidelines for selecting the number of inducing points.
The Relationship Between No-Regret Learning and Online Conformal Prediction
Ramya Ramalingam (University of Pennsylvania), Aaron Roth (University of Pennsylvania)
OptimizationAdversarial AttackReinforcement LearningTabularTime Series
🎯 What it does: This paper explores the relationship between regret learning and online conforming prediction, particularly how to ensure marginal coverage in adversarial environments. The authors observe that standard regret guarantees can derive marginal coverage in independent and identically distributed (i.i.d.) environments, but this connection fails in adversarial settings or when group conditional coverage is required. Through analysis, the authors demonstrate a close relationship between threshold-calibrated coverage and exchange regret in adversarial environments, and propose a new algorithm to achieve group conditional coverage guarantees.
The Ripple Effect: On Unforeseen Complications of Backdoor Attacks
Rui Zhang (University of Electronic Science and Technology of China), Yang Zhang (CISPA Helmholtz Center for Information Security)
ClassificationAdversarial AttackTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: A quantitative study of backdoor attacks in pre-trained language models is conducted, assessing their unintended effects on non-target downstream tasks and proposing mitigation strategies.
The Role of Randomness in Stability
Max Hopkins (Princeton University), Shay Moran (Technion)
OptimizationSafty and Privacy
🎯 What it does: This study investigates the randomness complexity of stability (replicability and differential privacy) in learning and statistics, proving its near equivalence to global stability (the best replication rate without randomized algorithms), and establishes the correspondence between the randomness complexity of PAC learning and the Littlestone dimension.
The Role of Sparsity for Length Generalization in LLMs
Noah Golowich (Massachusetts Institute of Technology), Eran Malach (Harvard University)
TransformerLarge Language ModelText
🎯 What it does: This paper studies the generalization ability of large language models when predicting sequences that exceed the training context length, proposing that sparse dependency (tokens are only related to a fixed number k of previous tokens) is a core condition for achieving length generalization. It further provides a theoretical framework, proofs, and improvements for positional encoding.
The Sample Complexity of Online Strategic Decision Making with Information Asymmetry and Knowledge Transportability
Jiachen Hu (Peking University), Zhuoran Yang (Yale University)
OptimizationReinforcement Learning
🎯 What it does: An online strategic interaction model is proposed to address the principal-agent decision problem under information asymmetry and knowledge transfer. A model learning algorithm based on NPIV is designed, and it is proven that an ε-optimal strategy can be learned with a sample complexity of O~(1/ε²) in the target environment.
The Sharpness Disparity Principle in Transformers for Accelerating Language Model Pre-Training
Jinbo Wang (Peking University), Lei Wu (Peking University)
OptimizationComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: This paper analyzes the curvature (sharpness) of various modules in the Transformer and finds significant differences, proposing a Blockwise Learning Rate (Blockwise LR) strategy based on these differences to accelerate the pre-training of large-scale language models.
The Sparse-Plus-Low-Rank Quasi-Newton Method for Entropic-Regularized Optimal Transport
Chenrui Wang (Shanghai University of Finance and Economics), Yixuan Qiu (Shanghai University of Finance and Economics)
OptimizationImage
🎯 What it does: A sparse + low-rank approximation quasi-Newton method is proposed for solving large-scale entropy-regularized optimal transport problems.
The Surprising Agreement Between Convex Optimization Theory and Learning-Rate Scheduling for Large Model Training
Fabian Schaipp (Inria), Francis Bach (Inria)
OptimizationLarge Language ModelImageText
🎯 What it does: This study investigates learning rate scheduling in large-scale model training, demonstrating that its behavior is highly consistent with the suboptimality upper bounds in non-smooth convex optimization theory, and utilizes this theory to guide practical scheduling design.
The Surprising Effectiveness of Test-Time Training for Few-Shot Learning
Ekin Akyürek, Jacob Andreas (Massachusetts Institute of Technology)
OptimizationMeta LearningTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Significantly improved few-shot learning performance by performing temporary parameter updates on the language model during inference (Test-Time Training, TTT).
The Synergy of LLMs & RL Unlocks Offline Learning of Generalizable Language-Conditioned Policies with Low-fidelity Data
Thomas Pouplin (University of Cambridge), Mihaela van der Schaar (University of Cambridge)
Robotic IntelligenceTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
🎯 What it does: This paper proposes the TEDUO training pipeline, which automates backward labeling, state abstraction, offline RL learning, and LLM supervised fine-tuning through LLM, enabling the learning of generalizable language-conditioned policies from unlabeled low-quality offline data.
The underlying structures of self-attention: symmetry, directionality, and emergent dynamics in Transformer training
Matteo Saponati (Institute of Neuroinformatics, ETH Zurich and University of Zurich), Benjamin F Grewe
TransformerLarge Language ModelTextMultimodalityAudio
🎯 What it does: This paper constructs a mathematical framework by equating the self-attention matrix $W_{qk}=W_qW_k^T$ with a bilinear form, revealing that autoregressive training produces a column-dominant directional structure, while bidirectional training generates a symmetric structure. It is validated on various Transformers (BERT, GPT, LLaMA, Mistral, etc.) and multimodal data, further demonstrating that symmetric initialization of $W_{qk}$ in encoder-only models can significantly accelerate convergence and improve final loss.
The Underlying Universal Statistical Structure of Natural Datasets
Noam Itzhak Levi (École Polytechnique Fédérale de Lausanne), Yaron Oz (Raymond and Beverly Sackler School of Physics and Astronomy)
Data SynthesisImage
🎯 What it does: This study investigates the feature-feature covariance matrix of real-world and synthetic datasets, analyzing its spectral distribution, power law scaling, and statistical properties in random matrix theory (RMT), and proposes a correlated Gaussian data model (CGD) that can explain the observed spectrum.
The Value of Prediction in Identifying the Worst-Off
Unai Fischer-Abaigar (University of Munich), Juan Carlos Perdomo (Harvard University)
Tabular
🎯 What it does: This paper studies the value of using predictive models to identify the most vulnerable groups in social welfare distribution and compares it with the benefits of expanding screening resources.
Theoretical guarantees on the best-of-n alignment policy
Ahmad Beirami (Google DeepMind), Ananda Theertha Suresh (Google Research)
TransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Theoretical research on the best-of-n alignment strategy for generative language models is conducted, deriving its probability mass function, providing upper and lower bounds for KL divergence and win rate, proving that the commonly used KL formula is only an upper bound, proposing a more compact KL estimator, and comparing best-of-n with another rejection sampling method—rewind-and-repeat.
Theoretical Limitations of Ensembles in the Age of Overparameterization
Niclas Dern (Technical University of Munich), Geoff Pleiss (University of British Columbia)
Tabular
🎯 What it does: The study investigates the ensemble method of random feature regressors (RF) under over-parameterization conditions and demonstrates its equivalence to a single infinite-width model.
Theoretical Performance Guarantees for Partial Domain Adaptation via Partial Optimal Transport
Jayadev Naram (Chalmers University of Technology), Giuseppe Durisi (Chalmers University of Technology)
Domain AdaptationImage
🎯 What it does: This paper derives a generalization upper bound for the partial domain adaptation (PDA) problem based on partial optimal transport and proposes a theoretically guided weight allocation scheme and the corresponding learning algorithm WARMPOT.
Theoretically Unmasking Inference Attacks Against LDP-Protected Clients in Federated Vision Models
Quan Minh Nguyen, My T. Thai (University of Florida)
Federated LearningSafty and PrivacyTransformerGenerative Adversarial NetworkImage
🎯 What it does: This study investigates that in federated visual models, even with local differential privacy (LDP) protection, malicious servers can leak client data through membership inference attacks in low polynomial time.
Thermalizer: Stable autoregressive neural emulation of spatiotemporal chaos
Christian Pedersen (New York University), Joan Bruna (New York University)
Diffusion modelTime SeriesSequential
🎯 What it does: An algorithm named Thermalizer is proposed to stabilize autoregressive neural simulators to address the prediction instability of spatiotemporal chaotic systems.
Thickness-aware E(3)-Equivariant 3D Mesh Neural Networks
Sungwon Kim (KAIST), Chanyoung Park (KAIST)
Graph Neural NetworkAuto EncoderMesh
🎯 What it does: A thickness-aware E(3)-equivariant 3D mesh neural network (T-EMNN) is proposed, which captures thickness-related interactions while maintaining computational efficiency, and utilizes a data-driven coordinate system to achieve E(3)-equivariant or invariant spatial information representation.
Think Smarter not Harder: Adaptive Reasoning with Inference Aware Optimization
Zishun Yu (University of Illinois Chicago), Han Fang (MetaAI)
OptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningTextChain-of-Thought
🎯 What it does: This paper proposes Inference Budget-Constrained Policy Optimization (IBPO), which introduces budget constraints in LLMs to achieve adaptive allocation of inference length.
Think Twice, Act Once: A Co-Evolution Framework of LLM and RL for Large-Scale Decision Making
Xu Wan (Zhejiang University), Mingyang Sun (Peking University)
OptimizationTransformerLarge Language ModelReinforcement LearningAgentic AISequential
🎯 What it does: Proposes the ACE (Agents Co-Evolution) framework, which allows the LLM to act as both a policy Actor and a value Critic during the offline training phase to refine RL trajectories, thereby improving the learning efficiency and effectiveness of large-scale industrial decision-making.
Thinking LLMs: General Instruction Following with Thought Generation
Tianhao Wu (Meta), Sainbayar Sukhbaatar (Meta)
TransformerLarge Language ModelReinforcement LearningPrompt EngineeringText
🎯 What it does: A training method is proposed that enables existing LLMs to generate internal thoughts before answering, thereby enhancing their ability to follow general instructions.
Three-Dimensional Trajectory Prediction with 3DMoTraj Dataset
Hao Zhou (Great Bay University), Fei Luo (Great Bay University)
Autonomous DrivingOptimizationRecurrent Neural NetworkAuto EncoderPoint CloudTime Series
🎯 What it does: This study investigates 3D trajectory prediction, proposing the 3DMoTraj dataset and a prediction framework that decouples the three axes and refines associations.
Tight and Fast Bounds for Multi-Label Learning
Yi-Fan Zhang (Southeast University), Min-Ling Zhang (Southeast University)
ClassificationOptimization
🎯 What it does: A theoretical framework for multi-label learning is proposed, utilizing the smoothness of the smooth base loss function to derive a label-independent and faster convergence rate generalization upper bound through a new vector contraction inequality and local Rademacher complexity, and this method is extended to the analysis of macro-average AUC.
Tightening Causal Bounds via Covariate-Aware Optimal Transport
Sirui Lin (Stanford University), Peter Glynn (Stanford University)
OptimizationTabular
🎯 What it does: An optimal transport framework based on mirror covariates is proposed, which relaxes the conditional optimal transport problem to an unconditional OT and retains covariate information through a penalty term, thereby obtaining a tighter partially identified interval for causal effects.
Tilted Sharpness-Aware Minimization
Tian Li (University of Chicago), Jeff Bilmes
OptimizationTransformerSupervised Fine-TuningImageText
🎯 What it does: Proposed the Tilted Sharpness-Aware Minimization (TSAM), which enhances model generalization by exponentially tilting and reweighting multiple local optima in the neighborhood.
Time Series Representations with Hard-Coded Invariances
Thibaut Germain (Universite Paris-Saclay), Laurent Oudre (Universite Paris-Saclay)
Convolutional Neural NetworkContrastive LearningTime Series
🎯 What it does: A hard-coded invariant convolution layer is proposed, achieving invariance to amplitude scaling, offset, and smoothing trends in time series modeling.
Time to Spike? Understanding the Representational Power of Spiking Neural Networks in Discrete Time
Duc Anh Nguyen (Ludwig-Maximilians-Universität München), Gitta Kutyniok (Ludwig-Maximilians-Universität München)
Spiking Neural NetworkSequential
🎯 What it does: This paper studies a discrete-time spiking neural network (SNN) model based on leaky integrate-and-fire (LIF) neurons, exploring its potential in energy efficiency and expressive capability.
Time-Aware World Model for Adaptive Prediction and Control
Anh N Nhu, Ming Lin (University of Maryland)
Robotic IntelligenceReinforcement LearningWorld ModelTime Series
🎯 What it does: This paper proposes a Time-Aware World Model (TAWM), which explicitly incorporates the time step Δt as an input condition in each prediction step and uses multi-scale Δt sampling during training to construct a world model that can adapt to different observation rates.
Time-VLM: Exploring Multimodal Vision-Language Models for Augmented Time Series Forecasting
Siru Zhong (Hong Kong University of Science and Technology), Yuxuan Liang (Hong Kong University of Science and Technology)
TransformerReinforcement LearningVision Language ModelMultimodalityTime Series
🎯 What it does: The Time-VLM framework is proposed, which integrates pre-trained visual-language models with visual, textual, and raw temporal information from time series data to enhance time series prediction performance.
TimeBase: The Power of Minimalism in Efficient Long-term Time Series Forecasting
Qihe Huang (University of Science and Technology of China), Yang Wang (University of Science and Technology of China)
Time Series
🎯 What it does: A minimalistic time series forecasting model called TimeBase is proposed, which utilizes basis functions to extract features and achieve long-term forecasting through segment-level predictions.
TimeBridge: Non-Stationarity Matters for Long-term Time Series Forecasting
Peiyuan Liu (Tsinghua Shenzhen International Graduate School), Shu-Tao Xia
TransformerTime SeriesFinance Related
🎯 What it does: The TimeBridge framework is proposed to address the different impacts of non-stationarity on short-term and long-term modeling in multivariate time series forecasting, achieving unified modeling through integrated attention (removing short-term non-stationarity) and cointegration attention (preserving long-term cointegration).
TimeDART: A Diffusion Autoregressive Transformer for Self-Supervised Time Series Representation
Daoyu Wang (University of Science and Technology of China), Qi Liu (University of Science and Technology of China)
ClassificationRepresentation LearningTransformerDiffusion modelContrastive LearningTime Series
🎯 What it does: This paper proposes a self-supervised time series pre-training framework called TimeDART, which can simultaneously learn global trends and local details.
TimeFilter: Patch-Specific Spatial-Temporal Graph Filtration for Time Series Forecasting
Yifan Hu (Tsinghua University), Shirui Pan (Griffith University)
Graph Neural NetworkMixture of ExpertsTime Series
🎯 What it does: The TimeFilter framework is proposed, which completes multivariate time series forecasting by constructing spatial-temporal graphs and adaptively filtering each patch.
TimePoint: Accelerated Time Series Alignment via Self-Supervised Keypoint and Descriptor Learning
Ron Shapira Weber (BenGurion University of the Negev), Oren Freifeld (BenGurion University of the Negev)
ClassificationComputational EfficiencyContrastive LearningTime SeriesElectrocardiogram
🎯 What it does: TimePoint is developed, a self-supervised keypoint detection and descriptor learning framework designed to accelerate and enhance the dynamic time warping (DTW) alignment of time series.
TimePro: Efficient Multivariate Long-term Time Series Forecasting with Variable- and Time-Aware Hyper-state
Xiaowen Ma (Huawei Noah's Ark Lab), Xinghao Chen (Huawei Noah's Ark Lab)
Time Series
🎯 What it does: A multivariate long-term time series forecasting model called TimePro based on Mamba has been constructed, utilizing variable and time-aware hyperstates to model the multi-delay relationships between variables.
TimeStacker: A Novel Framework with Multilevel Observation for Capturing Nonstationary Patterns in Time Series Forecasting
Qinglong Liu (Harbin Institute of Technology), Haifeng Li (Harbin Institute of Technology)
TransformerTime SeriesFinance Related
🎯 What it does: This study investigates non-stationary time series forecasting and proposes the TimeStacker framework, which captures frequency features of different scales through multi-layer observation stacking.
TimeStep Master: Asymmetrical Mixture of Timestep LoRA Experts for Versatile and Efficient Diffusion Models in Vision
Shaobin Zhuang (Shanghai Jiao Tong University), Yali Wang (Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences)
Domain AdaptationComputational EfficiencyKnowledge DistillationTransformerMixture of ExpertsDiffusion modelImageVideo
🎯 What it does: The TimeStep Master (TSM) paradigm is proposed, which uses different LoRA experts for different time steps in diffusion models and implements asynchronous mixing in two stages (cultivation and assembly) to enhance multi-scale noise modeling and efficiency.
TIMING: Temporality-Aware Integrated Gradients for Time Series Explanation
Hyeongwon Jang (KAIST), Eunho Yang (KAIST)
Explainability and InterpretabilityRecurrent Neural NetworkTime SeriesElectronic Health Records
🎯 What it does: An interpretable method for time series models, TIMING, is proposed, along with new evaluation metrics: Cumulative Prediction Difference (CPD) and Cumulative Prediction Preservation (CPP) to more fairly measure attribution quality.
TINED: GNNs-to-MLPs by Teacher Injection and Dirichlet Energy Distillation
Ziang Zhou (Hong Kong Polytechnic University), Shiqi Shen (Tencent)
Knowledge DistillationGraph Neural NetworkGraph
🎯 What it does: Transfer the knowledge hierarchy of GNN to MLP through teacher injection and Dirichlet energy distillation;
TinyMIG: Transferring Generalization from Vision Foundation Models to Single-Domain Medical Imaging
Chuang LIU, Haogang Zhu (Beihang University)
SegmentationDomain AdaptationKnowledge DistillationSupervised Fine-TuningImageBiomedical DataMagnetic Resonance Imaging
🎯 What it does: In the single-source domain generalization task, the TinyMIG framework is proposed, which utilizes the prior distribution of visual foundation models and local features to guide and distill a lightweight medical image segmentation model, achieving strong generalization capability with no significant inference overhead.
TLLC: Transfer Learning-based Label Completion for Crowdsourcing
Wenjun Zhang (China University of Geosciences), Chaoqun Li (China University of Geosciences)
ClassificationDomain AdaptationContrastive LearningTabular
🎯 What it does: A label completion method based on transfer learning, TLLC, is proposed to address the issue of label missing due to sparse annotations by workers in crowdsourcing scenarios.
TMetaNet: Topological Meta-Learning Framework for Dynamic Link Prediction
Hao Li (Wuhan University), Hao Jiang (Wuhan University)
Recommendation SystemMeta LearningGraph Neural NetworkGraph
🎯 What it does: This paper proposes a topological meta-learning framework named TMetaNet, which combines high-order topological features obtained from Dowker Zigzag Persistence (DZP) to improve the parameter updates of dynamic graph neural networks in link prediction tasks.
To Each Metric Its Decoding: Post-Hoc Optimal Decision Rules of Probabilistic Hierarchical Classifiers
Roman Plaud (Institut Polytechnique de Paris), Thomas Bonald (Institut Polytechnique de Paris)
ClassificationRecognitionConvolutional Neural NetworkImage
🎯 What it does: This study investigates a post-hoc optimal decoding strategy based on given evaluation metrics in hierarchical classification, deriving optimal decision rules for different candidate prediction sets and implementing a scalable algorithm.
To Steer or Not to Steer? Mechanistic Error Reduction with Abstention for Language Models
Anna Hedström (J.P. Morgan AI Research), Manuela Veloso (J.P. Morgan AI Research)
Large Language ModelSupervised Fine-TuningText
🎯 What it does: The MERA framework is proposed, utilizing activation layer linear detectors for adaptive and reliable error mitigation interventions in language models, supporting abstention when uncertain.
Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning
DiJia Su (Meta AI), Qinqing Zheng (Meta AI)
TransformerLarge Language ModelSupervised Fine-TuningTextChain-of-Thought
🎯 What it does: This paper proposes compressing the early reasoning steps in Chain of Thought (CoT) into discrete latent tokens and mixing them with text tokens to enhance the reasoning performance of large language models (LLMs).
Token Cleaning: Fine-Grained Data Selection for LLM Supervised Fine-Tuning
Jinlong Pang (University of California, Santa Cruz), Yang Liu (University of California, Santa Cruz)
Data-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: A general Token-level cleaning pipeline is proposed, utilizing influence functions to evaluate Token quality and filtering out uninformative Tokens through thresholding, in order to enhance the performance of LLMs in downstream tasks during the supervised fine-tuning phase.
Token Coordinated Prompt Attention is Needed for Visual Prompting
Zichen Liu (Peking University), Jiahuan Zhou (Peking University)
ClassificationRecognitionTransformerPrompt EngineeringImage
🎯 What it does: The Token Coordinated Prompt Attention (TCPA) module is proposed, which allocates dedicated prompts for CLS and image tokens in visual prompts and achieves interaction through an attention mechanism.
Token Signature: Predicting Chain-of-Thought Gains with Token Decoding Feature in Large Language Models
Peijie Liu (Tsinghua University), Yong Li (Tsinghua University)
OptimizationComputational EfficiencyTransformerLarge Language ModelTextBenchmarkChain-of-Thought
🎯 What it does: This paper analyzes the token probability distribution (Token Signature) during the decoding of large language models and proposes two metrics, Instance SC and Aggregated SC, to predict the effectiveness of chain-of-thought (CoT) reasoning under different tasks. Based on Instance SC, a mechanism for dynamically selecting between CoT and direct answers (Dynamic CoT) is developed; additionally, a voting method is proposed to transfer this mechanism to closed-source models.
Tokenized Bandit for LLM Decoding and Alignment
Suho Shin (University of Maryland), MohammadTaghi Hajiaghayi (University of Maryland)
TransformerLarge Language ModelReinforcement LearningText
🎯 What it does: This paper proposes two novel bandit frameworks: tokenized linear bandit (TLB) and tokenized multi-armed bandit (TMAB), to theorize the decoding and alignment problems of LLMs. It proves that learning is infeasible under unstructured assumptions and provides a sublinear regret algorithm under the DDMC assumption.
TokenSwift: Lossless Acceleration of Ultra Long Sequence Generation
Tong Wu (State Key Laboratory of General Artificial Intelligence), Zilong Zheng (State Key Laboratory of General Artificial Intelligence)
GenerationComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: Proposes the TOKENSWIFT framework, achieving lossless acceleration of ultra-long (up to 100K) sequence generation in LLMs;
ToMA: Token Merge with Attention for Diffusion Models
Wenbo Lu (New York University), Shengjie Wang (New York University)
GenerationData SynthesisOptimizationComputational EfficiencyDiffusion modelImage
🎯 What it does: For high-resolution image generation using diffusion models, a training-independent Token Merge with Attention (ToMA) framework is proposed, which performs token merging and recovery in a GPU-friendly manner during attention computation.
Tool Unlearning for Tool-Augmented LLMs
Jiali Cheng (University of Massachusetts Lowell), Hadi Amiri (University of Massachusetts Lowell)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes the task of Tool Unlearning for tool-enhanced large language models (LLMs), aiming to selectively delete the model's ability to use specific tools without affecting other tools and general capabilities.
TopInG: Topologically Interpretable Graph Learning via Persistent Rationale Filtration
Cheng Xin (Rutgers University), Jiaxin Ding (Shanghai Jiao Tong University)
Explainability and InterpretabilityGraph Neural NetworkGraph
🎯 What it does: The TOPING framework is proposed, which learns edge importance sequences through graph neural networks and constructs persistent homology filters to automatically identify subgraphs (rationale) that have a decisive impact on predictions, achieving interpretable graph learning.
TOPLOC: A Locality Sensitive Hashing Scheme for Trustless Verifiable Inference
Jack Min Ong (Prime Intellect), Johannes Hagemann (Prime Intellect)
CompressionComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: A lightweight verifiable reasoning scheme called TOPLOC is proposed, which utilizes the top-k values of the hidden layer activations of LLM for local sensitive hashing and encodes them as integer domain polynomials to generate verifiable proofs.
Topological Signatures of Adversaries in Multimodal Alignments
Minh N. Vu (Los Alamos National Laboratory), Manish Bhattarai (Los Alamos National Laboratory)
Anomaly DetectionAdversarial AttackContrastive LearningImageMultimodality
🎯 What it does: This study investigates the topological features of multimodal (image-text) alignment models under adversarial attacks, proposing two types of topological contrastive losses for detecting adversarial samples.
Topology-Aware Dynamic Reweighting for Distribution Shifts on Graph
Weihuang Zheng (Southeast University), Youyong Kong (Southeast University)
ClassificationDomain AdaptationGraph Neural NetworkGraphBenchmark
🎯 What it does: A Topology-Aware Dynamic Reweighting (TAR) framework is proposed to address the issue of distribution shift between the training and testing sets in node classification tasks. The framework dynamically adjusts sample weights through gradient flow on the graph topology, ensuring the model remains robust under potentially worst-case distributions.
Topology-aware Neural Flux Prediction Guided by Physics
Haoyang Jiang (William and Mary), Yi He (William and Mary)
Graph Neural NetworkGraphPhysics Related
🎯 What it does: A new physics-guided neural flow prediction framework (PhyNFP) is proposed, aimed at enhancing the sensitivity of graph neural networks (GNNs) to high-frequency components in directed graphs, thereby better capturing flow dynamics.
TopoTune: A Framework for Generalized Combinatorial Complex Neural Networks
Mathilde Papillon (University California Santa Barbara), Nina Miolane (University California Santa Barbara)
OptimizationGraph Neural NetworkGraphBenchmark
🎯 What it does: This paper proposes Generalized Combinatorial Complex Neural Networks (GCCNs) and its lightweight implementation framework TopoTune, which systematically converts any neural network into a corresponding model for Topological Deep Learning (TDL) and supports training across different topological domains (simplicial, cellular, hypergraph).
Toward a Unified Theory of Gradient Descent under Generalized Smoothness
Alexander Tyurin (AIRI), Alexander Tyurin (Skoltech)
Optimization
🎯 What it does: This paper proposes and analyzes the convergence properties of gradient descent under the assumption of ℓ-smoothness, and presents a new step size rule.
Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
Xunkai Li (Beijing Institute of Technology), Guoren Wang (Beijing Institute of Technology)
Knowledge DistillationRepresentation LearningData-Centric LearningGraph Neural NetworkGraph
🎯 What it does: A directed graph learning framework for data centers, EDEN, is proposed, which significantly enhances the representation learning effectiveness of DiGNN by constructing a Hierarchical Knowledge Tree (HKT) and combining knowledge distillation driven by information entropy and mutual information.
Toward Efficient Kernel-Based Solvers for Nonlinear PDEs
Zhitong Xu (University of Utah), Houman Owhadi (California Institute of Technology)
OptimizationComputational EfficiencyTabularBenchmarkPhysics Related
🎯 What it does: A new kernel-based framework for solving nonlinear partial differential equations is proposed, using standard kernel interpolation to model the solution and directly computing derivatives through automatic differentiation, thus eliminating the complex Gram matrix construction involved in traditional methods for embedding differential operators into kernels.
Toward Robust Hyper-Detailed Image Captioning: A Multiagent Approach and Dual Evaluation Metrics for Factuality and Coverage
Saehyung Lee (Seoul National University), Sungroh Yoon (Seoul National University)
TransformerLarge Language ModelAgentic AIVision Language ModelImageTextMultimodality
🎯 What it does: This paper proposes the CapMAS multi-agent scheme, which collaborates LLM and MLLM to fact-check and correct ultra-detailed image descriptions, while constructing a dual evaluation framework (factuality and coverage) and corresponding datasets.
Towards a Formal Theory of Representational Compositionality
Eric Elmoznino (Mila - Quebec AI Institute), Guillaume Lajoie (Mila - Quebec AI Institute)
Data SynthesisCompressionText
🎯 What it does: This paper proposes a formal definition of Representational Compositionality based on algorithmic information theory, and provides corresponding quantitative metrics C_Z and C_L.
Towards a General Time Series Forecasting Model with Unified Representation and Adaptive Transfer
Yihang Wang (East China Normal University), Chenjuan Guo (East China Normal University)
Anomaly DetectionTransformerTime Series
🎯 What it does: A general time series prediction model ROSE based on multi-domain pre-training is proposed, which can be fine-tuned with a small amount of data or directly inferred for different downstream tasks.
Towards a Mechanistic Explanation of Diffusion Model Generalization
Matthew Niedoba (University of British Columbia), Frank Wood (University of British Columbia)
RestorationGenerationDiffusion modelImage
🎯 What it does: Analyze the generalization mechanism of diffusion models and propose a patch-based denoiser (PSPC) that is untrained to approximate the output of network denoisers.
Towards a Unified Framework of Clustering-based Anomaly Detection
Zeyu Fang (Zhejiang University), Jiajun Bu (Hangzhou)
Anomaly DetectionOptimizationRepresentation LearningAuto EncoderTabularFinance Related
🎯 What it does: A unified theoretical framework called UniCAD is proposed, which jointly optimizes representation learning, clustering, and unsupervised anomaly detection.
Towards an Explainable Comparison and Alignment of Feature Embeddings
Mohammad Jalali (Chinese University of Hong Kong), Farzan Farnia (Chinese University of Hong Kong)
OptimizationExplainability and InterpretabilityRepresentation LearningContrastive LearningImageText
🎯 What it does: An interpretable embedding comparison and alignment method called SPEC is proposed, which utilizes the spectral decomposition of the kernel difference matrix to identify the differences in clustering between two embedding models, and achieves embedding alignment by minimizing the spectral distance.
Towards Attributions of Input Variables in a Coalition
Xinhao Zheng (Shanghai Jiao Tong University), Quanshi Zhang (Shanghai Jiao Tong University)
Explainability and InterpretabilityImageText
🎯 What it does: This paper studies the attribution conflict problem caused by the grouping of input variables (coalition) in interpretative models, proposing a joint Shapley attribution formula based on AND-OR interactions, and providing three metrics to evaluate the credibility of coalitions.
Towards Better-than-2 Approximation for Constrained Correlation Clustering
Andreas Kalavas (Max Planck Institute for Informatics), Nithin Varma (University of Cologne)
Optimization
🎯 What it does: An improved approximation algorithm for a conditionally solvable constraint-related clustering problem is proposed, which is better than 2;
Towards Black-Box Membership Inference Attack for Diffusion Models
Jingwei Li (Tsinghua University), Jingzhao Zhang (Tsinghua University)
GenerationAdversarial AttackTransformerDiffusion modelImage
🎯 What it does: A black-box method for membership inference attacks (MIA) is proposed, which utilizes only the Variation API of diffusion models to determine whether a given image has been used for model training.
Towards characterizing the value of edge embeddings in Graph Neural Networks
Dhruv Rohatgi (Massachusetts Institute of Technology), Andrej Risteski (Carnegie Mellon University)
Graph Neural NetworkGraph
🎯 What it does: This paper studies the impact of maintaining and updating edge embeddings in Graph Neural Networks (GNNs) on model expressiveness and training performance, providing both theoretical and empirical evidence.
Towards Cost-Effective Reward Guided Text Generation
Ahmad Rashid (University of Waterloo), Pascal Poupart (University of Waterloo)
GenerationComputational EfficiencyReinforcement LearningText
🎯 What it does: A new Reward-Guided Text Generation (RGTG) method is proposed, aimed at improving generation efficiency and reducing computational overhead during inference by calling the reward model only once.
Towards Efficient Online Tuning of VLM Agents via Counterfactual Soft Reinforcement Learning
Lang Feng (Nanyang Technological University), Bo An (Nanyang Technological University)
OptimizationTransformerReinforcement LearningVision Language ModelText
🎯 What it does: This paper proposes an online fine-tuning method called CoSo (Counterfactual Soft Reinforcement Learning), which uses counterfactual reasoning to evaluate the causal impact of each token in sentences generated by VLM on the final executable actions. This allows for the entropy term to be weighted by importance within a maximum entropy reinforcement learning framework, significantly improving the exploration efficiency of the text action space and accelerating policy improvement.
Towards Escaping from Class Dependency Modeling for Multi-Dimensional Classification
Teng Huang (Southeast University), Min-Ling Zhang (Southeast University)
ClassificationRepresentation LearningImageTabular
🎯 What it does: This paper proposes a new multi-dimensional classification method DCOM, which achieves decoupling between different dimensions by learning a hidden factor, thus avoiding the direct modeling of complex dependencies between classes.
Towards flexible perception with visual memory
Robert Geirhos (Google DeepMind), Jonathon Shlens (Google DeepMind)
ClassificationRetrievalTransformerContrastive LearningImage
🎯 What it does: A retrieval-based visual memory framework is proposed, which utilizes a pre-trained visual encoder to retrieve similar samples in a high-dimensional feature space using kNN and aggregates labels for image classification.
Towards Global-level Mechanistic Interpretability: A Perspective of Modular Circuits of Large Language Models
Yinhan He (University of Virginia), Jundong Li (University of Virginia)
Explainability and InterpretabilityTransformerLarge Language ModelReinforcement LearningTextBiomedical Data
🎯 What it does: The research proposes a global interpretability framework for large language models, ModCirc, which achieves task-independent mechanism explanations by constructing a reusable modular circuit (MC) vocabulary.