NeurIPS 2025 Papers — Page 47
Conference on Neural Information Processing Systems · 5275 papers
The Lighthouse of Language: Enhancing LLM Agents via Critique-Guided Improvement
Ruihan Yang (Fudan University), Deqing Yang (Fudan University)
OptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningAgentic AIText
🎯 What it does: Proposed and implemented Critique-Guided Improvement (CGI), a two-player framework that separates the actor (generating candidate actions) from the critic (providing natural language critiques and correction suggestions), utilizing critique to guide LLM agents in continuously improving their decisions in interactive environments.
The Logical Expressiveness of Temporal GNNs via Two-Dimensional Product Logics
Marco Sälzer (RPTU Kaiserslautern Landau), Martin Lange (University of Kassel)
Graph Neural NetworkGraph
🎯 What it does: The theoretical analysis of the logical expressiveness of Temporal Graph Neural Networks (TGNN) is conducted, linking it to two-dimensional product logic (PTL_PY × K), and demonstrating the differences in expressiveness among different TGNN structures.
The Matrix: Infinite-Horizon World Generation with Real-Time Moving Control
Ruili Feng (Tongyi Lab), Hongyang Zhang (University of Waterloo)
GenerationData SynthesisTransformerDiffusion modelVideo
🎯 What it does: Developed The Matrix, a world simulator capable of generating unlimited 720p high-fidelity video streams with real-time frame-level interactive control at 16 FPS, and can transfer to real-world environments under zero-shot conditions.
The Mirage of Performance Gains: Why Contrastive Decoding Fails to Mitigate Object Hallucinations in MLLMs?
Hao Yin (University of Science and Technology of China), Zilei Wang (University of Science and Technology of China)
Large Language ModelPrompt EngineeringContrastive LearningMultimodality
🎯 What it does: Evaluate and reveal the limitations of contrastive decoding methods in alleviating hallucinations in multimodal large language models, demonstrating that the improvement mainly stems from unidirectional adjustments to the output distribution and adaptive realizable constraints;
The Narrow Gate: Localized Image-Text Communication in Native Multimodal Models
Alessandro Pietro Serra (SISSA), Alberto Cazzaniga (Area Science Park)
Explainability and InterpretabilityTransformerVision Language ModelImageTextMultimodality
🎯 What it does: This study investigates the communication mechanism between images and text in native multimodal visual language models (such as Chameleon and Emu3), finding that they achieve narrow-gate information transfer through a single end-of-image (EOI) token.
The Non-Linear Representation Dilemma: Is Causal Abstraction Enough for Mechanistic Interpretability?
Denis Sutter (ETH Zürich), Tiago Pimentel (ETH Zürich)
Explainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: This study investigates the effectiveness of causal abstraction in the interpretability of machine learning, examining how, without linear constraints, any algorithm can perfectly align with any neural network, leading to the so-called non-linear representation dilemma.
The Nuclear Route: Sharp Asymptotics of ERM in Overparameterized Quadratic Networks
Vittorio Erba (École Polytechnique Fédérale de Lausanne), Florent Krzakala (École Polytechnique Fédérale de Lausanne)
OptimizationTabular
🎯 What it does: In the high-dimensional limit, the convergence and generalization performance of empirical risk minimization (ERM) for over-parameterized two-layer quadratic activation networks are studied, providing precise asymptotic expressions for training error, testing error, and global minima.
The Omni-Expert: A Computationally Efficient Approach to Achieve a Mixture of Experts in a Single Expert Model
Sohini Saha (Duke University), Boyla Mainsah
RecognitionComputational EfficiencyRecurrent Neural NetworkMixture of ExpertsAudio
🎯 What it does: This paper proposes and implements the Omni-Expert (OE) model, which combines a single expert network with sub-task specific feature transformations to perform the speech dereverberation task in cochlear implant (CI) devices, and is trained and evaluated on real speech and room impulse response data.
The Overthinker's DIET: Cutting Token Calories with DIfficulty-AwarE Training
Weize Chen (Tsinghua University), Maosong Sun (Tsinghua University)
CompressionTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Proposes the DIET framework, which utilizes RL combined with real-time difficulty estimation to dynamically compress tokens generated by LLMs, significantly reducing verbose outputs caused by excessive reasoning;
The Parameterized Complexity of Computing the VC-Dimension
Florent Foucaud (Universite Clermont Auvergne), Prafullkumar Tale (Indian Institute of Science Education and Research Pune)
🎯 What it does: This paper studies the algorithmic complexity of computing VC-dimension, proving time lower bounds under the Exponential Time Hypothesis (ETH), and provides fixed-parameter algorithms and approximation algorithms based on maximum degree, dimension, and tree width.
The Persistence of Neural Collapse Despite Low-Rank Bias
Connall Garrod (Mathematical Institute University of Oxford), Jonathan P. Keating (Mathematical Institute University of Oxford)
Image
🎯 What it does: This paper systematically analyzes the impact of low-rank bias on the structure of neural collapse (DNC) in deep neural networks by constructing deep unconstrained feature models (linear and ReLU), and proves that DNC is not globally optimal in multilayer cases under cross-entropy loss, but is still easily captured by gradient descent.
The Power of Iterative Filtering for Supervised Learning with (Heavy) Contamination
Adam Klivans, Arsen Vasilyan (University of Texas at Austin)
OptimizationTabular
🎯 What it does: This paper proposes an iterative polynomial filtering algorithm and utilizes this algorithm to construct an efficient supervised learning method under the conditions of 'limited pollution' and 'heavy pollution', further deriving approximately optimal learning algorithms for various concept classes (such as intersecting hyperplanes, monotonic functions, convex sets, etc.);
The Price of Opportunity Fairness in Matroid Allocation Problems
Rémi Castera (Moroccan Center for Game Theory University Mohammed VI Polytechnic), Vianney Perchet (Criteo AI Lab)
Optimization
🎯 What it does: In this paper, the author studies the social welfare loss caused by fair allocation compared to optimal unconstrained allocation when implementing opportunity fairness constraints in the matroid allocation problem, specifically quantifying and defining the Price of Fairness (PoF);
The Price of Sparsity: Sufficient Conditions for Sparse Recovery using Sparse and Sparsified Measurements
Youssef Chaabouni (Massachusetts Institute of Technology), David Gamarnik (Massachusetts Institute of Technology)
Large Language Model
🎯 What it does: This study investigates the problem of recovering the support of sparse signals under sparse measurement matrices, providing sufficient conditions under high signal-to-noise ratio (SNR → ∞) and revealing the information-theoretic critical point; it also explores the feasibility and cost of recovering signals after sparsifying dense measurement matrices.
The Primacy of Magnitude in Low-Rank Adaptation
Zicheng Zhang (JD.com), Junxing Hu
TransformerLarge Language ModelSupervised Fine-TuningTextMultimodality
🎯 What it does: This paper discusses the principle of LoRA update magnitude and proposes a basis-basis initialization LoRAM without SVD, which improves the convergence speed and performance of LoRA.
The Promise of RL for Autoregressive Image Editing
Saba Ahmadi (Mila - Quebec AI Institute), Aishwarya Agrawal (Mila - Quebec AI Institute)
Image TranslationGenerationOptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningImageMultimodalityChain-of-Thought
🎯 What it does: A self-regressive model-based image editing framework called EARL is proposed, utilizing three training paradigms: supervised fine-tuning, reinforcement learning, and chain of thought (CoT). It ultimately proves that RL combined with a multimodal LLM discriminator is the most effective strategy.
The quest for the GRAph Level autoEncoder (GRALE)
Paul Krzakala (Telecom Paris), Rémi Flamary (Ecole Polytechnique)
GenerationRepresentation LearningGraph Neural NetworkAuto EncoderGraph
🎯 What it does: A graph-level autoencoder GRALE is proposed, which can encode graphs of arbitrary size into a shared Euclidean space and decode back to the complete graph from this embedding space.
The Quest for Universal Master Key Filters in DS-CNNs
Zahra Babaiee (Technische Universität Wien), Radu Grosu (Massachusetts Institute of Technology)
CompressionOptimizationConvolutional Neural NetworkAuto EncoderImage
🎯 What it does: This study investigates the structure of depth convolution kernels in depth separable convolution networks, proposing and validating the hypothesis that only 8 principal key filters are needed to approximate the original thousands of convolution kernels.
The Quotient Bayesian Learning Rule
Mykola Lukashchuk (Eindhoven University of Technology), Bert de Vries (Eindhoven University of Technology)
OptimizationTabular
🎯 What it does: Proposes the Quasi-Bayesian Learning Rule (QBLR), which represents non-exponential family distributions (such as Student-t) as the marginal of the minimal exponential family through quotient manifold structure, and then performs natural gradient updates in the exponential family space.
The Rich and the Simple: On the Implicit Bias of Adam and SGD
Bhavya Vasudeva (University of Southern California), Mahdi Soltanolkotabi (University of Southern California)
OptimizationTabular
🎯 What it does: The paper studies the implicit biases of Adam and SGD, finding that Adam tends to learn richer features and produce nonlinear decision boundaries when training two-dimensional ReLU networks, while SGD is more prone to exhibit a 'simplification bias' and learn linear decision boundaries.
The Rise of Parameter Specialization for Knowledge Storage in Large Language Models
Yihuai Hong (New York University), Wenxuan Zhang (Singapore University of Technology and Design)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper quantifies the specialization degree of knowledge storage by activating screening and masking the MLP parameters of 20 open-source large language models, and verifies the causal impact of specialization enhancement on model performance and hallucinations in multiple-choice and open-ended generation tasks.
The Structural Complexity of Matrix-Vector Multiplication
Emile Timothy Anand (Georgia Institute of Technology), Rose McCarty (Georgia Institute of Technology)
OptimizationComputational EfficiencyGraph
🎯 What it does: A matrix-based VC-dimension (or its corrupted version) is proposed to preprocess and quickly compute matrix-vector products, supporting both static and dynamic (row/column addition/deletion) operations.
The Structure of Relation Decoding Linear Operators in Large Language Models
Miranda Anna Christ (HUN-REN Alfréd Rényi Institute of Mathematics), Dániel Varga (HUN-REN Alfréd Rényi Institute of Mathematics)
CompressionOptimizationTransformerLarge Language ModelText
🎯 What it does: This study investigates the structure of linear relationship decoders in Transformer language models, exploring their organization and compressing the entire relationship set into a small tensor network model.
The Surprising Effectiveness of Negative Reinforcement in LLM Reasoning
Xinyu Zhu (Princeton University), Yu Meng (Princeton University)
Reinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: This study investigates the impact of positive and negative rewards in RLVR on the reasoning performance of large language models, finding that using only negative sample reinforcement can significantly enhance reasoning effectiveness.
The third pillar of causal analysis? A measurement perspective on causal representations
Dingling Yao (Institute of Science and Technology Austria), Francesco Locatello (Institute of Science and Technology Austria)
Representation LearningContrastive LearningVideo
🎯 What it does: Proposes to view causal representation learning as a measurement model, treating the learned representations as proxy measurements of latent causal variables.
The Underappreciated Power of Vision Models for Graph Structural Understanding
Xinjian Zhao (Chinese University of Hong Kong), Tianshu Yu (Chinese University of Hong Kong)
ClassificationRecognitionRepresentation LearningGraph Neural NetworkTransformerGraphBenchmark
🎯 What it does: By transforming graphs into visual images and utilizing existing visual models to learn graph structures, a new benchmark called GraphAbstract is proposed to evaluate models' cognitive abilities regarding global graph structures.
The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoning
Shivam Agarwal (University of Illinois Urbana Champaign), Hao Peng (University of Illinois Urbana Champaign)
OptimizationTransformerLarge Language ModelReinforcement LearningTextPhysics Related
🎯 What it does: Three unsupervised post-training and inference methods based on entropy minimization (EM-FT, EM-RL, EM-INF) are proposed and evaluated, enhancing the performance of large language models on complex reasoning, physics, and programming tasks by using only unlabeled data or without updating model parameters.
The Unseen Threat: Residual Knowledge in Machine Unlearning under Perturbed Samples
Hsiang Hsu (JPMorganChase Global Technology Applied Research), Chun-Fu Chen (JPMorganChase AI Research)
ClassificationAdversarial AttackConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: This study investigates the risk of residual knowledge in machine unlearning and proposes the RURK method to mitigate the model's erroneous memory of forgotten samples after perturbation.
The VLLM Safety Paradox: Dual Ease in Jailbreak Attack and Defense
Yangyang Guo (National University of Singapore), Mohan Kankanhalli (National University of Singapore)
Safty and PrivacyAdversarial AttackTransformerLarge Language ModelVision Language ModelImageTextMultimodality
🎯 What it does: This paper studies the security paradox of Vision Large Language Models (VLLM), which are both susceptible to jailbreak attacks and can be suppressed by simple defense mechanisms, and proposes a detection-response pipeline scheme based on LLM.
The World Is Bigger: A Computationally-Embedded Perspective on the Big World Hypothesis
Alex Lewandowski (University of Alberta), Marlos C. Machado (University of Alberta)
Meta LearningReinforcement LearningAgentic AITime SeriesSequential
🎯 What it does: This paper proposes a computationally embedded framework that views agents as finite automata within an environment and introduces interactivity as a measure based on algorithmic complexity to quantify the agent's ability to continuously adapt. A reinforcement learning algorithm is designed to maximize interactivity, and a self-predictive continuous learning evaluation task is constructed.
Theoretical Benefit and Limitation of Diffusion Language Model
Guhao Feng (Peking University), Di He (Peking University)
TransformerDiffusion modelSequential
🎯 What it does: This study investigates the theoretical efficiency and limitations of the Masked Diffusion Model (MDM), proving the dependency of its sampling steps under different evaluation metrics.
Theoretical Guarantees for the Retention of Strict Nash Equilibria by Coevolutionary Algorithms
Alistair Benford (University of Birmingham), Per Kristian Lehre (University of Birmingham)
Optimization
🎯 What it does: This paper studies the stability of co-evolutionary algorithms (CoEAs) in maintaining strict Nash equilibrium under multiple action spaces, providing theoretical thresholds regarding mutation strength, selection operations, and stability. It also validates the theoretical limits through empirical experiments and further derives the upper bound of regret for CoEAs based on the stability results.
Theoretical Insights into In-context Learning with Unlabeled Data
Yingcong Li (University of Michigan), Samet Oymak (Bilkent University)
ClassificationTransformerTabular
🎯 What it does: This paper studies the theory and practice of semi-supervised context learning (SS-ICL) under Gaussian mixture models and explores how the depth of Transformers can utilize unlabeled samples.
Theoretical Investigation of Adafactor for Non-Convex Smooth Optimization
Yusu Hong (Zhejiang University), Junhong Lin (Zhejiang University)
OptimizationTransformerLarge Language ModelText
🎯 What it does: This paper studies the convergence of Adafactor in non-convex smooth optimization, providing theoretical convergence rates under three scenarios: full batch, random (without clipping), and with update clipping, and for the first time proves that it can find stationary points with high probability.
Theoretically Grounded Framework for LLM Watermarking: A Distribution-Adaptive Approach
Haiyun He (Hong Kong University of Science and Technology), Yuheng Bu (University of California Santa Barbara)
OptimizationTransformerLarge Language ModelText
🎯 What it does: A unified theoretical framework is proposed to achieve joint optimization of LLM watermarking, and based on this framework, a distortion-free, distribution-adaptive DAWA algorithm is implemented.
Theory-Driven Label-Specific Representation for Incomplete Multi-View Multi-Label Learning
Quanjiang Li (National University of Defense Technology), Chenping Hou (National University of Defense Technology)
Representation LearningGraph Neural NetworkMultimodality
🎯 What it does: A theory-driven label-specific representation framework (TDLSR) is proposed to address the problem of incomplete multi-view multi-label learning.
ThermalGen: Style-Disentangled Flow-Based Generative Models for RGB-to-Thermal Image Translation
Jiuhong Xiao (New York University), Giuseppe Loianno (University of California, Berkeley)
Image TranslationGenerationDiffusion modelFlow-based ModelGenerative Adversarial NetworkImage
🎯 What it does: We propose ThermalGen, a flow model-based RGB-to-Thermal image translation framework that can synthesize high-quality thermal images from RGB inputs.
Think before Recommendation: Autonomous Reasoning-enhanced Recommender
Xiaoyu Kong (Taobao and Tmall Group of Alibaba), Xiang Wang (National University of Singapore)
Recommendation SystemTransformerLarge Language ModelReinforcement LearningPrompt EngineeringText
🎯 What it does: Proposed two reinforcement learning-based LLM recommendation frameworks, RecZero and RecOne, which learn reasoning and scoring predictions directly on a single LLM without the need for a teacher model and multi-stage distillation.
Think Only When You Need with Large Hybrid-Reasoning Models
Lingjie Jiang (Peking University), Furu Wei (Microsoft Research)
Reinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: A large-scale hybrid reasoning model (LHRM) is proposed, which can adaptively choose between Thinking or No-Thinking modes based on the context of user queries.
Think or Not? Exploring Thinking Efficiency in Large Reasoning Models via an Information-Theoretic Lens
Xixian Yong (Renmin University of China), Xian Wu (Tencent)
Computational EfficiencyTransformerLarge Language ModelTextBenchmarkChain-of-Thought
🎯 What it does: Quantify the thinking efficiency of large reasoning models from an information-theoretic perspective and propose an entropy-based adaptive stopping strategy to reduce the length of reasoning chains.
Think or Not? Selective Reasoning via Reinforcement Learning for Vision-Language Models
Jiaqi WANG, Mike Zheng Shou (National University of Singapore)
TransformerSupervised Fine-TuningReinforcement LearningVision Language ModelMultimodality
🎯 What it does: A two-stage training framework called TON is designed to teach visual-language models when to reason in reinforcement learning, significantly reducing reasoning length while maintaining or even improving performance.
Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chains
Wenhui Tan, Jian Luan (Xiaomi)
CompressionTransformerLarge Language ModelReinforcement LearningTextChain-of-Thought
🎯 What it does: A framework called CoLaR is proposed, which can dynamically compress the LLM inference chain in the latent space, supporting 'silent' inference and allowing for dynamic adjustment of inference speed through a compression factor.
Think-RM: Enabling Long-Horizon Reasoning in Generative Reward Models
Ilgee Hong (Georgia Institute of Technology), Tuo Zhao (Amazon)
Reinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextChain-of-Thought
🎯 What it does: A generative reward model framework named Think-RM was designed and trained, capable of generating reasoning chains of up to thousands of tokens through an internal 'thinking' process, and a training pipeline for RLHF using pairwise preference was proposed;
ThinkAct: Vision-Language-Action Reasoning via Reinforced Visual Latent Planning
Chi-Pin Huang (NVIDIA), Fu-En Yang (NVIDIA)
Robotic IntelligenceTransformerLarge Language ModelReinforcement LearningVision-Language-Action ModelVideoMultimodality
🎯 What it does: The ThinkAct framework is proposed, utilizing a dual-system structure to achieve visual-language-action (VLA) reasoning: first, a multimodal large language model (MLLM) is used for reinforcement learning-driven visual latent planning (generating 2D trajectories), and then the planned latent is injected into a downstream action policy (DiT-Policy) for robust execution, supporting long-term planning, few-shot adaptation, and self-correction.
Thinker: Learning to Think Fast and Slow
Stephen Chung (University of Cambridge), Jie Fu (Shanghai AI Lab)
TransformerLarge Language ModelReinforcement LearningTextBenchmark
🎯 What it does: The Thinker task is proposed, which divides single-turn QA into four stages (quick thinking, verification, slow thinking, and summarization), training the LLM's intuition, evaluation, refinement, and integration abilities through multi-stage reward mechanisms.
Thinking in Character: Advancing Role-Playing Agents with Role-Aware Reasoning
Yihong Tang (Harbin Institute of Technology), Min Zhang (Harbin Institute of Technology)
Knowledge DistillationLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: A role-aware reasoning (RAR) method is proposed, enabling large language models to generate internal thoughts that align with character settings, addressing issues of character deviation and style drift.
Thinking vs. Doing: Improving Agent Reasoning by Scaling Test-Time Interaction
Junhong Shen (Carnegie Mellon University), Aviral Kumar (Carnegie Mellon University)
Robotic IntelligenceTransformerLarge Language ModelReinforcement LearningTextMultimodality
🎯 What it does: This paper proposes a novel testing moment scale dimension called Interaction Scaling, which enhances the agent's information acquisition and behavior adjustment capabilities in dynamic environments by increasing the number of interaction steps. Based on this, we designed TTI (Test-Time Interaction) — an online reinforcement learning framework that adapts the agent to extend the interaction length during deployment.
Thinkless: LLM Learns When to Think
Gongfan Fang (National University of Singapore), Xinchao Wang (National University of Singapore)
OptimizationTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
🎯 What it does: Proposes the Thinkless framework, enabling large language models to adaptively switch between short answers and long chain reasoning;
ThinkSound: Chain-of-Thought Reasoning in Multimodal LLMs for Audio Generation and Editing
Huadai Liu (Hong Kong University of Science and Technology), Wei Xue (Hong Kong University of Science and Technology)
GenerationData SynthesisTransformerLarge Language ModelMultimodalityChain-of-ThoughtAudio
🎯 What it does: Proposes the ThinkSound framework, which utilizes the chain-of-thought (CoT) of multimodal large language models to achieve phased generation and editing of audio from video.
This Time is Different: An Observability Perspective on Time Series Foundation Models
Ben Cohen (Datadog AI Research), Othmane Abou-Amal (Datadog AI Research)
TransformerTime SeriesBenchmark
🎯 What it does: A zero-copy pre-training model TOTO specifically designed for observable time series is proposed, and a large observable data benchmark BOOM is constructed.
Thompson Sampling for Multi-Objective Linear Contextual Bandit
Somangchan Park, Min-hwan Oh
Recommendation SystemOptimizationReinforcement LearningTabular
🎯 What it does: A Thompson Sampling algorithm for multi-objective linear contextual bandits, called MOL-TS, is proposed, and an 'effective Pareto regret' metric is introduced to evaluate the Pareto efficiency of cumulative rewards.
Thompson Sampling in Function Spaces via Neural Operators
Rafael Oliveira (CSIRO Data61), Edwin V. Bonilla (CSIRO Data61)
OptimizationReinforcement LearningTabular
🎯 What it does: A Thompson Sampling method based on Neural Operators is proposed for optimizing known functional objectives of unknown operators in function space.
Thought Communication in Multiagent Collaboration
Yujia Zheng (Carnegie Mellon University), Kun Zhang (Carnegie Mellon University)
Auto EncoderText
🎯 What it does: This paper proposes a framework for 'Thought Communication' that enables multiple agents to collaborate by directly exchanging latent thoughts rather than using natural language.
Thoughts Are All Over the Place: On the Underthinking of Long Reasoning Models
Yue Wang (Tencent), Dong Yu (Tencent)
TransformerLarge Language ModelTextChain-of-Thought
🎯 What it does: This study investigates the phenomenon of 'underthinking' in long reasoning models (LRMs), proposes a token efficiency-based metric, and designs a Thinking Interruption Penalty (TIP) decoding strategy to suppress premature switching of thought processes, thereby enhancing reasoning efficiency and accuracy.
Thresholds for sensitive optimality and Blackwell optimality in stochastic games
Stephane Gaubert, Ricardo D. Katz (Cifasis Conicet)
Optimization
🎯 What it does: The study refines the average payoff criterion in perfect information two-player zero-sum random games, proposing the Hessel threshold and d-sensitive threshold, and provides their upper bounds;
Through the River: Understanding the Benefit of Schedule-Free Methods for Language Model Training
Minhak Song (Korea Advanced Institute of Science and Technology), Chulhee Yun (Korea Advanced Institute of Science and Technology)
OptimizationTransformerLarge Language ModelText
🎯 What it does: This paper analyzes the performance of the Schedule-Free (SF) method in the pre-training of large-scale language models, exploring its optimization behavior on the valley loss surface, and proposes an improved version of SF-AdamW based on theory and experiments to enhance robustness and scalability.
Thumb on the Scale: Optimal Loss Weighting in Last Layer Retraining
Nathan Stromberg (Arizona State University), Lalitha Sankar (Arizona State University)
ClassificationOptimizationConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: In the fine-tuning of the last layer of large models (LLR), an optimal loss weighting method considering parameterized ratios is proposed to enhance the performance of minority classes and balance the error rates across classes.
Tight analyses of first-order methods with error feedback
Daniel Berg Thomsen (INRIA), Aymeric Dieuleveut (CNRS)
Optimization
🎯 What it does: Under the conditions of single agent, L-smooth, and μ-strongly convex functions, a compact convergence analysis of Compressed Gradient Descent (CGD), Classical Error Feedback (EF), and EF 21 is conducted, providing the optimal Lyapunov function and convergence rate, and proving that they have the optimal step size under this setting.
Tight Asymptotics of Extreme Order Statistics
Matias Romero, Jose Correa
🎯 What it does: This study investigates the asymptotic growth of the expected values of the maximum and second maximum (as well as any ℓ-th maximum) in independent and identically distributed random samples as the sample size n increases, providing the tightest upper and lower bounds for these expectations.
Tight Bounds for Answering Adaptively Chosen Concentrated Queries
Emma Rapoport (Tel Aviv University), Uri Stemmer (Google Research)
Safty and PrivacyComputational Efficiency
🎯 What it does: This paper explores the maximum number of concentrated statistical queries that can be accurately answered under the premise of correlation in data samples, presenting an infeasibility proof and a simplified positive result.
Tight Bounds for Maximum Weight Matroid Independent Set and Matching in the Zero Communication Model
Ilan Doron-Arad (Technion)
Optimization
🎯 What it does: This paper proposes a zero-communication model and presents deterministic algorithms for solving the maximum weight matroid independent set (MW-IS) and maximum weight matching (MWM) under this model.
Tight Bounds on the Distortion of Randomized and Deterministic Distributed Voting
Mohammad Abam, Masoud Seddighin (Tehran Institute for Advanced Studies)
🎯 What it does: This paper studies the distortion bounds of randomized and deterministic mechanisms in distributed voting under metric spaces, providing nearly optimal upper and lower bounds for most objectives.
Tight Generalization Bounds for Large-Margin Halfspaces
Kasper Green Larsen (Aarhus University), Natascha Schalburg (Aarhus University)
🎯 What it does: A generalization error upper bound for large margin hyperplanes is proposed, and it is proven that this upper bound matches the lower bound in the trade-off between margin, empirical margin loss, sample size, and confidence, achieving asymptotic optimality;
Tight High-Probability Bounds for Nonconvex Heavy-Tailed Scenario under Weaker Assumptions
Weixin An (Xidian University), Hongying Liu (Tianjin University)
OptimizationFederated LearningRecurrent Neural NetworkImageText
🎯 What it does: This paper studies the high-probability convergence and generalization analysis of the gradient clipping method (Clipped SGD) and the clipped batch gradient algorithm (FedCBG) in federated learning under the conditions of non-convex, non-smooth settings with heavy-tailed noise;
Tight Lower Bounds and Improved Convergence in Performative Prediction
Pedram Khorsandi (Mila Quebec AI Institute Universite de Montreal), Gauthier Gidel (Mila Quebec AI Institute Universite de Montreal)
Recommendation SystemOptimizationComputational EfficiencyTabularFinance Related
🎯 What it does: This paper proposes the Affine Risk Minimizers (ARM) method, which utilizes linear combinations of historical training snapshots to accelerate the convergence of models to stable points in performative prediction.
Tightening Regret Lower and Upper Bounds in Restless Rising Bandits
Cristiano Migali (Politecnico di Milano), Alberto Maria Metelli (Politecnico di Milano)
OptimizationReinforcement Learning from Human FeedbackTabular
🎯 What it does: This paper theoretically studies the Rising and Rising Concave multi-armed bandit (MAB) problems, providing lower and upper bounds for both types of problems, and proposes a new algorithm called RC-BE.
Tighter CMI-Based Generalization Bounds via Stochastic Projection and Quantization
Milad Sefidgaran (Huawei Technologies France), Abdellatif Zaidi (Universite Gustave Eiffel)
Information Theory
🎯 What it does: By introducing random projection and lossy compression, a conditional mutual information (CMI) framework is reconstructed, resulting in a tighter generalization error upper bound than traditional CMI;
Tiled Flash Linear Attention: More Efficient Linear RNN and xLSTM Kernels
Maximilian Beck (Johannes Kepler University Linz), Sepp Hochreiter (Johannes Kepler University Linz)
Computational EfficiencyRecurrent Neural NetworkSequential
🎯 What it does: Designed and implemented the Tiled Flash Linear Attention (TFLA) algorithm and its efficient kernel on mLSTM, and proposed a Sigmoid input gate version of mLSTM to enhance the computational efficiency and training stability of long sequence linear RNNs.
Time Reversal Symmetry for Efficient Robotic Manipulations in Deep Reinforcement Learning
Yunpeng Jiang (Shanghai Jiao Tong University), Yutong Ban (Shanghai Jiao Tong University)
Robotic IntelligenceReinforcement LearningTabular
🎯 What it does: This paper introduces time-reversal symmetry and designs a TR-DRL framework to enhance the sample efficiency of deep reinforcement learning in robotic manipulation tasks.
Time Series Generation Under Data Scarcity: A Unified Generative Modeling Approach
Tal Gonen (Ben-Gurion University of Negev), Omri Azencot (Ben-Gurion University of Negev)
GenerationData SynthesisDiffusion modelTime SeriesSequentialFinance Related
🎯 What it does: A unified time series generation framework is proposed and implemented, combining cross-domain pre-training and few-shot fine-tuning, capable of generating high-quality time series with extremely low data volumes.
Time-Embedded Algorithm Unrolling for Computational MRI
Junno Yun (University of Minnesota), Mehmet Akcakaya
RestorationConvolutional Neural NetworkBiomedical DataMagnetic Resonance Imaging
🎯 What it does: A time-embedded algorithm expansion framework is proposed for the reconstruction of undersampled magnetic resonance imaging (MRI).
Time-Evolving Dynamical System for Learning Latent Representations of Mouse Visual Neural Activity
Liwei Huang (Peking University), Yonghong Tian (Peking University)
Representation LearningRecurrent Neural NetworkAuto EncoderContrastive LearningTime SeriesSequential
🎯 What it does: A temporal latent variable model, TE-ViDS, is proposed to learn low-dimensional temporal evolution representations from neural firing in the visual cortex of mice, further decomposed into stimulus-related external latent variables and internal latent variables influenced by internal states.
Time-Masked Transformers with Lightweight Test-Time Adaptation for Neural Speech Decoding
Ebrahim Feghhi (University of California), Jonathan Kao (University of California)
RecognitionComputational EfficiencyTransformerAudio
🎯 What it does: Improved neural speech decoding algorithm to achieve real-time, low-cost decoding.
Time-o1: Time-Series Forecasting Needs Transformed Label Alignment
Hao Wang (Xiaohongshu Inc), Zhouchen Lin (Peking University)
TransformerTime Series
🎯 What it does: A learning objective based on label sequence transformation, Time-o1, is proposed for time series prediction.
Time-R1: Post-Training Large Vision Language Model for Temporal Video Grounding
Ye Wang (Renmin University of China), Qin Jin (Renmin University of China)
OptimizationTransformerReinforcement LearningVision Language ModelVideoMultimodalityBenchmarkChain-of-Thought
🎯 What it does: We propose Time-R1, a post-training framework based on reinforcement learning that enhances the generalization ability of large-scale visual language models in temporal video localization tasks.
Time-uniform and Asymptotic Confidence Sequence of Quantile under Local Differential Privacy
Leheng Cai (Tsinghua University), Shuyuan Wu (Shanghai University of Finance and Economics)
OptimizationSafty and PrivacyTabular
🎯 What it does: An online, O(κ) memory local differential privacy algorithm is proposed, capable of constructing time-uniform asymptotic confidence sequences for estimating quantiles.
TimeEmb: A Lightweight Static-Dynamic Disentanglement Framework for Time Series Forecasting
Mingyuan Xia (Jilin University), Bo Yang (Jilin University)
Time Series
🎯 What it does: A lightweight static-dynamic decomposition framework called TimeEmb is proposed for time series forecasting.
Timely Clinical Diagnosis through Active Test Selection
Silas Ruhrberg Estévez (University of Cambridge), Mihaela van der Schaar (University of Cambridge)
ClassificationExplainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelBiomedical DataElectronic Health Records
🎯 What it does: This paper proposes the ACTMED framework, which utilizes Bayesian experimental design and large language models to achieve active test selection and transparent reasoning in clinical diagnosis.
TimePerceiver: An Encoder-Decoder Framework for Generalized Time-Series Forecasting
Jaebin Lee (Sungkyunkwan University), Hankook Lee (Sungkyunkwan University)
TransformerTime Series
🎯 What it does: The TIMEPERCEIVER framework is proposed, unifying the encoder, decoder, and training strategy, enabling multivariate time series forecasting for extrapolation, interpolation, and filling of time segments at any position.
TimeWak: Temporal Chained-Hashing Watermark for Time Series Data
Zhi Wen Soi (University of Neuchâtel), Lydia Y. Chen (Delft University of Technology)
Diffusion modelTime SeriesFinance Related
🎯 What it does: This paper proposes TimeWak, a generative watermarking scheme for multivariate time series diffusion models that can embed detectable watermarks in the data space.
TimeXL: Explainable Multi-modal Time Series Prediction with LLM-in-the-Loop
Yushan Jiang (University of Connecticut), Haifeng Chen (NEC Labs America)
Explainability and InterpretabilityTransformerLarge Language ModelMultimodalityTime SeriesBiomedical DataFinance Related
🎯 What it does: Proposes the TimeXL framework, which combines multimodal time series forecasting with interactions from three types of LLMs (prediction, reflection, refinement) to achieve interpretable predictions.
TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learning
Andreas Auer (NXAI GmbH), Sepp Hochreiter (Johannes Kepler University Linz)
Recurrent Neural NetworkTime SeriesBenchmark
🎯 What it does: Developed and released TiRex, a pre-trained time series zero-shot prediction model based on xLSTM.
TITAN: A Trajectory-Informed Technique for Adaptive Parameter Freezing in Large-Scale VQE
Yifeng Peng (Stevens Institute of Technology), Yuxuan Du (Nanyang Technological University)
OptimizationConvolutional Neural NetworkReinforcement LearningTabularPhysics Related
🎯 What it does: The TITAN framework is proposed and implemented, utilizing deep learning to predict and freeze redundant parameters in VQE in advance, thereby reducing measurement overhead while maintaining or improving energy estimation accuracy.
Titans: Learning to Memorize at Test Time
Ali Behrouz (Google Research), Vahab Mirrokni (Google Research)
RetrievalOptimizationSafty and PrivacyRecurrent Neural NetworkTransformerLarge Language ModelText
🎯 What it does: A deep neural long short-term memory module that learns memory during testing is proposed, and based on this, three Titans architectures (Context, Gate, Layer) are constructed to improve performance in language modeling, reasoning, and long sequence retrieval tasks under large contexts.
To Distill or Decide? Understanding the Algorithmic Trade-off in Partially Observable RL
Yuda Song, Drew Bagnell
Knowledge DistillationReinforcement LearningSequential
🎯 What it does: This paper discusses the performance and algorithmic trade-offs of using privileged expert distillation in partially observable reinforcement learning compared to standard RL with only frame stacking under different environmental dynamics. It proposes the theories of approximate decodability and belief contraction, and provides an analysis of the perturbed Block MDP model.
To Think or Not To Think: A Study of Thinking in Rule-Based Visual Reinforcement Fine-Tuning
Ming Li (Shanghai AI Laboratory), Kaipeng Zhang (Shanghai AI Laboratory)
ClassificationTransformerLarge Language ModelReinforcement LearningPrompt EngineeringImageMultimodality
🎯 What it does: This study investigates the impact of explicit thinking processes in Rule-based Fine-Tuning (RFT) on Multimodal Large Language Models (MLLMs), proposing methods such as No-Thinking RFT, Think-After-Answer, and Adaptive-Thinking, and conducting systematic experiments on tasks like image classification and visual reasoning.
ToF-IP: Time-of-Flight Enhanced Sparse Inertial Poser for Real-time Human Motion Capture
Yuan Yao (Xiamen University), Yipeng Qin (Cardiff University)
Pose EstimationRecurrent Neural NetworkTransformerSimultaneous Localization and MappingVideoTime Series
🎯 What it does: A real-time, full-body human motion capture system has been achieved by combining low-cost IMU and lightweight ToF sensors.
Token Bottleneck: One Token to Remember Dynamics
Taekyung Kim (NAVER AI Lab), Sangdoo Yun (NAVER AI Lab)
Representation LearningRobotic IntelligenceTransformerContrastive LearningVideo
🎯 What it does: A self-supervised visual pre-training framework named Token Bottleneck (ToBo) is proposed, which compresses reference frames using a single bottleneck token and reconstructs them with a minimal number of target frame patches to learn temporal representations of dynamic scenes, thereby enhancing the performance of sequential tasks such as robot control and video label propagation.
Token Embeddings Violate the Manifold Hypothesis
Michael Robinson (American University), Tony Chiang (University of Washington)
Large Language ModelText
🎯 What it does: This paper explores whether the embedding space of large language models satisfies manifold or fiber bundle assumptions through the design of statistical tests.
Token Perturbation Guidance for Diffusion Models
Javad Rajabi (University of Toronto), Babak Taati (University of Toronto)
GenerationData SynthesisTransformerDiffusion modelImage
🎯 What it does: A training-independent and condition-independent Token Perturbation Guidance (TPG) method is proposed to enhance the generation quality and semantic alignment of diffusion models.
Token-Level Self-Play with Importance-Aware Guidance for Large Language Models
Tue Le (Hanoi University of Science and Technology), Trung Le (Monash University)
Knowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
🎯 What it does: We propose SWIFT, a teacher-guided self-adversarial fine-grained token-weighted training method for aligning and distilling knowledge from large language models.
TokenSqueeze: Performance-Preserving Compression for Reasoning LLMs
Yuxiang Zhang (Zhejiang University), Jieping Ye (Alibaba Cloud)
CompressionOptimizationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Proposed the TokenSqueeze method to achieve inference chain compression while maintaining performance.
TokenSwap: A Lightweight Method to Disrupt Memorized Sequences in LLMs
Parjanya Prajakta Prashant (University of California San Diego), Babak Salimi (University of California San Diego)
GenerationOptimizationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes a method to prevent large language models from producing memorized outputs during the post-inference stage—TokenSwap. It suppresses the model's direct reproduction of training data by replacing probabilities on high-frequency grammatical vocabulary with those from a small auxiliary model.
TokMan:Tokenize Manhattan Mask Optimization for Inverse Lithography
Yiwen Wu (ShanghaiTech University), Jingyi Yu (ShanghaiTech University)
SegmentationOptimizationTransformerDiffusion modelImage
🎯 What it does: The paper addresses the segmentation and discretization of Manhattan layouts in inverse lithography technology, utilizing a diffusion transformer to learn optical correction in the discrete symbol space, ultimately outputting a binary mask that meets manufacturing constraints.
TOMCAT: Test-time Comprehensive Knowledge Accumulation for Compositional Zero-Shot Learning
Xudong Yan (Beijing Jiaotong University), Songhe Feng (Beijing Jiaotong University)
Knowledge DistillationRepresentation LearningTransformerPrompt EngineeringContrastive LearningImageTextMultimodality
🎯 What it does: A framework named TOMCAT is proposed, which continuously accumulates multimodal knowledge (visual and textual) using unlabeled data during the testing phase of CZSL, and updates category prototypes through adaptive weighting to address the issue of label space distribution drift.
Too Late to Recall: Explaining the Two-Hop Problem in Multimodal Knowledge Retrieval
Constantin Venhoff (University of Oxford), Neel Nanda
RetrievalTransformerVision Language ModelImageMultimodalityChain-of-Thought
🎯 What it does: This paper systematically evaluates the performance of Visual Language Models (VLM) in factual recall tasks and proposes a two-hop (entity-representation + factual-recall) problem framework to explain the accuracy drop of VLM compared to its LLM baseline.
Tool-Augmented Spatiotemporal Reasoning for Streamlining Video Question Answering Task
Sunqi Fan (Tsinghua University), Shuojin Yang (Tsinghua University)
RecognitionObject DetectionComputational EfficiencyTransformerLarge Language ModelPrompt EngineeringVideo
🎯 What it does: This paper constructs a lightweight video toolbox consisting of 22 types that cover space, time, and general functions, and proposes a Star Alternating Time-Space Reasoning framework (STAR) to achieve step-by-step localization and reasoning of 3D RoI in video question-answering tasks.
ToolRL: Reward is All Tool Learning Needs
Cheng Qian (University of Illinois Urbana-Champaign), Heng Ji (University of Illinois Urbana-Champaign)
Reinforcement Learning
🎯 What it does: This paper systematically studies reward design in reinforcement learning and proposes a refined reward framework for Tool Integrated Reasoning (TIR). It implements the ability to use tools in LLMs from scratch using various RL algorithms (GRPO, PPO), significantly improving tool selection and invocation performance.
Top-H Decoding: Adapting the Creativity and Coherence with Bounded Entropy in Text Generation
Erfan Baghaei Potraghloo (University of Southern California), Massoud Pedram (Intel AI)
GenerationOptimizationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes and implements an entropy-constrained adaptive sampling method called Top-H, aimed at balancing creativity and coherence in large language model generation.
TopER: Topological Embeddings in Graph Representation Learning
Astrit Tola (Florida State University), Baris Coskunuzer (University of Texas at Dallas)
ClassificationRepresentation LearningGraph Neural NetworkGraphBenchmark
🎯 What it does: A low-dimensional graph embedding method called TopER based on topological data analysis is proposed, which uses the filtering sequence of the graph to obtain two coefficients (intercept and slope) through linear regression to represent the structural evolution of the graph.