ICLR 2025 Papers — Page 34
International Conference on Learning Representations · 3704 papers
Theory on Mixture-of-Experts in Continual Learning
Hongbo Li (Singapore University of Technology and Design), Ness Shroff
Mixture of ExpertsImage
🎯 What it does: The theoretical analysis of the behavior of sparse gated Mixture-of-Experts (MoE) in continual learning (CL) is conducted, proving its ability to achieve expert specialization, correct routing, and balanced load, while providing explicit expressions for forgetting and generalization errors, and validating theoretical conclusions through experiments with linear models and deep networks.
Theory on Score-Mismatched Diffusion Models and Zero-Shot Conditional Samplers
Yuchen Liang (Ohio State University), Ness Shroff (Ohio State University)
RestorationGenerationData SynthesisSuper ResolutionDiffusion modelScore-based ModelImage
🎯 What it does: This paper studies diffusion models where the training distribution does not match the target distribution, providing a general convergence guarantee and applying it to zero-shot conditional sampling, further designing a BO-DDNM sampler that minimizes bias.
Theory, Analysis, and Best Practices for Sigmoid Self-Attention
Jason Ramapuram (Apple), Russell Webb
Computational EfficiencyTransformerSupervised Fine-TuningImageTextAudio
🎯 What it does: This paper provides an in-depth theoretical and empirical analysis of the sigmoid self-attention mechanism, demonstrating its effectiveness as an alternative in transformer architectures, and introduces FLASHSIGMOID, a hardware-aware and memory-efficient implementation.
ThermalGaussian: Thermal 3D Gaussian Splatting
Rongfeng Lu (Hangzhou Dianzi University), anke xue
GenerationData SynthesisGaussian SplattingImageMultimodality
🎯 What it does: This paper introduces ThermalGaussian, which is the first to transfer 3D Gaussian Splatting to multi-modal (RGB and thermal) scene reconstruction, capable of simultaneously generating high-quality RGB and thermal images.
Think Then React: Towards Unconstrained Action-to-Reaction Motion Generation
Wenhui Tan (Renmin University of China), Ruihua Song (Renmin University of China)
GenerationPose EstimationRobotic IntelligenceTransformerLarge Language ModelSupervised Fine-TuningVideoText
🎯 What it does: Proposes the Think-Then-React (TTR) framework for action-to-reaction generation in a prompt-free, online, multi-person context, and implements a unified spatial-pose tokenizer and a two-stage (thinking-reacting) generation process.
Think Thrice Before You Act: Progressive Thought Refinement in Large Language Models
Chengyu Du (Fudan University), Yanghua Xiao (Fudan University)
TransformerLarge Language ModelReinforcement LearningPrompt EngineeringTextChain-of-Thought
🎯 What it does: The Progressive Thought Refinement (PTR) framework is proposed, enabling large language models to iteratively improve answers through multi-round reasoning.
Think while You Generate: Discrete Diffusion with Planned Denoising
Sulin Liu (Massachusetts Institute of Technology), Rafael Gomez-Bombarelli
GenerationData SynthesisTransformerDiffusion modelImageText
🎯 What it does: A 'Planning-Denoising' framework for discrete diffusion, DDPD, is proposed, dividing the generation process into two parts: a planner and a denoiser.
Think-on-Graph 2.0: Deep and Faithful Large Language Model Reasoning with Knowledge-guided Retrieval Augmented Generation
Shengjie Ma (International Digital Economy Academy), Jian Guo (Renmin University of China)
GenerationRetrievalTransformerLarge Language ModelTextFinance RelatedRetrieval-Augmented Generation
🎯 What it does: This paper presents Think-on-Graph 2.0 (ToG-2), a tightly coupled KG×Text RAG framework that iteratively combines knowledge graphs and document retrieval to enhance the deep reasoning of LLMs.
ThinK: Thinner Key Cache by Query-Driven Pruning
Yuhui Xu (Salesforce AI Research), Doyen Sahoo (Salesforce AI Research)
CompressionComputational EfficiencyTransformerSequentialBenchmark
🎯 What it does: A query-driven KV cache channel pruning method called THINK is proposed, which significantly reduces memory consumption during long sequence inference.
ThinkBot: Embodied Instruction Following with Thought Chain Reasoning
Guanxing Lu (Tsinghua University), Yansong Tang (Tsinghua University)
Object DetectionRobotic IntelligenceTransformerLarge Language ModelPrompt EngineeringTextMultimodalityBenchmarkChain-of-Thought
🎯 What it does: This paper proposes ThinkBot, which utilizes large language model reasoning to complete sparse instructions and provides interactive locations using a multimodal object locator, thereby achieving efficient embedded instruction following.
Three Mechanisms of Feature Learning in a Linear Network
Yizhou Xu (Ecole Polytechnique Federale de Lausanne), Liu Ziyin (Massachusetts Institute of Technology)
ClassificationRepresentation LearningImageOrdinary Differential Equation
🎯 What it does: This paper presents an exact analysis of the learning dynamics of a linear network with a hidden layer at arbitrary widths, revealing two phases: kernel learning and feature learning.
ThunderKittens: Simple, Fast, and $\textit{Adorable}$ Kernels
Benjamin Frederick Spector, Christopher Re (Stanford University)
OptimizationComputational EfficiencyBenchmark
🎯 What it does: The THUNDERKITTENS (TK) framework is proposed, which implements high-performance AI kernels using a set of simple and composable abstractions (matrix tiles, asynchronous LCSF templates, persistent block scheduling) covering various operators such as GEMM, attention, linear attention, and state space models.
TidalDecode: Fast and Accurate LLM Decoding with Position Persistent Sparse Attention
Lijie Yang (Carnegie Mellon University), Zhihao Jia (Carnegie Mellon University)
GenerationComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: The TidalDecode framework is proposed, utilizing position-persistent sparse attention to achieve fast and high-quality generation in LLM decoding.
TIGER: Time-frequency Interleaved Gain Extraction and Reconstruction for Efficient Speech Separation
Mohan Xu (Tsinghua University), Xiaolin Hu (Tsinghua University)
RecognitionOptimizationComputational EfficiencyConvolutional Neural NetworkAudio
🎯 What it does: A lightweight time-frequency domain speech separation model called TIGER is proposed, and a more realistic EchoSet dataset is constructed.
TIGeR: Unifying Text-to-Image Generation and Retrieval with Large Multimodal Models
Leigang Qu (National University of Singapore), Tat-Seng Chua (National University of Singapore)
GenerationRetrievalTransformerLarge Language ModelVision Language ModelImageTextMultimodalityBenchmarkRetrieval-Augmented Generation
🎯 What it does: A unified framework for text-to-image generation and retrieval, TIGeR-ONE, is proposed, along with the construction of the TIGeR-Bench benchmark covering creative and knowledge domains.
Tight Clusters Make Specialized Experts
Stefan Nielsen, Tan Minh Nguyen
OptimizationAdversarial AttackTransformerMixture of ExpertsImageText
🎯 What it does: This paper proposes an Adaptive Clustering Router (AC router) and the corresponding ACMoE layer, which can match tokens with experts in a feature-weighted transformation space, significantly improving the convergence speed, robustness, and overall performance of sparse mixture of experts models.
Tight Lower Bounds under Asymmetric High-Order Hölder Smoothness and Uniform Convexity
Site Bai, Brian Bullins (Purdue University)
Optimization
🎯 What it does: This paper addresses high-order Hölder smooth objective functions with uniform convexity, providing matching optimal lower bounds in two asymmetric cases (q > p + ν and q < p + ν). It proves that any upper bound algorithm requires at least Ω((Hσ)^{2/3(p+ν)-2}(σ/ε)^{2(q-pν)/q(3(p+ν)-2)}) calls or Ω((Hσ)^{2/3(p+ν)-2}+loglog((σp+νHq)^{1/(p+ν-q)}/ε)) calls.
Tight Time Complexities in Parallel Stochastic Optimization with Arbitrary Computation Dynamics
Alexander Tyurin (AIRI)
Optimization
🎯 What it does: This paper proposes a unified computational model, deriving the lower bound of time complexity for distributed stochastic optimization under arbitrary computational dynamics, and proves that Rennala SGD and Malenia SGD achieve this lower bound in both homogeneous and heterogeneous scenarios, completing a theoretical optimality analysis.
Tighter Privacy Auditing of DP-SGD in the Hidden State Threat Model
Tudor Ioan Cebere (Inria), Nicolas Papernot (University of Toronto)
Safty and PrivacyConvolutional Neural NetworkGaussian SplattingImageTabular
🎯 What it does: This paper conducts a closer audit of the privacy leakage of DP-SGD under the hidden state threat model.
Time After Time: Deep-Q Effect Estimation for Interventions on When and What to do
Yoav Wald (New York University), Rajesh Ranganath (New York University)
TransformerReinforcement LearningTime SeriesSequentialBiomedical DataFinance Related
🎯 What it does: This paper proposes an offline policy evaluation method for handling irregular time points and observations—Earliest Disagreement Q-Evaluation (EDQ), which can estimate the causal effects of treatment timing and content at continuous time points.
Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts
Xiaoming Shi (Xiaohongshu Inc), Ming Jin (Griffith University)
TransformerMixture of ExpertsTime Series
🎯 What it does: A time series Transformer architecture called TIME-MOE based on sparse Mixture-of-Experts is proposed to build a scalable foundational model for time series.
Time-to-Event Pretraining for 3D Medical Imaging
Zepeng Frazier Huo, Nigam Shah
ClassificationConvolutional Neural NetworkTransformerImageBiomedical DataComputed TomographyElectronic Health Records
🎯 What it does: This paper proposes and implements a time-to-event (TTE) pre-training framework that utilizes large-scale longitudinal electronic health records (EHR) supervision to enhance the performance of 3D medical imaging models in predicting future disease risks.
TimeInf: Time Series Data Contribution via Influence Functions
Yizi Zhang (Columbia University), Yongchan Kwon (Columbia University)
Anomaly DetectionTime Series
🎯 What it does: This paper proposes TimeInf, a contribution measurement method for time series data based on influence functions, aimed at identifying anomalies and assessing data quality.
TimeKAN: KAN-based Frequency Decomposition Learning Architecture for Long-term Time Series Forecasting
Songtao Huang (Shanghai Artificial Intelligence Laboratory), LEI BAI
Convolutional Neural NetworkTime Series
🎯 What it does: The TimeKAN model is proposed, achieving efficient prediction of long sequences through multi-layer moving average preprocessing, frequency decomposition, deep convolution, and multi-order KAN learning.
TimeMixer++: A General Time Series Pattern Machine for Universal Predictive Analysis
Shiyu Wang (Griffith University), Ming Jin (Griffith University)
ClassificationAnomaly DetectionOptimizationTransformerTime Series
🎯 What it does: A universal time series pattern machine (TIMEMIXER++) has been constructed, capable of simultaneously capturing seasonal and trend features in multi-scale and multi-period time series data, and achieving cross-task prediction, classification, anomaly detection, and missing value imputation.
Timer-XL: Long-Context Transformers for Unified Time Series Forecasting
Yong Liu (Tsinghua University), Mingsheng Long (Tsinghua University)
TransformerTime Series
🎯 What it does: This paper presents Timer-XL, a causal Transformer capable of handling long contexts and unifying multivariate time series forecasting, using multivariate next-token prediction to unify different types of forecasting tasks.
TimeSuite: Improving MLLMs for Long Video Understanding via Grounded Tuning
Xiangyu Zeng (Nanjing University), Limin Wang (Nanjing University)
RecognitionGenerationTransformerLarge Language ModelSupervised Fine-TuningVideoText
🎯 What it does: This work proposes TimeSuite, which utilizes Token Shuffle, TAPE, and time-based instruction tuning to transform short video MLLM for long video understanding and temporal localization.
TIPS: Text-Image Pretraining with Spatial awareness
Kevis-kokitsi Maninis, Andre Araujo
ClassificationSegmentationRetrievalTransformerContrastive LearningImageTextMultimodality
🎯 What it does: A general image-text pre-training model named TIPS is proposed, which combines contrastive learning, self-distillation, and masked image modeling to significantly improve the performance of dense visual tasks and global tasks.
TIS-DPO: Token-level Importance Sampling for Direct Preference Optimization With Estimated Weights
Aiwei Liu (Tsinghua University), Meng Cao (Apple)
OptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningPrompt EngineeringText
🎯 What it does: Proposes Token-level Importance Sampling based Direct Preference Optimization (TIS-DPO), assigning weights to each token and improving the optimization process.
TLDR: Token-Level Detective Reward Model for Large Vision Language Models
Deqing Fu (University of Southern California), Lawrence Chen (Meta)
OptimizationExplainability and InterpretabilityReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningVision Language ModelImageTextMultimodality
🎯 What it does: The Token-Level Detective Reward Model (TLDR) is proposed, which enhances the accuracy, interpretability, and self-correction ability of visual language models (VLM) by providing fine-grained rewards for each text token.
To Clip or not to Clip: the Dynamics of SGD with Gradient Clipping in High-Dimensions
Noah Marshall (McGill University), Elliot Paquette (McGill University)
OptimizationTabularStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: This paper studies the impact of gradient clipping on learning dynamics for the least squares problem under high-dimensional stochastic gradient descent (SGD) streaming training, providing theoretical analysis and empirical validation.
To Code or Not To Code? Exploring Impact of Code in Pre-training
Viraat Aryabumi (Cohere), Sara Hooker (Cohere)
GenerationData SynthesisAI Code AssistantTransformerLarge Language ModelText
🎯 What it does: A systematic study on the impact of incorporating code data into the pre-training of large-scale language models, exploring how different settings of code ratio, code quality, initialization methods, and pre-training cooling phases affect performance on non-code tasks (natural language inference, world knowledge, generation quality, and code performance).
To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning
Zayne Rea Sprague, Greg Durrett (University of Texas at Austin)
TransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: This paper explores the effectiveness of prompt-based Chain-of-Thought (CoT) through systematic meta-analysis and experimental evaluation across different tasks, finding that it significantly enhances performance primarily in mathematical and symbolic reasoning tasks.
To Tackle Adversarial Transferability: A Novel Ensemble Training Method with Fourier Transformation
Wanlin Zhang (University of Science and Technology of China), Hu Ding (University of Science and Technology of China)
Adversarial AttackImage
🎯 What it does: This paper proposes an integrated training method based on frequency domain transformation (FDT), which significantly enhances adversarial robustness by introducing random noise or targeted attack noise at low amplitude frequencies to allocate the vulnerable directions of sub-models.
To Trust or Not to Trust? Enhancing Large Language Models' Situated Faithfulness to External Contexts
Yukun Huang (Duke University), Bhuwan Dhingra (Duke University)
TransformerLarge Language ModelTextChain-of-Thought
🎯 What it does: Evaluate the embodied credibility of large language models when faced with potentially erroneous external contexts, and propose two methods, Self-Guided Confidence Reasoning (SCR) and Rule-Based Confidence Reasoning (RCR), to enhance the model's balanced judgment of internal and external knowledge.
ToddlerDiffusion: Interactive Structured Image Generation with Cascaded Schrödinger Bridge
Eslam Mohamed BAKR (King Abdullah University of Science and Technology), Mohamed Elhoseiny (King Abdullah University of Science and Technology)
GenerationData SynthesisDiffusion modelImage
🎯 What it does: A phased diffusion generation framework (ToddlerDiffusion) is proposed, which first generates sketches and color palettes, and then gradually synthesizes the final RGB image.
TODO: Enhancing LLM Alignment with Ternary Preferences
Yuxiang Guo (Meituan Inc.), Jiaqi Zhang (Meituan Inc.)
Recommendation SystemOptimizationTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: This paper proposes integrating a three-level grading system (excellent, good, poor) into preference modeling, improving the Bradley-Terry model to TOBT, and designing the TODO algorithm based on this to enhance the preference alignment effect of LLMs.
Token Statistics Transformer: Linear-Time Attention via Variational Rate Reduction
Ziyang Wu (University of California Berkeley), Benjamin David Haeffele
OptimizationComputational EfficiencyTransformerImageTextSequential
🎯 What it does: This paper proposes the Token Statistics Transformer (TOST), which uses a white-box design based on the MCR-2 variational objective to construct a Token Statistics Self-Attention (TSSA) module that does not require pairwise similarity calculations and has a linear growth in computational and memory complexity, replacing traditional self-attention to achieve an efficient Transformer.
Token-Supervised Value Models for Enhancing Mathematical Problem-Solving Capabilities of Large Language Models
Jung Hyun Lee (NAVER Cloud), Kang Min Yoo (NAVER Cloud)
OptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
🎯 What it does: A new 'Token-Supervised Value Model (TVM)' is proposed and validated as a verifier for solving mathematical problems, utilizing the probability of each token reaching the correct answer for direct value assessment, thereby significantly improving the problem-solving accuracy of LLMs in tree search reasoning.
TokenFormer: Rethinking Transformer Scaling with Tokenized Model Parameters
Haiyang Wang (Peking University), Bernt Schiele (Max Planck Institute for Informatics)
TransformerLarge Language ModelImageText
🎯 What it does: Tokenformer is proposed, utilizing attention to treat model parameters as scalable tokens, replacing traditional linear projections, and supporting gradual model expansion without the need for retraining from scratch.
TOMATO: Assessing Visual Temporal Reasoning Capabilities in Multimodal Foundation Models
Ziyao Shangguan (Yale University), Arman Cohan (Allen Institute for AI)
TransformerLarge Language ModelVideoMultimodalityBenchmark
🎯 What it does: A new video understanding benchmark called TOMATO is proposed and constructed, specifically to evaluate the visual temporal reasoning capabilities of multimodal foundational models.
Tool-Planner: Task Planning with Clusters across Multiple Tools
Yanming Liu (Zhejiang University), Tianyu Du (Zhejiang University)
TransformerLarge Language ModelPrompt EngineeringBenchmark
🎯 What it does: A task planning framework called Tool-Planner based on tool clustering is proposed for more efficient invocation and error correction of external APIs in large language models.
ToolACE: Winning the Points of LLM Function Calling
Weiwen Liu (Shanghai Jiao Tong University), Enhong Chen (University of Science and Technology of China)
Large Language ModelReinforcement LearningTextBenchmark
🎯 What it does: ToolACE is proposed—a fully automated data generation pipeline designed to construct accurate, complex, and rich function call data to enhance the tool usage capabilities of large language models (LLMs).
ToolDial: Multi-turn Dialogue Generation Method for Tool-Augmented Language Models
Jeonghoon Shim (Seoul National University), Yohan Jo (Seoul National University)
GenerationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Constructed and released the ToolDial dataset, which contains 11,111 multi-turn dialogues for evaluating Tool-Augmented Language Models (TALM) in real-world scenarios regarding dialogue state tracking, action prediction, and answer credibility.
ToolGen: Unified Tool Retrieval and Calling via Generation
Renxi Wang (LibrAI), Haonan Li (Mohamed bin Zayed University of Artificial Intelligence)
GenerationRetrievalTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: Directly embed tool knowledge into the LLM vocabulary, using virtual tokens to achieve a unified generation process for tool retrieval and invocation;
TOP-ERL: Transformer-based Off-Policy Episodic Reinforcement Learning
Ge Li (Karlsruhe Institute of Technology), Gerhard Neumann (Karlsruhe Institute of Technology)
Robotic IntelligenceTransformerReinforcement LearningSequential
🎯 What it does: We propose TOP-ERL, an offline episodic reinforcement learning algorithm that uses a Transformer as a critic, capable of efficiently updating N-step returns under the premise of generating the entire action sequence at once.
TopoDiffusionNet: A Topology-aware Diffusion Model
Saumya Gupta (Stony Brook University), Chao Chen (Stony Brook University)
GenerationData SynthesisDiffusion modelImage
🎯 What it does: This paper proposes TopoDiffusionNet (TDN), a method that guides diffusion models to generate images with specified Betti numbers (connected components or holes) through topological constraints.
TopoGaussian: Inferring Internal Topology Structures from Visual Clues
Xiaoyu Xiong (Tsinghua University), Tao Du (University of Massachusetts Amherst)
GenerationOptimizationGaussian SplattingImageVideoPoint Cloud
🎯 What it does: A complete meshless pipeline based on Gaussian Splatting and particle differential simulation has been developed to infer the internal topology of opaque objects from multi-view images and videos, achieving physically consistent motion matching and manufacturable 3D printing models.
Topograph: An Efficient Graph-Based Framework for Strictly Topology Preserving Image Segmentation
Laurin Lux (Technical University of Munich), Johannes C. Paetzold (Weill Cornell Medicine)
SegmentationConvolutional Neural NetworkImage
🎯 What it does: A component graph-based graphical framework is proposed to enforce topological consistency during the training of segmentation networks, which can efficiently identify and correct topological errors.
TopoLM: brain-like spatio-functional organization in a topographic language model
Neil Rathi (EPFL), Martin Schrimpf (EPFL)
TransformerLarge Language ModelTextBiomedical DataMagnetic Resonance Imaging
🎯 What it does: We propose and train TopoLM, a language model that incorporates two-dimensional spatial encoding and spatial smoothness loss into the Transformer, achieving the spatial functional organization of the brain's language system.
Topological Blindspots: Understanding and Extending Topological Deep Learning Through the Lens of Expressivity
Yam Eitan (Technion Israel Institute of Technology), Haggai Maron (Technion Israel Institute of Technology)
Graph Neural NetworkGraphBenchmark
🎯 What it does: This paper first analyzes the expressive power of the Higher-Order Message Passing (HOMP) framework from a topological perspective, proving that it cannot distinguish fundamental topological/metric invariants such as diameter, orientability, planarity, and homology groups. Subsequently, it proposes Multi-Cell Networks (MCN) and its scalable version, Scalable Multi-Cell Networks (SMCN), which achieve full expressiveness by introducing multi-cell covariant layers, significantly enhancing the ability to discern the aforementioned invariants while maintaining computational efficiency. Based on this, three novel CC benchmarks (Torus dataset, cross diameter and second Betti number prediction tasks of Lifted ZINC) are constructed, and comparisons are made with traditional GNNs, HOMP, and other high-expressiveness models on these benchmarks and real graph datasets (ZINC-12K, MOLHIV, MOLESOL). SMCN outperforms the control group in metrics such as MAE, ROC-AUC, and RMSE, achieving a cross diameter prediction accuracy of 92.8% and a second Betti number prediction accuracy of 99.6%.
Topological Schrödinger Bridge Matching
Maosheng Yang (Delft University of Technology)
Graph Neural NetworkSpiking Neural NetworkGraphTime SeriesPhysics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: The Schrödinger bridge problem is defined on a topological space, and a corresponding generative model is constructed to match node/edge signal distributions.
Topological Zigzag Spaghetti for Diffusion-based Generation and Prediction on Graphs
Yuzhou Chen (University of California), Yulia Gel
GenerationDrug DiscoveryGraph Neural NetworkDiffusion modelGraphTime SeriesBiomedical Data
🎯 What it does: This paper proposes a new method for graph diffusion generation and prediction, centered around a 'zigzag spaghetti (ZS)' topological summary constructed based on zigzag persistence, which captures high-order topological features of multi-scale temporal evolution graphs and embeds them into the graph diffusion model.
TopoNets: High performing vision and language models with brain-like topography
Mayukh Deb (Georgia Tech), Apurva Ratan Murty (Georgia Tech)
ClassificationGenerationCompressionConvolutional Neural NetworkTransformerLarge Language ModelVision Language ModelImageText
🎯 What it does: A loss function named TopoLoss is proposed and implemented, enabling visual (ResNet-18/50, ViT-b32) and language (GPT-Neo-125M, NanoGPT) models to form spatial topological structures similar to the brain cortex during training, while maintaining or even improving classification or language generation performance.
TorchTitan: One-stop PyTorch native solution for production ready LLM pretraining
Wanchao Liang (Meta), Stratos Idreos (Harvard University)
TransformerLarge Language ModelText
🎯 What it does: The TORCHTITAN framework is proposed, which unifies and extends the native distributed training technology of PyTorch, creating a one-stop solution for LLM pre-training.
ToVE: Efficient Vision-Language Learning via Knowledge Transfer from Vision Experts
Yuanchen Wu (Shanghai University), Xiaoqiang Li (Tencent)
Computational EfficiencyKnowledge DistillationRepresentation LearningTransformerMixture of ExpertsVision Language ModelImageTextMultimodality
🎯 What it does: The TOVE framework is proposed, achieving efficient visual language learning by transferring the knowledge of pre-trained visual experts to the visual tokens of the CLIP visual encoder.
Toward Efficient Multi-Agent Exploration With Trajectory Entropy Maximization
Tianxu Li (Nanjing University of Aeronautics and Astronautics), Kun Zhu (Nanjing University of Aeronautics and Astronautics)
OptimizationRobotic IntelligenceReinforcement LearningContrastive LearningSequentialBenchmark
🎯 What it does: A multi-agent exploration method TEE based on trajectory entropy maximization is proposed, which is integrated as an intrinsic reward into MARL, significantly enhancing multi-agent exploration and cooperation performance.
Toward Exploratory Inverse Constraint Inference with Generative Diffusion Verifiers
Runyi Zhao (Chinese University of Hong Kong), Guiliang Liu (Chinese University of Hong Kong)
Robotic IntelligenceReinforcement LearningDiffusion modelContrastive LearningSequential
🎯 What it does: Learn diverse and feasible sets of constraints from offline demonstration data to satisfy expert behavior.
Toward Generalizing Visual Brain Decoding to Unseen Subjects
Xiangtao Kong (Hong Kong Polytechnic University), Lei Zhang (Hong Kong Polytechnic University)
RetrievalConvolutional Neural NetworkTransformerContrastive LearningImageVideoMagnetic Resonance Imaging
🎯 What it does: This paper constructs a large dataset containing 177 subjects and a total of 3,127 image-fMRI pairs, and proposes a unified whole-brain decoding framework capable of retrieving visual information from unseen subjects.
Toward Guidance-Free AR Visual Generation via Condition Contrastive Alignment
Huayu Chen (Tsinghua University), Jun Zhu (Tsinghua University)
GenerationTransformerContrastive LearningImage
🎯 What it does: Condition Contrastive Alignment (CCA) is proposed, which significantly improves image generation quality without guidance (no CFG) through a fine-tuning process on a pre-trained autoregressive visual generative model, while halving the sampling cost.
Toward Understanding In-context vs. In-weight Learning
Bryan Chan (University of Alberta), Dale Schuurmans (Google DeepMind)
TransformerLarge Language ModelText
🎯 What it does: This paper studies the emergence and disappearance mechanisms of 'in-context learning' (ICL) and 'weight learning' (IWL) in large language models, and proposes a theoretical framework to explain their distributional dependencies.
Towards a Complete Logical Framework for GNN Expressiveness
Tuo Xu (Independent Researcher)
Graph Neural NetworkGraph
🎯 What it does: A framework is proposed to map any aggregation-combination based Graph Neural Network (GACNN) to an equivalent logical formula, systematically evaluating its expressive power.
Towards a General Time Series Anomaly Detector with Adaptive Bottlenecks and Dual Adversarial Decoders
Qichao Shentu (East China Normal University), Chenjuan Guo (East China Normal University)
Anomaly DetectionTransformerAuto EncoderGenerative Adversarial NetworkTime Series
🎯 What it does: A general-purpose time series anomaly detection model DADA has been developed, which can be directly applied to multi-domain target datasets in zero-shot scenarios.
Towards a learning theory of representation alignment
Francesco Insulla (Stanford University), Lorenzo Rosasco (Massachusetts Institute of Technology)
Representation LearningReview/Survey Paper
🎯 What it does: This paper reviews and unifies various alignment metrics (kernel alignment, distance alignment, independence tests, etc.), proposes a stitching framework based on learning theory, and provides the theoretical relationship between stitching error and kernel alignment.
Towards a Theoretical Understanding of Synthetic Data in LLM Post-Training: A Reverse-Bottleneck Perspective
Zeyu Gan (Renmin University of China), Yong Liu (Renmin University of China)
GenerationData SynthesisLarge Language ModelText
🎯 What it does: This study investigates the role of synthetic data in post-training of LLMs, establishes a distribution model for synthetic data generation, proposes an inverse bottleneck framework, and derives an upper bound on generalization error using information theory.
Towards a Unified and Verified Understanding of Group-Operation Networks
Wilson Wu (University of Colorado Boulder), Jason Gross (Independent)
Explainability and InterpretabilityComputational EfficiencyGraph
🎯 What it does: This study explores the internal structure of single hidden layer neural networks trained on binary operation tasks in finite groups, revealing previously unrecognized structures and providing a more complete description of these models, aiming to unify the explanations of previous work.
Towards Auto-Regressive Next-Token Prediction: In-context Learning Emerges from Generalization
Zixuan Gong (Renmin University of China), Yong Liu (Renmin University of China)
GenerationOptimizationTransformerLarge Language ModelTextStochastic Differential Equation
🎯 What it does: A full-process pre-training and ICL framework based on the autoregressive next word prediction (AR-NTP) paradigm is proposed, and a two-layer expected PAC-Bayesian generalization upper bound is provided, proving that the ICL capability arises from the generalization of sequences and topics.
Towards Automated Knowledge Integration From Human-Interpretable Representations
Kasia Kobalczyk, Mihaela van der Schaar (University of Cambridge)
Meta LearningLarge Language ModelImageTime Series
🎯 What it does: This paper proposes and implements an automated knowledge integration meta-learning framework—Informed Meta-Learning (IML), and demonstrates how to automatically inject human-interpretable knowledge, such as natural language, into machine learning models using the Informed Neural Process (INP) as an example.
Towards Bridging Generalization and Expressivity of Graph Neural Networks
Shouheng Li (Australian National University), Qing Wang (University of Antwerp)
Graph Neural NetworkGraph
🎯 What it does: This paper studies the relationship between the expressiveness and generalization of graph neural networks, proposing a generalization bound based on k-variance margins;
Towards Calibrated Deep Clustering Network
Yuheng Jia (Southeast University), Junhui Hou (City University of Hong Kong)
ClassificationRepresentation LearningContrastive LearningImage
🎯 What it does: A dual-head deep clustering network (CDC) is proposed, which simultaneously calibrates the confidence of the model output during the unsupervised clustering process and dynamically selects high-confidence samples for pseudo-label self-training using the calibrated confidence, significantly improving clustering accuracy and the reliability of confidence.
Towards Certification of Uncertainty Calibration under Adversarial Attacks
Cornelius Emde (University of Oxford), Adel Bibi (University of Oxford)
OptimizationAdversarial AttackConvolutional Neural NetworkImage
🎯 What it does: A provably secure framework for model confidence calibration under adversarial attacks is proposed and studied, providing worst-case bounds for Brier score and ECE, and implementing approximate calibration certification and adversarial calibration training based on mixed-integer programming.
Towards Continuous Reuse of Graph Models via Holistic Memory Diversification
Ziyue Qiao (Great Bay University), Hui Xiong (Hong Kong University of Science and Technology)
Graph Neural NetworkGraph
🎯 What it does: This paper proposes an incremental learning method DMSG for the ever-growing graph structured data, which can continuously train the model when new tasks arise while retaining knowledge from old tasks.
Towards counterfactual fairness through auxiliary variables
Bowei Tian (University of Maryland), Ang Li (University of Maryland)
Tabular
🎯 What it does: A causal framework EXOC is proposed that utilizes auxiliary nodes (S') and control nodes (S'') to achieve counterfactual fairness for sensitive attributes while maintaining prediction accuracy.
Towards Domain Adaptive Neural Contextual Bandits
Ziyan Wang (Georgia Institute of Technology), Hao Wang (Rutgers University)
Domain AdaptationRecommendation SystemReinforcement LearningImage
🎯 What it does: A cross-domain contextual bandwidth algorithm DABand based on deep representation is proposed, utilizing feedback from a low-cost source domain to achieve zero-shot learning in a high-cost target domain.
Towards Effective Evaluations and Comparisons for LLM Unlearning Methods
Qizhou Wang (Hong Kong Baptist University), Masashi Sugiyama (RIKEN Center for Advanced Intelligence Project)
Large Language ModelText
🎯 What it does: This paper proposes a unified evaluation framework UWC to measure the 'unlearning' effect of large language models and to fairly compare existing unlearning methods.
Towards Empowerment Gain through Causal Structure Learning in Model-Based Reinforcement Learning
Hongye Cao (Nanjing University), Yang Gao (Nanjing University)
Robotic IntelligenceReinforcement LearningSequential
🎯 What it does: A framework named Empowerment through Causal Learning (ECL) has been developed, which combines empowerment-driven exploration with causal structure learning for efficient control and task learning in model-based reinforcement learning.
Towards Explaining the Power of Constant-depth Graph Neural Networks for Structured Linear Programming
Qian Li (Shenzhen International Center for Industrial and Applied Mathematics), Ruoyu Sun (School of Data Science, The Chinese University of Hong Kong)
OptimizationGraph Neural NetworkGraph
🎯 What it does: This paper proves that graph neural networks (GNNs) with constant depth and constant width can effectively solve sparse binary linear programming problems, and provides an implementation and experimental validation based on distributed algorithms.
Towards Fast, Specialized Machine Learning Force Fields: Distilling Foundation Models via Energy Hessians
Ishan Amin (University of California), Aditi S. Krishnapriyan (University of California)
Computational EfficiencyKnowledge DistillationGraph Neural NetworkBiomedical Data
🎯 What it does: The study proposes a method for transferring representations from large-scale foundational models (FM) to smaller, faster specialized machine learning force fields (MLFF) through knowledge distillation of the energy Hessian matrix, significantly accelerating inference while maintaining or improving accuracy.
Towards Faster Decentralized Stochastic Optimization with Communication Compression
Rustem Islamov (Universität Basel), Sebastian U Stich
OptimizationConvolutional Neural NetworkImageTabular
🎯 What it does: A decentralized compression communication algorithm named MoTEF is proposed, which combines momentum tracking and error feedback to achieve optimal non-convex stochastic optimization convergence rates under arbitrary data heterogeneity.
Towards Federated RLHF with Aggregated Client Preference for LLMs
Feijie Wu (Purdue University), Jing Gao (State University of New York at Albany)
Federated LearningReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningTextBenchmark
🎯 What it does: This study proposes the use of federated learning to collect diverse user preferences and implements RLHF alignment for large language models through a binary selector, constructing two frameworks: FedBis and FedBiscuit.
Towards Foundation Models for Mixed Integer Linear Programming
Sirui Li (Massachusetts Institute of Technology), Beibin Li (Microsoft Research)
OptimizationGraph Neural NetworkLarge Language ModelContrastive LearningTextChain-of-Thought
🎯 What it does: A LLM-based MILP-Evolve evolutionary framework is proposed to generate a vast and diverse range of MILP classes, and on this basis, a single GNN+attention-based model is trained, which can achieve multi-class generalization in three major tasks: predicting the integrality gap of mixed-integer linear programming, branch learning, and MILP-text alignment.
Towards General-Purpose Model-Free Reinforcement Learning
Scott Fujimoto (Meta Platforms), Michael Rabbat (Meta Platforms)
Reinforcement LearningTabularBenchmark
🎯 What it does: A general model-free reinforcement learning algorithm MR.Q is designed, which utilizes model-based representation learning to generate approximately linear value function state-action embeddings, and is applicable to various observation and action spaces under a single hyperparameter setting.
Towards Generalizable Reinforcement Learning via Causality-Guided Self-Adaptive Representations
Yupei Yang (Shanghai Jiao Tong University), Lei Xu (Shanghai Jiao Tong University)
Representation LearningReinforcement LearningWorld ModelSequential
🎯 What it does: This paper proposes an interpretable causal self-adaptive representation method (CSR) for rapid transfer and adaptation of reinforcement learning models in the scenarios of distribution changes and environmental space expansion.
Towards Generalization Bounds of GCNs for Adversarially Robust Node Classification
Wen Wen (Huazhong Agricultural University), Hong Chen (Xi'an Jiaotong University)
ClassificationAdversarial AttackGraph Neural NetworkGraph
🎯 What it does: This paper studies the adversarial robust generalization of Graph Convolutional Networks (GCN) under node classification tasks and provides a high-probability theoretical upper bound.
Towards Hierarchical Rectified Flow
Yichi Zhang (University of Illinois Urbana Champaign), Zhizhen Zhao (University of Illinois Urbana Champaign)
GenerationData SynthesisComputational EfficiencyFlow-based ModelRectified FlowImageMultimodalityStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Proposes a Hierarchical Reversible Flow (HRF) model that utilizes multi-level ODEs to capture multimodal velocity/acceleration distributions, achieving crossable paths and straighter paths, thereby improving sampling efficiency.
Towards Homogeneous Lexical Tone Decoding from Heterogeneous Intracranial Recordings
Di Wu (Westlake University), Mohamad Sawan (Westlake University)
TransformerAuto EncoderBiomedical Data
🎯 What it does: A framework for cross-subject unified brain speech tone decoding, H2DiLR, is proposed, which can separate homogeneous and heterogeneous information from heterogeneous EEG recordings.
Towards hyperparameter-free optimization with differential privacy
Ruixuan Liu (Emory University), Zhiqi Bu (Amazon)
OptimizationSafty and PrivacyTransformerLarge Language ModelImageText
🎯 What it does: A hyperparameter-free differential privacy optimization framework called HyFreeDP is proposed, which automatically adjusts the learning rate and uses automatic gradient clipping based on privacy loss estimation.
Towards Improving Exploration through Sibling Augmented GFlowNets
Kanika Madan (Mila Quebec AI Institute), Yoshua Bengio (Mila Quebec AI Institute)
Drug DiscoveryReinforcement LearningFlow-based ModelGenerative Adversarial Network
🎯 What it does: This paper proposes and implements Sibling Augmented Generative Flow Networks (SA-GFN), which significantly improves the exploration efficiency and training convergence speed of GFlowNet by using independent exploration sub-networks and a behavior main network, along with RND intrinsic rewards to relabel trajectories.
Towards Interpreting Visual Information Processing in Vision-Language Models
Clement Neo (Nanyang Technological University), Fazl Barez
Object DetectionExplainability and InterpretabilityTransformerVision Language ModelImage
🎯 What it does: This paper studies the processing of visual input in visual language models (VLMs) and finds that object information is highly localized to the corresponding visual tokens, which are gradually mapped to interpretable text embeddings at different layers.
Towards Learning High-Precision Least Squares Algorithms with Sequence Models
Jerry Weihong Liu, Christopher Re
OptimizationTransformerSequential
🎯 What it does: This paper explores whether sequence models can learn high-precision numerical algorithms (gradient descent) for the least squares problem and evaluates their performance from the perspectives of numerical accuracy and generality.
Towards Marginal Fairness Sliced Wasserstein Barycenter
Khai Nguyen (University of Texas at Austin), Nhat Ho (University of Texas at Austin)
OptimizationImagePoint Cloud
🎯 What it does: Proposed and solved the 'marginally fair sliced Wasserstein barycenter' problem, constraining the distances between each margin to be similar.
Towards Multiple Character Image Animation Through Enhancing Implicit Decoupling
Jingyun Xue (Sun Yat-sen University), Wenhan Luo (HKUST)
GenerationData SynthesisPose EstimationDiffusion modelOptical FlowVideoBenchmark
🎯 What it does: A multi-condition guided diffusion framework is proposed to address the multi-character image animation problem, capable of achieving stable and high-quality pose-controlled animations in complex backgrounds and multi-character scenes.
Towards Neural Scaling Laws for Time Series Foundation Models
Qingren Yao (Hong Kong University of Science and Technology), Shirui Pan (Griffith University)
TransformerTime Series
🎯 What it does: This study explores the neural scaling laws of Time Series Fundamental Models (TSFMs), focusing on the scaling behavior of encoder and decoder Transformers on ID and OOD data.
Towards Optimal Multi-draft Speculative Decoding
Zhengmian Hu (University of Maryland), Heng Huang (University of Maryland)
GenerationOptimizationTransformerLarge Language ModelText
🎯 What it does: This study investigates the acceptance rate of Multi-Draft Speculative Decoding (MDSD), proposing to transform the optimal acceptance rate into a subset selection problem and measuring the upper limit on real text distributions, validating the gap between existing algorithms and theory, and suggesting a greedy draft construction to achieve a higher acceptance rate.
Towards Out-of-Modal Generalization without Instance-level Modal Correspondence
Zhuo Huang (Sydney AI Centre), Tongliang Liu (Sydney AI Centre)
ClassificationRetrievalDomain AdaptationContrastive LearningImageVideoTextMultimodality
🎯 What it does: This paper proposes a cross-modal generalization framework COX that does not require instance-level modal correspondence, aiming to infer the labels of unknown modalities using the knowledge of known modalities.
Towards Principled Evaluations of Sparse Autoencoders for Interpretability and Control
Aleksandar Makelov, Neel Nanda
Explainability and InterpretabilityTransformerLarge Language ModelAuto EncoderText
🎯 What it does: This paper proposes a principled evaluation framework based on a supervised feature dictionary to measure the decoupling and controllability of Sparse Autoencoders (SAE) in large language models for key tasks such as Indirect Object Identification (IOI). The authors construct a supervised dictionary on GPT-2 Small and then use this dictionary as a benchmark to evaluate the reconstruction sufficiency, necessity, sparse control effects, and interpretability of SAE.
Towards Realistic Data Generation for Real-World Super-Resolution
Long Peng (University of Science and Technology of China), Zheng-Jun Zha (University of Science and Technology of China)
RestorationGenerationData SynthesisSuper ResolutionDiffusion modelContrastive LearningImage
🎯 What it does: This paper proposes an unsupervised RealDGen framework that utilizes a diffusion model decoupled from content and degradation to generate large-scale, realistic, and adaptable paired data from unpaired real low-resolution (LR) and high-resolution (HR) images, thereby enhancing the generalization performance of image super-resolution models in real-world scenarios.
Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodology
Xiangyu Wang (Beihang University), Si Liu (Beihang University)
Object DetectionAutonomous DrivingRobotic IntelligenceTransformerLarge Language ModelReinforcement LearningVision Language ModelTextMultimodalityBenchmark
🎯 What it does: A realistic drone vision-language navigation platform TRAVEL, a goal-oriented dataset, and the UAV-Need-Help evaluation benchmark are proposed, along with the development of a hierarchical trajectory generation navigation model based on multimodal LLM.
Towards Robust Alignment of Language Models: Distributionally Robustifying Direct Preference Optimization
Junkang Wu (University of Science and Technology of China), Xiangnan He (University of Science and Technology of China)
Recommendation SystemOptimizationTransformerSupervised Fine-TuningReinforcement LearningText
🎯 What it does: This study investigates the robustness of Direct Preference Optimization (DPO) under point-to-point and pairwise noise, and proposes the Distributionally Robustifying DPO (Dr.DPO) framework, which enhances resistance to pairwise noise while maintaining point-to-point robustness through an additional hyperparameter β′.
Towards Robust and Parameter-Efficient Knowledge Unlearning for LLMs
Sungmin Cha (New York University), Moontae Lee (LG AI Research)
TransformerLarge Language ModelText
🎯 What it does: This paper studies the problem of knowledge unlearning in large language models and proposes a low-rank knowledge unlearning framework called LoKU, which addresses issues of instability and excessive forgetting associated with traditional gradient ascent (GA) methods.