ICML 2026 Papers — Page 60
International Conference on Machine Learning · 6554 papers
TN-SHAP-G: Graph-Structured Tensor Network Surrogates for Shapley Values and Interactions
Farzaneh Heidari (Universit´ e de Montr´ eal), Guillaume Rabusseau (Universit´ e de Montr´ eal)
Explainability and InterpretabilityDrug DiscoveryGraph Neural NetworkGraphBenchmark
🎯 What it does: Construct a graph-structured aligned tensor network (TN) surrogate model to learn the multilinear extension of masked graph games from limited black-box model queries, then deterministically recover Shapley values and their interaction terms directly through polynomial interpolation and closed-form integration.
To Grok Grokking: Provable Grokking in Ridge Regression
Mingyue Xu (Purdue University), Itay Safran (Ben-Gurion University of the Negev)
OptimizationRepresentation LearningHyperparameter SearchTabular
🎯 What it does: Studied the 'grokking' phenomenon that occurs when training over-parameterized linear models with gradient descent in ridge regression with weight decay, and provided theoretical guarantees for the slow transition from training error to test error.
ToaSt: Token Channel Selection and Structured Pruning for Efficient ViT
Hyunchan Moon (LG Electronics), Steven L. Waslander (University of Toronto)
ClassificationObject DetectionSegmentationComputational EfficiencyKnowledge DistillationTransformerSupervised Fine-TuningAuto EncoderContrastive LearningImage
🎯 What it does: This paper proposes the ToaSt framework, which decouples the compression of Vision Transformer (ViT) by applying structured paired weight pruning to the multi-head self-attention (MHSA), and using training-agnostic Token Channel Selection (TCS) for channel-level compression in the FFN.
TodoEvolve: Learning to Architect Agent Planning Systems
Jiaxi Liu (Shanghai Jiao Tong University), Junchi Yan (Shanghai Jiao Tong University)
Autonomous DrivingOptimizationRobotic IntelligenceMeta LearningAI Code AssistantTransformerLarge Language ModelReinforcement LearningAgentic AIPrompt EngineeringTextSequentialBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: This paper proposes TodoEvolve, a meta-planning framework capable of autonomously synthesizing and dynamically updating task-specific planning architectures.
Token Sample Complexity of Attention
Léa Bohbot (Paris Sciences et Lettres University), François-Xavier Vialard (Gustave Eiffel University)
Explainability and InterpretabilityComputational EfficiencyRepresentation LearningTransformerText
🎯 What it does: Study the attention convergence behavior of Transformer in extremely long sequences, and propose the concept of token sample complexity.
Token Sparse Attention: Efficient Long-Context Inference with Interleaved Token Selection
Dongwon Jo (Seoul National University), Jae-Joon Kim (Seoul National University)
Computational EfficiencyTransformerLarge Language ModelText
🎯 What it does: Proposes Token Sparse Attention, a dynamic and reversible token-level sparse attention mechanism, which maintains attention to all tokens by compressing Q/K/V at each layer and decompressing back to the full sequence.
Token-Efficient Change Detection in LLM APIs
Timothee Chauvin, Gilles Tredan (LAAS)
Anomaly DetectionExplainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelPrompt EngineeringContrastive LearningTextBenchmark
🎯 What it does: Studies the black-box change detection of LLM APIs, proposing the B3IT method, which achieves low-cost, high-sensitivity continuous monitoring by identifying and monitoring boundary inputs (Border Inputs) under low-temperature conditions.
Token-Free Hierarchical Indexing for RAG beyond LLM-based Summarization
Yifan Wei (Beihang University), Angsheng Li (Beihang University)
RetrievalComputational EfficiencyRepresentation LearningGraph Neural NetworkTransformerPrompt EngineeringContrastive LearningTextGraphBenchmarkRetrieval-Augmented Generation
🎯 What it does: Propose SeRAG, a token-free hierarchical indexing framework that constructs a knowledge tree for RAG using a structured graph guided by structural entropy, avoiding the high cost of traditional LLM summaries.
Token-Level LLM Collaboration via FusionRoute
Nuoya Xiong (Meta), Zhuokai Zhao (Meta AI)
OptimizationFederated LearningComputational EfficiencyAI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningMixture of ExpertsTextBenchmark
🎯 What it does: Proposed the FUSIONROUTE framework, which实现了 token-level selection of specialized LLMs at each decoding step, and provided complementary logits for correction through a lightweight router;
Token-Sparse Medical Multimodal Reasoning via Dual-Stream Reinforcement Learning
Kaitao Chen (Fudan University), Mianxin Liu (Shanghai Artificial Intelligence Laboratory)
Explainability and InterpretabilityComputational EfficiencyRepresentation LearningTransformerReinforcement LearningPrompt EngineeringVision Language ModelContrastive LearningMultimodalityBiomedical DataBenchmark
🎯 What it does: To address the visual information redundancy in medical multimodal reasoning, we propose the ViToS dual-stream reinforcement learning framework, which simultaneously learns visual localization and sparse visual token reasoning on the same policy model.
TokenDrop: Token-Level Importance-Aware Backward Propagation Skipping for Efficient LLM Fine-Tuning
Beomseok Kim (Seoul National University), Dongsuk Jeon (Seoul National University)
Computational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Propose TokenDrop, a backward propagation skipping method based on token importance evaluation, significantly reducing activation memory and computational cost during LLM fine-tuning.
Tokenised Flow Matching for Hierarchical Simulation Based Inference
Giovanni Charles (Imperial College London), Elizaveta Semenova (Imperial College London)
OptimizationFederated LearningComputational EfficiencyRepresentation LearningTransformerDiffusion modelScore-based ModelFlow-based ModelContrastive LearningTabularTime SeriesSequentialBenchmarkPhysics Related
🎯 What it does: The paper proposes a hierarchical simulation inference framework based on tokenized flow matching (TFMPE), which utilizes likelihood factorization (LF) to train with a single-site simulator, and subsequently synthesizes multi-site observations for posterior estimation.
TokenRatio: Principled Token-Level Preference Optimization via Ratio Matching
Truong Nguyen (Hanoi University of Science and Technology), Trung Le (Monash University)
OptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningContrastive LearningText
🎯 What it does: Propose Token-level Bregman Preference Optimization (TBPO), achieving preference alignment for language models through token-level Bradley-Terry models and Bregman density ratio matching.
TokenSwap: Backdoor Attack on the Compositional Understanding of Large Vision-Language Models
Zhifang Zhang (Southeast University), Joey Tianyi Zhou (Agency for Science, Technology and Research)
Safty and PrivacyAdversarial AttackTransformerSupervised Fine-TuningPrompt EngineeringVision Language ModelImageTextMultimodality
🎯 What it does: This paper proposes a backdoor attack method called TokenSwap, targeting the compositional understanding ability of large vision-language models, making the model swap the subject and object positions in the image when triggered, resulting in outputs inconsistent with real scenarios.
TokSuite: Measuring the Impact of Tokenizer Choice on Language Model Behavior
Gül Sena Altıntaş (University of Toronto), Colin Raffel (University of Toronto)
Explainability and InterpretabilityComputational EfficiencyRepresentation LearningTransformerLarge Language ModelPrompt EngineeringTextMultimodalityBenchmarkRetrieval-Augmented Generation
🎯 What it does: Studied the impact of different tokenizers on the behavior and robustness of multilingual language models, constructing 14 models with identical architectures but different tokenizers and a perturbation benchmark covering five languages
TOM-SWE: User Mental Modeling For Software Engineering Agents
Xuhui Zhou (Carnegie Mellon University), Xingyao Wang (All Hands AI)
AI Code AssistantTransformerLarge Language ModelAgentic AITextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Propose a dual-agent architecture called ToM-SWE, pairing a specialized Theory-of-Mind (ToM) agent with a Software Engineering (SWE) agent. The ToM agent continuously maintains the user's intent, preferences, and emotions, enabling the SWE agent to generate more accurate code and interact more effectively.
ToMAP: Training Opponent-Aware LLM Persuaders with Theory of Mind
Peixuan Han (University Of Illinois Urbana Champaign), Jiaxuan You (University Of Illinois Urbana Champaign)
Reinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: Train an LLM model named ToMAP that can autonomously generate persuasive arguments in multi-turn dialogues based on the opponent's psychological state.
ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestration
Hongjin SU, Pavlo Molchanov (NVIDIA)
Computational EfficiencyData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextTabularBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Train and deploy an 8B-parameter Orchestrator model, which dynamically invokes diverse tools and sub-models during multi-round interactions using reinforcement learning to accomplish complex reasoning tasks;
TopAdapter: Topology-Aware Prompt Tuning for Efficient Point Cloud Understanding
Changshuo Wang (University College London), Prayag Tiwari (Halmstad University)
ClassificationRecognitionComputational EfficiencyTransformerPrompt EngineeringContrastive LearningPoint Cloud
🎯 What it does: Proposes TopAdapter, a parameter-efficient fine-tuning framework that leverages topological information, specifically designed for point cloud understanding tasks;
TopBench: A Benchmark for Implicit Predictive Reasoning in Tabular Question Answering
An-Yang Ji (Nanjing University), Han-Jia Ye (Nanjing University)
Recommendation SystemAutonomous DrivingFederated LearningExplainability and InterpretabilityComputational EfficiencyDrug DiscoveryAI Code AssistantReinforcement Learning from Human FeedbackNeural Architecture SearchProtein Structure PredictionTransformerLarge Language ModelAgentic AIPrompt EngineeringTextTabularBenchmarkFinance RelatedRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Propose the TOPBENCH benchmark to evaluate the ability of LLMs in implicitly predicting reasoning in table-based question answering.
TopoDistill: Distilling Global System Topology for Causal Discovery in Multivariate Time Series
Zehao Liu (Hangzhou Dianzi University), Yuyu Yin (Hangzhou Dianzi University)
Knowledge DistillationRepresentation LearningContrastive LearningTime Series
🎯 What it does: This paper proposes the TopoDistill framework, which improves single-variable delay embedding through knowledge distillation and temporal contrastive learning, making the shadow manifold smoother in noisy environments and enhancing the robustness of CCM causal discovery.
Topological Active Inference for Task Disambiguation
Yangbo Wei (Shanghai Jiao Tong University), Lei He (Eastern Institute of Technology)
ClassificationRecommendation SystemExplainability and InterpretabilityReinforcement Learning from Human FeedbackTransformerLarge Language ModelContrastive LearningTextBenchmark
🎯 What it does: Propose a topology-based active inference framework (Topological Active Inference, TAI), which identifies and clusters real intentions by persistent homology after semantically embedding candidate solutions generated from LLMs, thus raising the most information-gain clarifying questions during the interaction process, significantly reducing semantic blind spots, and improving the efficiency of user intention recognition.
Topology-Aware Contrastive Learning: Regulating Representation Connectivity via Persistent Homology
Jiaxin Sun (Shanxi University), Yang Wang (Shanxi University)
ClassificationRepresentation LearningConvolutional Neural NetworkGraph Neural NetworkContrastive LearningImage
🎯 What it does: Propose a topology-aware contrastive learning framework based on persistent homology, replacing pure geometric convergence with topological connectivity.
Topology-Preserving Neural Operator Learning via Hodge Decomposition
Dongzhe Zheng (Princeton University), Christine Allen-Blanchette (Princeton University)
Graph Neural NetworkMeshGraphPhysics Related
🎯 What it does: This paper proposes a topology-preserving neural operator, HSD, based on Hodge decomposition, to learn physical field operators on surfaces or porous geometric meshes.
Torus Graphs for Large Scale Neural Phase Analysis
Jack Goffinet (Duke University), David Carlson
GenerationData SynthesisExplainability and InterpretabilityComputational EfficiencyRepresentation LearningRecurrent Neural NetworkGraph Neural NetworkDiffusion modelScore-based ModelAuto EncoderContrastive LearningTime SeriesBiomedical DataElectrocardiogramStochastic Differential Equation
🎯 What it does: A scalable Torus Graph (TG) model was constructed and generalized for generative modeling, state-dependent modeling, and directional causal inference of large-scale neural phase data.
Tournament Style RL: Stabilizing Policy Optimization on Non Verifiable Problems
Gurusha Juneja (University of California, Santa Barbara), William Yang Wang (University of California, Santa Barbara)
GenerationData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningContrastive LearningText
🎯 What it does: Designed Tournament Style RL (TSRL), which trains language models by comparing with a fixed set of anchor points over multiple rounds, using win rate as a reward to address reward noise in non-verification tasks.
Toward Calibrated Mixture-of-Experts Under Distribution Shift
Gina Wong (Johns Hopkins University), Anqi Liu (Johns Hopkins University)
ClassificationDomain AdaptationExplainability and InterpretabilityComputational EfficiencyAdversarial AttackMixture of ExpertsContrastive LearningImageTextMultimodality
🎯 What it does: Study the calibration issue of Mixture-of-Experts (MoE) under distribution shift, and propose an adversarial reweighting training scheme based on entropy balancing to improve the overall prediction calibration.
Toward Culturally Aligned LLMs through Ontology-Guided Multi-Agent Reasoning
Wonduk Seo (Enhans), Yi Bu (Peking University)
Federated LearningExplainability and InterpretabilityData-Centric LearningTransformerLarge Language ModelAgentic AIPrompt EngineeringTextTabularRetrieval-Augmented Generation
🎯 What it does: Propose an ontology-driven multi-agent reasoning framework called OG-MAR, which leverages respondent value summaries from the World Values Survey (WVS) and ontological relationships to achieve culturally aligned LLM reasoning.
Toward Cybersecurity-Expert Small Language Models
Matan Levi (IBM Research), Yair Allouche (IBM Research)
Safty and PrivacyExplainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelPrompt EngineeringTextBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Propose the CyberPal 2.0 series of small language models (4B–20B) and develop the SecKnowledge 2.0 data augmentation pipeline, focusing on cybersecurity reasoning and decision-making;
Toward Effective Multimodal Graph Foundation Model: A Divide-and-Conquer Based Approach
Sicheng Liu (Beijing Institute of Technology), Guoren Wang (Beijing Institute of Technology)
ClassificationGenerationComputational EfficiencyRepresentation LearningGraph Neural NetworkTransformerMixture of ExpertsAuto EncoderContrastive LearningImageTextMultimodalityGraph
🎯 What it does: Designed and implemented a foundation model called PLANET for multi-modal graphs, addressing modality interaction and alignment issues through a hierarchical strategy, supporting node classification, link prediction, and generation tasks from graphs to text/images.
Toward Identifiable Sparse Autoencoders
Walter Nelson (Institute of Science and Technology Austria), Francesco Locatello (Institute of Science and Technology Austria)
OptimizationRepresentation LearningAuto EncoderContrastive LearningImageText
🎯 What it does: This study investigates the identifiability of sparse autoencoders (SAE) and proposes an improved scheme, iSAE, to enhance the stability of the dictionary and sparse codes.
Toward Robust Multilingual Adaptation of LLMs for Low-Resource Languages
Haolin Li (Tsinghua University), Biqing Huang (Tsinghua University)
RetrievalDomain AdaptationTransformerLarge Language ModelReinforcement LearningPrompt EngineeringContrastive LearningTextMultimodality
🎯 What it does: Proposes the LiRA framework, combining Arca and LaSR to achieve robust multilingual adaptation for low-resource languages, enhancing the performance of LLMs in retrieval, ranking, and reasoning tasks
Toward Safe Quantization-Aware Fine-tuning: Understanding and Mitigating Safety Alignment Degradation
Yuning Yang (University of Electronic Science and Technology of China), Guisong Liu (Southwestern University of Finance and Economics)
Safty and PrivacyComputational EfficiencyKnowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringContrastive LearningText
🎯 What it does: This paper studies the safety alignment degradation caused by quantization-aware fine-tuning (QLoRA) in large language models and proposes the ExSQF method to restore and enhance safety.
Toward Scalable and Valid Conditional Independence Testing with Spectral Representations
Alek Fröhlich (Istituto Italiano di Tecnologia), Massimiliano Pontil (Istituto Italiano di Tecnologia)
Explainability and InterpretabilityComputational EfficiencyRepresentation LearningContrastive LearningImageTabularBiomedical DataBenchmark
🎯 What it does: Proposed a scalable conditional independence test method called SpectralCIT based on spectral representation learning, which constructs a simple test statistic using the low-rank singular value decomposition of the partial correlation operator, and provides theoretical guarantees for type I error and power.
Toward Stable Value Alignment: Introducing Independent Modules for Consistent Value Guidance
Wenhao Chen (Peking University), Guojie Song (Peking University)
Safty and PrivacyReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningPrompt EngineeringText
🎯 What it does: Propose an independent value module, SVGT, to achieve stable value alignment for large language models.
Toward Structural Multimodal Representations: Specialization, Selection, and Sparsification via Mixture-of-Experts
Hahyeon Choi (Seoul National University), Nojun Kwak (Seoul National University)
Explainability and InterpretabilityComputational EfficiencyRepresentation LearningTransformerMixture of ExpertsContrastive LearningMultimodality
🎯 What it does: Proposed a multi-modal representation learning framework called S3 (Specialization, Selection, Sparsification), which achieves structured processing of multi-modal information by splitting the input into semantic experts, task-oriented routing, and sparsification during inference.
Toward Subspace-Perturbed Trajectory-Aware Backdoor Attacks in Deep Reinforcement Learning
Yaguan Qian (Zhejiang University of Science and Technology), Zhen Lei
Adversarial AttackReinforcement LearningAuto EncoderContrastive LearningImageTime SeriesSequential
🎯 What it does: Propose the SpecDRL framework to achieve stealthy backdoor injection in deep reinforcement learning, addressing three major challenges: trigger location, injection timing, and reward consistency.
Toward Training Superintelligent Software Agents through Self-Play SWE-RL
Yuxiang Wei (Meta FAIR), Sida Wang (Meta FAIR)
Autonomous DrivingOptimizationFederated LearningComputational EfficiencyData-Centric LearningRobotic IntelligenceDrug DiscoveryAI Code AssistantReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningPrompt EngineeringDiffusion modelWorld ModelTextSequentialBenchmark
🎯 What it does: Through self-supervised training, LLMs autonomously generate and fix bugs in warehouse environments without human-labeled data, achieving continuous self-improvement of software agents.
Toward Understanding Adversarial Distillation: Why Robust Teachers Fail
Hongsin Lee (KAIST), Hye Won Chung (KAIST)
Explainability and InterpretabilityKnowledge DistillationAdversarial AttackConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: Propose a theoretical explanation for the failure mechanism of adversarial distillation, pointing out that the teacher's confident supervision on unlearnable samples leads to students memorizing noise, resulting in robust overfitting; and provides a teacher selection criterion based on the prediction entropy of the teacher on unlearnable samples.
Towards A Generative Protein Evolution Machine with DPLM-Evo
Xinyou Wang (Nanjing University), Quanquan Gu (ByteDance Seed)
Drug DiscoveryProtein Structure PredictionTransformerDiffusion modelBiomedical Data
🎯 What it does: Proposed an evolutionary discrete diffusion model, DPLM-Evo, which explicitly models substitution, insertion, and deletion edit operations in protein sequences;
Towards a Holistic Understanding of Selection Bias for Causal Effect Identification
Yiwen Qiu (Institute of Science and Technology Austria), Francesco Locatello (Institute of Science and Technology Austria)
OptimizationFederated LearningExplainability and InterpretabilityComputational EfficiencyData-Centric LearningTabularBiomedical DataElectronic Health Records
🎯 What it does: A unified framework based on distribution classes is proposed, providing necessary and sufficient conditions for the identifiability of the average treatment effect under selection bias, achieving identifiability across various selection mechanisms and distribution families.
Towards a Science of AI Agent Reliability
Stephan Rabanser (Princeton University), Arvind Narayanan (Princeton University)
Safty and PrivacyExplainability and InterpretabilityLarge Language ModelAgentic AITextBenchmark
🎯 What it does: A four-dimensional reliability framework based on safety-critical engineering is proposed, defining 12 indicators, and evaluating the reliability of 15 models on two benchmarks.
Towards a Unified Generative Model for Scarce Time Series with Domain Experts
Zihao Yao (Tongji University), Yaying Zhang (Tongji University)
GenerationData SynthesisTransformerPrompt EngineeringMixture of ExpertsDiffusion modelTime SeriesFinance Related
🎯 What it does: Designed and trained a cross-domain time series generation framework called TimeMoDE, based on diffusion transformers and mixture-of-experts networks, specifically for low-sample environments.
Towards Achieving Optimal Strong Regret and Constraint Violation via Computationally Efficient Model-free RL
Xiyue Peng (ShanghaiTech University), Xin Liu (ShanghaiTech University)
OptimizationComputational EfficiencyReinforcement LearningVision-Language-Action ModelAuto EncoderContrastive LearningTabularChain-of-Thought
🎯 What it does: This paper studies finite-horizon constrained Markov decision processes (CMDP) under linear function approximation and proposes a model-free APMPO algorithm, achieving near-optimal sublinear convergence under strong penalty and strong constraint violation metrics.
Towards Atoms of Large Language Models
Chenhui Hu (Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences), Jun Zhao (Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences)
Explainability and InterpretabilityComputational EfficiencyRepresentation LearningTransformerLarge Language ModelAuto EncoderText
🎯 What it does: Investigate the fundamental representation units of LLMs, propose the Atom Theory, and identify atoms.
Towards Cold-Start Drafting and Continual Refining: A Value-Driven Memory Approach with Application to NPU Kernel Synthesis
Yujie Zheng (Shanghai Jiao Tong University), Muning Wen (MemTensor (Shanghai) Technology Co., Ltd)
OptimizationAI Code AssistantReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningAgentic AITextRetrieval-Augmented Generation
🎯 What it does: Designed and implemented EvoKernel, a self-evolving agentic framework that utilizes value-driven memory retrieval to achieve cold-start drafting and continuous optimization of NPU kernels.
Towards Complete Multi-Agent Coordination Policy Learning via Denoising Maximum Entropy Optimization
Guanghao Li (Nanjing University), Yang Yu (Nanjing University)
OptimizationRecurrent Neural NetworkSpiking Neural NetworkTransformerReinforcement LearningDiffusion modelScore-based ModelAuto EncoderContrastive LearningTabularTime SeriesSequentialBenchmark
🎯 What it does: Designed a complete partitioning strategy for parameter sharing in multi-agent reinforcement learning, combined with a diffusion model to achieve the Dspic algorithm, enabling efficient collaborative learning among multiple agents.
Towards Completeness in Causal Discovery from Soft Interventions with Known Targets
Zihan Zhou (Johns Hopkins University), Murat Kocaoglu (Johns Hopkins University)
Explainability and InterpretabilityComputational EfficiencyRepresentation LearningGraphTabular
🎯 What it does: The study addresses causal structure learning using data with known soft intervention targets in the presence of potential confounding variables, proposing an improved I-FCI algorithm and proving its incompleteness.
Towards Context-Invariant Safety Alignment for Large Language Models
Yixu Wang (Fudan University), Yingchun Wang (Shanghai Artificial Intelligence Laboratory)
OptimizationSafty and PrivacyExplainability and InterpretabilityReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningPrompt EngineeringContrastive LearningText
🎯 What it does: Proposed a new alignment training framework called Anchor Invariance Regularization (AIR), which enhances the context invariance of large language models (LLMs) in safety, ethical reasoning, and mathematical tasks by enforcing consistency between open-ended generation and verifiable prompts used as anchors.
Towards Diffeomorphism-Equivariant Neural Networks via Canonicalization
Josephine Elisabeth Oettinger (University of Luebeck), Carola-Bibiane Schönlieb (University of Cambridge)
ClassificationImage TranslationSegmentationData SynthesisConvolutional Neural NetworkScore-based ModelFlow-based ModelAuto EncoderGenerative Adversarial NetworkContrastive LearningOptical FlowImageBiomedical DataComputed TomographyReview/Survey PaperStochastic Differential Equation
🎯 What it does: Propose a canonicalization method based on energy regularization, enabling pre-trained networks to achieve diffeomorphism equivariance without retraining or large-scale data augmentation.
Towards Disentangled Preference Optimization Dynamics: Suppress the Loser, Preserve the Winner
Wei Chen (South China University Of Technology), Zhou Wang (University Of Waterloo)
OptimizationRepresentation LearningReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningPrompt EngineeringText
🎯 What it does: This paper proposes a unified incentive-score decomposition by analyzing the gradient dynamics of the preference optimization process, identifying a decoupling path that can achieve 'suppressing inferior responses while maintaining high-quality responses' during training, and designs a reward calibration (RC) plugin that enables different objectives to more easily enter this decoupling path without modifying the original objectives, thereby improving the alignment of large language models.
Towards Diverse Scientific Hypothesis Search with Large Language Models
Haorui Wang (Georgia Institute of Technology), Yuanqi Du (Microsoft Research New England)
OptimizationDrug DiscoveryTransformerLarge Language ModelReinforcement LearningMixture of ExpertsTextBenchmark
🎯 What it does: Propose a scientific hypothesis search framework called EvoDiverse based on large language models, which utilizes parallel tempering to maintain the diversity of candidate hypotheses.
Towards Docking-oriented De Novo Ligand Design via Gradient Inversion
Zekai Chen (Beijing Institute of Technology), Guoren Wang (Beijing Institute of Technology)
Drug DiscoveryProtein Structure PredictionGraph Neural NetworkTransformerDiffusion modelScore-based ModelAuto EncoderGenerative Adversarial NetworkContrastive LearningPoint CloudGraphBiomedical Data
🎯 What it does: Proposed the MagicDock framework, achieving template-free and retrieval-free de novo ligand design, and supporting both protein and small molecule ligands.
Towards Effective Waste Segmentation for Automated Waste Recycling in Cluttered Background
Mamoona Javaid (Institute of Space Technology), Sajid Ghuffar (Institute of Space Technology)
SegmentationConvolutional Neural NetworkAuto EncoderContrastive LearningImage
🎯 What it does: Propose an end-to-end garbage segmentation network called EWSegNet, which combines spatial and frequency domain features to improve segmentation accuracy in cluttered backgrounds.
Towards Efficient and Expressive Offline RL via Flow-Anchored Noise-conditioned Q-Learning
Sungyoung Lee (University of Texas at Austin), Keshav Pingali (University of Texas at Austin)
Reinforcement LearningFlow-based ModelTabularTime SeriesSequential
🎯 What it does: Proposes the FAN algorithm, combining the regularization of one-time flow iteration behavior with single-noise distributed Q-learning, achieving efficient and expressive performance in offline reinforcement learning.
Towards Efficient Large Language Reasoning Models via Extreme-Ratio Chain-of-Thought Compression
Yuntian Tang (East China Normal University), Shaohui Lin (East China Normal University)
Computational EfficiencyKnowledge DistillationReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextChain-of-Thought
🎯 What it does: Proposes Extra-CoT, a three-stage extreme ratio Chain-of-Thought (CoT) compression framework, which can maintain high logical fidelity and improve reasoning accuracy under extremely low token budgets.
Towards Efficient LLMs Annealing with Principled Sample Selection
Yuanjian Xu (Hong Kong University of Science and Technology (Guangzhou)), Guang Zhang (Hong Kong University of Science and Technology (Guangzhou))
OptimizationComputational EfficiencyData-Centric LearningTransformerLarge Language ModelText
🎯 What it does: Proposes DiReCT, a training sample selection framework based on validation Hessian spectral analysis, to achieve more efficient sample scheduling during the LLMA annealing phase.
Towards Execution-Grounded Automated AI Research
Chenglei Si (Stanford University), Tatsunori Hashimoto (Stanford University)
AI Code AssistantReinforcement Learning from Human FeedbackNeural Architecture SearchTransformerLarge Language ModelReinforcement LearningPrompt EngineeringTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: This paper constructs a large-scale parallel automated idea executor that translates natural language research ideas into executable code and verifies their effectiveness through GPU parallel experiments.
Towards Feedback-to-Plan Decisions for Self-Evolving LLM Agents in CUDA Kernel Generation
Yee Hin Chong (Tsinghua University), Peng Qu (Tsinghua University)
OptimizationExplainability and InterpretabilityAI Code AssistantReinforcement Learning from Human FeedbackTransformerLarge Language ModelPrompt EngineeringTextBenchmarkChain-of-Thought
🎯 What it does: Propose the CUDAnalyst framework, which performs causal attribution at the generation level for feedback-to-plan decision-making in self-evolving LLM agents, adopting trajectory freezing, feedback injection, and Banzhaf cooperative attribution;
Towards Fine-Grained Robustness: Attention-Guided Test-Time Prompt Tuning for Vision-Language Models
Jia-Wei Hai (Southeast University), Xiu-Shen Wei (Southeast University)
ClassificationRecognitionAdversarial AttackTransformerPrompt EngineeringVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: Proposed an attention-guided test-time prompt tuning method called A-TPT to enhance the robustness of vision-language models in fine-grained tasks.
Towards Foundation Models for Zero-Shot Time Series Anomaly Detection: Leveraging Synthetic Data and Relative Context Discrepancy
Tian Lan (Tsinghua University), Chen Zhang (Tsinghua University)
Data SynthesisAnomaly DetectionRepresentation LearningTransformerAuto EncoderGenerative Adversarial NetworkContrastive LearningTime Series
🎯 What it does: Propose TimeRCD, a zero-shot time series anomaly detection framework based on pretraining with Relative Contextual Difference (RCD).
Towards Fully Parameter-Free Stochastic Optimization: Grid Search with Self-Bounding Analysis
Yuheng Zhao (Nanjing University), Peng Zhao (Nanjing University)
OptimizationHyperparameter SearchImage
🎯 What it does: This paper proposes a fully parameter-free stochastic optimization framework called GRASP, which uses self-bound analysis to achieve grid search without prior parameters.
Towards Functional Correctness of Large Code Models with Selective Generation
Jaewoo Jeong (POSTECH), Sangdon Park (POSTECH)
GenerationAI Code AssistantTransformerLarge Language ModelReinforcement LearningPrompt EngineeringDiffusion modelTextSequential
🎯 What it does: Propose a controllable selective code generation method, which uses automatically generated unit tests (via fuzz testing) to evaluate the functional correctness of the code, and self-abandons during the generation process based on confidence levels, thereby controlling the functional hallucination rate in code generation.
Towards Generative Graph Matching for Graph Edit Distance Computation
Wei Huang (University of New South Wales), Xuemin Lin (Shanghai Jiaotong University)
OptimizationComputational EfficiencyDrug DiscoveryGraph Neural NetworkTransformerDiffusion modelAuto EncoderContrastive LearningGraph
🎯 What it does: Propose a generative graph matching framework based on diffusion models, named DiffGED, for efficiently computing graph edit distance (GED) and generating multiple diverse edit paths.
Towards Hierarchy–Uniformity Equilibrium: Recovering Semantic Depth in Hypergraph Contrastive Learning
Ruiting Zhao (Zhejiang Normal University), Pietro Lio (University of Cambridge)
ClassificationRepresentation LearningGraph Neural NetworkTransformerAuto EncoderContrastive LearningGraph
🎯 What it does: Propose the HyperDepth framework to address the hierarchy-uniform conflict in hypergraph contrastive learning and restore the semantic depth of hyperedges.
Towards High-Fidelity CAD Generation via LLM-Driven Program Generation and Text-Based B-Rep Primitive Grounding
Jiahao Li (Fudan University), Xiangdong Zhou (Fudan University)
GenerationData SynthesisGraph Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextMeshGraph
🎯 What it does: Propose the FutureCAD framework, which utilizes LLM to generate executable CadQuery scripts and precisely locate B-Rep geometric primitives through text queries, thereby enabling the automatic generation of high-fidelity CAD models.
Towards Long-Horizon Interpretability: Efficient and Faithful Multi-Token Attribution for Reasoning LLMs
Wenbo Pan (City University of Hong Kong), Xiaohua Jia (City University of Hong Kong)
Explainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: Propose an efficient multi-word attribution method called FLASHTRACE for reasoning in large language models, which can compute attribution for continuous output paragraphs in one go and recursively trace the reasoning chain;
Towards Multimodal Large Language Models with Both Training and Inference Efficiency
Qianhao Yuan (Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences), Le Sun
Computational EfficiencyRepresentation LearningTransformerLarge Language ModelVision Language ModelContrastive LearningImageVideoTextMultimodality
🎯 What it does: This paper proposes a novel multimodal large language model architecture called SAISA, aiming to simultaneously improve training efficiency and inference efficiency.
Towards On-Policy SFT: Distribution Discriminant Theory and its Applications in LLM Training
Miaosen Zhang (Southeast University), Xin Geng (Southeast University)
Knowledge DistillationData-Centric LearningReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningContrastive LearningTextBenchmark
🎯 What it does: Proposes theoretical and methodological approaches for aligning the model's own distribution in supervised fine-tuning (SFT), utilizing distribution discrimination theory (DDT) to construct a centralized log-likelihood (CLL) discrimination metric, and further designing adaptive loss reweighting techniques IDFT and entropy-guided hybrid decoding methods Hinted Decoding;
Towards One-for-All Anomaly Detection for Tabular Data
Shiyuan Li (Griffith University), Heng Tao Shen (Tongji University)
Anomaly DetectionTransformerMixture of ExpertsContrastive LearningTabularBenchmark
🎯 What it does: Proposed a general model called OFA-TAD that can perform anomaly detection on different tabular datasets with a single training process.
Towards One-to-Many Temporal Grounding
Qi Xu (Wuhan University), Xiangtai Li (Bytedance)
RecognitionReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningVision Language ModelVideoTextMultimodalityBenchmarkChain-of-Thought
🎯 What it does: Proposed the One-to-Many Temporal Grounding (OMTG) task, which requires locating all non-overlapping time segments in a video for a single text query.
Towards Optimal Robustness in Learning-Augmented Paging
Peng Chen (Zhejiang University), Shuiguang Deng (Zhejiang University)
OptimizationTabularTime SeriesBenchmark
🎯 What it does: This paper studies the robustness issue of learning-enhanced paging algorithms, proposes the Relative Prediction Budget (RPB) primitive, and designs the RPB-ONOPT framework based on it; implements the RPB-OM algorithm, achieving optimal robustness with 1-consistency and H_k+O(1).
Towards Parameter-Free Temporal Difference Learning
Yunxiang LI, Sharan Vaswani (Simon Fraser University)
Reinforcement LearningTabularTime Series
🎯 What it does: Propose a TD(0) algorithm using an exponentially decaying step size, which can achieve parameter-free, projection-free, and average-free last-iterate convergence under both i.i.d. and Markov sampling.
Towards Pareto-Optimal Tool-Integrated Agents with Pareto Ranking Policy Optimization
Junyi Li (City University of Hong Kong), Xiangyu Zhao (City University of Hong Kong)
Autonomous DrivingOptimizationRobotic IntelligenceReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningAgentic AIPrompt EngineeringMixture of ExpertsTextBenchmark
🎯 What it does: Proposes ParetoPO, a two-stage multi-objective reinforcement learning framework for training LLM agents with integrated tools, balancing task accuracy and tool usage efficiency.
Towards Practical World Model-based Reinforcement Learning for Vision-Language-Action Models
Zhilong Zhang (Nanjing University), Yang Yu (Nanjing University)
Robotic IntelligenceTransformerSupervised Fine-TuningReinforcement LearningVision-Language-Action ModelWorld ModelImageTextMultimodality
🎯 What it does: Propose a world model based on a unified multimodal model, combining multi-perspective decoding and chunk-level branch rolling RL framework (VLA-MBPO), for efficient reinforcement learning fine-tuning of Vision-Language-Action models in simulated and real environments.
Towards Professional-Grade Financial Agents: Benchmarking, Tooling, and Structured Reasoning
Cheng Huang (University of Electronic Science and Technology of China), Yong Li (Tsinghua University)
TransformerLarge Language ModelAgentic AITextTabularBenchmarkFinance RelatedRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Proposed a professional-level financial agent, ProFinAgent, and constructed a financial tool universe and the ProFinR benchmark;
Towards Realistic Lifelong Re-identification: Identity Recurrence with Changing Clothes
Wuxuan Shi (Wuhan University), Mang Ye (Wuhan University)
RecognitionRetrievalConvolutional Neural NetworkTransformerContrastive LearningImageBenchmark
🎯 What it does: Propose the LCC Re-ID framework to address the problem of asynchronous re-identification in lifelong recognition where identities change with clothing.
Towards Reliable Marking and Verification of AI-Generated Text via Geometry-aware Sentence-level Watermarking
Yubing Ren (Chinese Academy of Sciences), Yanan Cao (Chinese Academy of Sciences)
GenerationData SynthesisSafty and PrivacyExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringContrastive LearningTextMultimodalityRetrieval-Augmented Generation
🎯 What it does: Proposes X-Guard, a geometry-aware sentence-level text watermarking method that can reliably embed and verify AI-generated text in black-box generation models.
Towards Resource-Efficient LLMs: End-to-End Energy Accounting of Distillation Pipelines
Katherine Lambert (University of Toronto), Sasha Luccioni (Sustainable AI Group)
Computational EfficiencyKnowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes a complete end-to-end energy accounting framework to measure the energy consumption throughout the entire process of large language model distillation (logit KD and synthetic SFT), covering four stages: teacher-side data generation, logit caching, student training, and evaluation; and systematically measures the energy-quality Pareto frontier for students of different scales (1B/7B/13B) under fixed hardware, software, dataset, and hyperparameter conditions; through analysis of the teacher resource reuse threshold, it provides the optimal distillation strategy in terms of energy consumption for practical deployment.
Towards Rule-Based Knowledge Sharing in Federated Learning
Zixuan Qin (Tianjin University), Qinghua Hu (Tianjin University)
ClassificationFederated LearningExplainability and InterpretabilityComputational EfficiencyKnowledge DistillationConvolutional Neural NetworkPrompt EngineeringContrastive LearningImage
🎯 What it does: Propose a rule-based federated learning framework, RFL, which utilizes interpretable category discriminative rules for knowledge sharing among clients with different architectures, avoiding feature semantic confusion and significantly reducing communication overhead.
Towards Scalable and Consistent 3D Editing
Ruihao Xia (East China University of Science and Technology), Pan Zhou (Singapore Management University)
GenerationData SynthesisTransformerPrompt EngineeringDiffusion modelScore-based ModelAuto EncoderGenerative Adversarial NetworkContrastive LearningImageTextPoint CloudMesh
🎯 What it does: A scalable and structure-preserving 3D editing method is studied, and a large paired 3D editing dataset and a mask-free editing model are proposed.
Towards Seed-Robust Safety Alignment in Text-to-Image Models
Zhenyu Wu (Beihang University), Xingxing Wei (Beihang University)
GenerationSafty and PrivacyDiffusion modelScore-based ModelContrastive LearningImageText
🎯 What it does: Proposed the Noise Contrastive Diffusion (NCD) framework to enhance the safety alignment stability of text-to-image diffusion models under different random seeds.
Towards Solving the Gilbert-Pollak Conjecture via Large Language Models
Yisi Ke (Peking University), Liwei Wang (Peking University)
Symbolic ComputationOptimizationComputational EfficiencyKnowledge DistillationAI Code AssistantReinforcement Learning from Human FeedbackTransformerLarge Language ModelPrompt EngineeringTextReview/Survey PaperPhysics Related
🎯 What it does: Using large language models (LLMs) to automatically generate structured geometric lemmas and convert them into verifiable functions to iteratively improve the lower bound of the Steiner ratio
Towards Spectroscopy: Susceptibility Clusters in Language Models
Andrew Gordon (Timaeus), Daniel Murfet (Timaeus)
Explainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelAuto EncoderContrastive LearningTextPhysics RelatedStochastic Differential Equation
🎯 What it does: This paper clusters the susceptibility of language models, revealing specialized responses of the model to hundreds of data distribution patterns internally, and identifies 510 interpretable clusters on Pythia-14M.
Towards Steering without Sacrifice: Principled Training of Steering Vectors for Prompt-only Interventions
Yuntai Bao (Zhejiang University), Xuhong Zhang (Zhejiang University)
OptimizationExplainability and InterpretabilityComputational EfficiencyRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringContrastive LearningTextBenchmark
🎯 What it does: Propose jointly training the parameters of the Steering Vector (SV) (direction and factor) and introduce Prompt-Only Steering Vector (PrOSV), to achieve the concept of steering without posterior factor selection, with intervention only on a small number of tokens in the prefix/suffix.
Towards Streaming Synchronized Spatial Audio Generation via Autoregressive Diffusion Transformer
Ke Lei (Zhejiang University), Zhou Zhao (Zhejiang University)
GenerationData SynthesisTransformerReinforcement LearningDiffusion modelAuto EncoderContrastive LearningVideoTextMultimodalityAudio
🎯 What it does: Propose the SwanSphere framework to generate synchronized high-quality first-order Ambisonics spatial audio from panoramic videos and text.
Towards Sub-Second Molecular Docking as a Structural Primitive: A Quantized Consistency Diffusion Framework
Kexin Zhang (ShanghaiTech University), Jingyi Yu (ShanghaiTech University)
Drug DiscoveryProtein Structure PredictionTransformerDiffusion modelContrastive LearningBiomedical Data
🎯 What it does: Proposed a quantized consistency diffusion framework that compresses molecular docking into sub-second-level structural primitives.
Towards the Explainability of Temporal Graph Networks via Memory Backtracking and Topological Attribution
Yazheng Liu (Hong Kong University of Science and Technology), Hui Xiong (Hong Kong University of Science and Technology)
Explainability and InterpretabilityGraph Neural NetworkGraphTime Series
🎯 What it does: Study the explainability of Temporal Graph Networks, proposing to interpret prediction results through memory backtracking and topological attribution trees.
Towards the Training of Deeper Predictive Coding Neural Networks
Chang Qi (Vienna University Of Technology), Tommaso Salvatori (Vienna University Of Technology)
ClassificationComputational EfficiencyRepresentation LearningConvolutional Neural NetworkSpiking Neural NetworkAuto EncoderContrastive LearningImage
🎯 What it does: This paper enhances the depth trainability of predictive coding networks by introducing mechanisms such as adaptive precision weighting, forward updates, and auxiliary neurons.
Towards Theoretical Understanding of Transformer Test-Time Computing: Investigation on In-Context Linear Regression
Xingwu Chen (University of Hong Kong), Difan Zou (University of Hong Kong)
Explainability and InterpretabilityComputational EfficiencyRepresentation LearningTransformerLarge Language ModelContrastive LearningTextTabularChain-of-Thought
🎯 What it does: Proposed a theoretical framework that combines random sampling with the Transformer inference process, and studied the impact of test-time computation (multi-step reasoning, number of samplings) on performance using the context linear regression task as an example.
Towards Trustworthy and Identifiable Virtual Face Generation
Chunyang Li (Fudan University), Weiqiang Wang (Ant Group)
GenerationSafty and PrivacyTransformerPrompt EngineeringDiffusion modelGenerative Adversarial NetworkContrastive LearningImage
🎯 What it does: Propose a trustworthy and identifiable virtual face (IVF) generation framework called TIVDiff, which combines virtual identity projection, pose-aware diffusion generation, and reversible watermark verification to achieve high-quality, controllable, and verifiable virtual faces.
Towards Trustworthy Video Anomaly Understanding: A Class-Guided Chain-of-Evaluation Metric and An Anomaly-focused Meta-Benchmark
Jiaxu Leng (Chongqing University of Posts and Telecommunications), Xinbo Gao (Chongqing University of Posts and Telecommunications)
Anomaly DetectionTransformerLarge Language ModelPrompt EngineeringVideoTextBenchmarkChain-of-Thought
🎯 What it does: This paper proposes a class-guided chain evaluation metric, CG-CoE, specifically designed to evaluate the ability of video anomaly understanding models to identify abnormal events, and constructs a corresponding credibility and robustness evaluation benchmark;
Towards Understanding Adam Convergence on Highly Degenerate Polynomials
Zhiwei Bai (Shanghai Jiao Tong University), Yaoyu Zhang (Shanghai Jiao Tong University)
Optimization
🎯 What it does: This paper studies the natural convergence of the Adam optimizer on high-order degenerate polynomial objective functions, proving that linear convergence can be achieved without using learning rate decay;
Towards Understanding Continual Factual Knowledge Acquisition of Language Models: From Theory to Algorithm
Haoyu Wang (Renmin University of China), Jun Xu (Renmin University of China)
Federated LearningExplainability and InterpretabilityComputational EfficiencyKnowledge DistillationRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningAuto EncoderContrastive LearningTextRetrieval-Augmented Generation
🎯 What it does: Investigate the dynamics of factual knowledge acquisition (cFKA) during the continuous pre-training of language models, construct a theoretical framework of a single-layer Transformer, reveal that regularization only regulates the convergence rate and data replay can transfer old knowledge, and based on this, propose the STOC generative replay method.
Towards Understanding Generalization of Federated Adversarial Learning: Perspective of Algorithmic Stability
Yongkang Yang (Huazhong Agricultural University), Rushi Lan (Guilin University of Electronic Technology)
OptimizationFederated LearningComputational EfficiencyAdversarial AttackSupervised Fine-TuningReinforcement LearningContrastive LearningImageTabular
🎯 What it does: Under the federated adversarial learning framework, this paper uniformly analyzes the generalization performance of different optimizers through algorithmic stability theory, revealing the deterioration of traditional SGD's generalization under strong adversarial attacks, and proposes the FalME method, which utilizes the smoothness of the Moreau envelope to simultaneously improve adversarial robustness and generalization.
Towards Understanding Massive Activations in Attention Sink Mechanism
Haiyu Wang (Chinese University of Hong Kong), Yuanyuan Lin (Chinese University of Hong Kong)
Explainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: This paper clarifies the intrinsic relationship between large activation and attention sink mechanisms through theoretical analysis and large-scale experimental interventions, proving that large activation is not a necessary condition for forming attention sinks, but rather plays a role in stabilizing self-attention.
Towards Understanding Modality Interaction in Multimodal Language Models via Partial Information Decomposition
Wanlong Fang (Nanyang Technological University), Alvin Chan (Nanyang Technological University)
Explainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageVideoTextMultimodalityBenchmarkAudio
🎯 What it does: This paper proposes using Partial Information Decomposition (PID) to analyze the modal interaction of multi-modal large language models at the decision level, and extends it to a tri-modal system by introducing Sensory PID; through PID, visual, textual, video, and audio modal information can be decomposed into three categories: unique, redundant, and synergistic, revealing the modal usage characteristics of the model on different benchmarks; further, PID diagnostic results are used to guide sample reweighting (PID-guided reweighting), improving multi-modal reasoning and alignment performance during LoRA fine-tuning.
Towards Understanding Steering Strength
Magamed Taimeskhanov (University of Würzburg), Damien Garreau (Université Côte d'Azur)
Safty and PrivacyExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Controlling the behavior of large language models (LLMs) during inference by linearly shifting (steering) the intermediate representations, and theoretically analyzing the steering strength α.
Towards Understanding the Dynamics of Low-Rank Adaptation
Shu Ding (Nanjing University), Wei Wang (Nanjing University)
OptimizationComputational EfficiencyKnowledge DistillationRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper provides a theoretical understanding of the dynamics of low-rank adaptation (LoRA) updates, revealing that the update dynamics can be viewed as projecting the gradient onto a subspace generated by a low-rank matrix, and proving that using an equiangular tight frame (ETF) to initialize the low-rank matrix is the optimal choice when gradients are unavailable.
Towards Uniformity and Alignment for Multimodal Representation Learning
Wenzhe Yin (University of Amsterdam), Stratis Gavves (University of Amsterdam)
GenerationRetrievalRepresentation LearningTransformerVision Language ModelDiffusion modelAuto EncoderContrastive LearningImageVideoTextMultimodalityAudio
🎯 What it does: Studied the alignment-uniformity conflict and internal alignment conflict caused by InfoNCE in multi-modal representation learning, and proposed UniAlign, which constructs a unified learning framework without conflicts by decoupling alignment and uniformity, using anchor-based alignment, and global Holder divergence theory.