ICLR 2026 Papers — Page 4
International Conference on Learning Representations · 5356 papers
Arbitrary-Order Block SignSGD for Memory-Efficient LLM Fine-Tuning
Yijie Zhou (Chinese University of Hong Kong), Shi Pu (Chinese University of Hong Kong)
OptimizationTransformerSupervised Fine-TuningText
🎯 What it does: Propose ABSignSGD, a block-coordinate based optimizer using sign gradients, to achieve memory and runtime efficiency for full-parameter fine-tuning of large language models without sacrificing performance.
Arbitrary-Shaped Image Generation via Spherical Neural Field Diffusion
Jiyuan Xia (University of Science and Technology of China), Zhiwei Xiong (University of Science and Technology of China)
GenerationDiffusion modelNeural Radiance FieldAuto EncoderImageTextMultimodality
🎯 What it does: Unify the controllable generation of spatial attributes such as viewpoint, field of view (FOV), and resolution, supporting various image shapes including perspective, panoramic, and fisheye views;
Architecture-Agnostic Test-Time Adaptation via Backprop-Free Embedding Alignment
MA Xiao, Dong Ma (University of Cambridge)
Domain AdaptationConvolutional Neural NetworkTransformerImage
🎯 What it does: Proposes a test-time adaptation method called Progressive Embedding Alignment (PEA), which corrects domain drift through layer-wise covariance alignment without using backpropagation.
Are Deep Speech Denoising Models Robust to Adversarial Noise?
Will Schwarzer (University of Massachusetts), Xiaoyu Liu (Meta)
Adversarial AttackAudio
🎯 What it does: A systematic adversarial attack study on four mainstream open-source speech denoising (DNS) models, demonstrating that psychoacoustic masking noise can be added to maintain audio audibility while rendering model outputs as unrecognizable garbled text; conducted human listening evaluations, experiments under different environments (SNR, reverberation, over-speech), and explored adversarial transmission and defense strategies;
Are EEG Foundation Models Worth It? Comparative Evaluation with Traditional Decoders in Diverse BCI Tasks
Liuyin Yang (KU Leuven), Marc M. Van Hulle (KU Leuven)
Convolutional Neural NetworkTransformerSupervised Fine-TuningAuto EncoderBiomedical DataAlzheimer's DiseaseBenchmark
🎯 What it does: Proposed and systematically evaluated multiple EEG base models (including the newly constructed ST-EEGFormer) under multi-task and multi-evaluation protocols, comparing their performance with classical CNNs and traditional non-neural decoders;
Are Global Dependencies Necessary? Scalable Time Series Forecasting via Local Cross-Variate Modeling
Kun Liu (East China Normal University), Cen Chen (East China Normal University)
Convolutional Neural NetworkAuto EncoderTime Series
🎯 What it does: Propose a time series prediction framework VPNet based on local cross-variable interaction, mapping inputs to a high-dimensional variate-patch field through a Patch-level autoencoder, then performing linear complexity local convolution processing with VarTCNBlock, and finally outputting prediction results.
Are LLMs Really Not Knowledgeable? Mining the Submerged Knowledge in LLMs' Memory
Xingjian Tao (Hong Kong University of Science and Technology (Guangzhou)), Jing Tang (Hong Kong University of Science and Technology (Guangzhou))
Explainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: This paper analyzes the output distribution of LLMs to reveal the gap between knowledge storage and expression in question-answering tasks, and proposes the Hits@k metric to evaluate potential knowledge.
Are Reasoning LLMs Robust to Interventions on their Chain-of-Thought?
Alexander von Recum (Helmholtz Munich), Zeynep Akata (Helmholtz Munich)
Explainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: This paper constructs a controllable intervention framework, applying seven different types of interventions (benign, neutral, adversarial) at fixed time steps in the chain-of-thought (CoT) of reasoning LLMs (RLLMs), evaluating the model's ability to self-correct and maintain robustness during the reasoning process.
Are we measuring oversmoothing in graph neural networks correctly?
Kaicheng Zhang (University of Edinburgh), Francesco Tudisco (University of Edinburgh)
ClassificationGraph Neural NetworkGraph
🎯 What it does: Investigate and evaluate the over-smoothing problem in graph neural networks (GNNs), proposing to measure over-smoothing using the continuous rank (numerical rank/effective rank) of the feature matrix.
ARES: Multimodal Adaptive Reasoning via Difficulty-Aware Token-Level Entropy Shaping
Shuang Chen (University of California Los Angeles), Nanyun Peng (University of California Los Angeles)
OptimizationComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextMultimodalityBenchmarkChain-of-Thought
🎯 What it does: Proposed and implemented a framework named ARES, aiming to enable multimodal large-scale reasoning models (MLRMs) to adaptively allocate reasoning steps based on task difficulty, avoiding excessive reasoning on simple tasks and insufficient reasoning on complex tasks.
ARFlow: Auto-regressive Optical Flow Estimation for Arbitrary-Length Videos via Progressive Next-Frame Forecasting
Jiuming Liu (Shanghai Jiao Tong University), Hesheng Wang (Shanghai Jiao Tong University)
Recurrent Neural NetworkTransformerOptical FlowVideo
🎯 What it does: This paper proposes an auto-regressive multi-frame optical flow estimation framework, ARFlow, which can predict optical flow frame by frame and utilizes historical optical flow for next-frame initialization and multi-scale temporal refinement.
Aria: an Agent for Retrieval and Iterative Auto-Formalization via Dependency Graph
Hanyu Wang (Peking University), Bin Dong (Peking University)
AI Code AssistantGraph Neural NetworkTransformerAgentic AITextGraphBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Propose Aria, an automated formal agent capable of recursively decomposing mathematical statements and automatically generating Lean 4 formalizations through Graph-of-Thought and Retrieval-Augmented Generation (RAG), with self-reflection in the compilation loop; simultaneously design AriaScorer for term-level semantic checks, retrieving Mathlib definitions and comparing them with the original text to improve accuracy.
ARINBEV: Bird's-Eye View Layout Estimation with Conditional Autoregressive Model
Jiyong Kwag (Ohio State University), Alper Yilmaz (Ohio State University)
SegmentationAutonomous DrivingTransformerImage
🎯 What it does: Propose a single-stage, decoder-only autoregressive model called ARINBEV, which directly estimates BEV maps using class encoding and entropy-guided masking, avoiding two-stage discrete representation learning.
ARM-FM: Automated Reward Machines via Foundation Models for Compositional Reinforcement Learning
Roger Creus Castanyer (Université de Montréal), Glen Berseth (Université de Montréal)
TransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Automatically generate reward machines (Reward Machines) from natural language descriptions using foundation models, and embed them into the reinforcement learning (RL) process to achieve decomposition of sparse reward tasks, generation of dense rewards, and cross-task knowledge transfer with zero-shot generalization.
ARMOR: Aligning Secure and Safe Large Language Models via Meticulous Reasoning
Zhengyue Zhao, Chaowei Xiao (Johns Hopkins University)
Safty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningPrompt EngineeringTextBenchmarkChain-of-Thought
🎯 What it does: Propose ARMOR, a method that enhances the security of large language models against advanced optimized jailbreak attacks by utilizing an updatable strategy library to extract core user intent through a structured 'detailed reasoning' process and performing security verification based on safety policies.
ARMOR: High-Performance Semi-Structured Pruning via Adaptive Matrix Factorization
Lawrence Liu (University of California, Los Angeles), Lin F. Yang (University of California, Los Angeles)
Computational EfficiencyTransformerLarge Language ModelText
🎯 What it does: Proposed a first-order post-training sparsification method called ARMOR, which decomposes the weight matrix into a 2:4 sparse core and a low-overhead block-diagonal wrapper to achieve efficient compression.
ARMs: Adaptive Red-Teaming Agent against Multimodal Models with Plug-and-Play Attacks
Zhaorun Chen (University of Chicago), Bo Li (Meta)
Safty and PrivacyExplainability and InterpretabilityAdversarial AttackLarge Language ModelVision Language ModelMultimodalityBenchmark
🎯 What it does: Propose ARMS, an adaptive multimodal red team agent capable of automatically generating diverse attack instances based on risk definitions and evaluating the security of Vision-Language Models (VLMs).
ARROW: An Adaptive Rollout and Routing Method for Global Weather Forecasting
Jindong Tian (East China Normal University), Bin Yang (East China Normal University)
TransformerReinforcement LearningMixture of ExpertsTime SeriesPhysics Related
🎯 What it does: Propose a global weather prediction framework ARROW based on adaptive rolling and routing, incorporating a multi-time-step prediction model and an adaptive rolling scheduler.
ARTDECO: Toward High-Fidelity On-the-Fly Reconstruction with Hierarchical Gaussian Structure and Feed-Forward Guidance
Guanghao Li (Shanghai Artificial Intelligence Laboratory), Jiangmiao Pang (Shanghai Artificial Intelligence Laboratory)
GenerationPose EstimationGaussian SplattingSimultaneous Localization and MappingImagePoint CloudMesh
🎯 What it does: Propose a unified framework ARTDECO for real-time high-fidelity 3D reconstruction from monocular image sequences, integrating the efficiency of feed-forward models with the accuracy of SLAM-based approaches.
Articulation in Motion: Prior-free Part Mobility Analysis for Articulated Objects By Dynamic-Static Disentanglement
Hao Ai (University of Birmingham), Eyal Ofek (University of Birmingham)
SegmentationGenerationPose EstimationGaussian SplattingVideoPoint Cloud
🎯 What it does: This paper proposes Articulation in Motion (AIM), which achieves joint segmentation, kinematic analysis, and 3D reconstruction without prior knowledge by utilizing only user-object interaction videos and initial state scans.
ArtUV: Artist-style UV Unwrapping
Yuguang Chen (Sun Yat-sen University), Chunchao Guo (Tencent Hunyuan)
GenerationGraph Neural NetworkTransformerAuto EncoderMeshBenchmark
🎯 What it does: Proposed a fully automatic, end-to-end UV unwrapping method called ArtUV, dividing the process into two steps: surface seam prediction and artist-style UV parameterization.
ArtVIP: Articulated Digital Assets of Visual Realism, Modular Interaction, and Physical Fidelity for Robot Learning
Zhao Jin (Beijing Innovation Center of Humanoid Robotics), Jian Tang (Beijing Innovation Center of Humanoid Robotics)
Data SynthesisDomain AdaptationRobotic IntelligenceImageMeshBenchmark
🎯 What it does: Propose the ArtVIP dataset, constructing 992 high-quality digital twin articulated objects covering 9 categories and 37 subcategories, accompanied by 6 simulation scenarios and pixel-level annotations for robot learning simulation and real-world validation.
ASCIIEval: Benchmarking Models' Visual Perception in Text Strings via ASCII Art
Qi Jia (Shanghai Artificial Intelligence Laboratory National University of Singapore), Guangtao Zhai (Shanghai Artificial Intelligence Laboratory National University of Singapore)
ClassificationRecognitionTransformerSupervised Fine-TuningPrompt EngineeringVision Language ModelTextMultimodalityBenchmark
🎯 What it does: This paper proposes the ASCIIEval benchmark, which evaluates the text-visual perception capabilities of LLMs and MLLMs using ASCII art;
ASGuard: Activation-Scaling Guard to Mitigate Targeted Jailbreaking Attack
Yein Park (Korea University), Jaewoo Kang (Korea University)
Safty and PrivacyExplainability and InterpretabilityComputational EfficiencyRepresentation LearningAdversarial AttackTransformerLarge Language ModelSupervised Fine-TuningTextSequentialBenchmark
🎯 What it does: Locate and repair security vulnerabilities in LLMs during tense attacks through Transformer circuit analysis, achieving precise secure alignment via channel-level activation scaling and preventive fine-tuning.
ASIDE: Architectural Separation of Instructions and Data in Language Models
Egor Zverev (Institute of Science and Technology Austria), Christoph H. Lampert (Institute of Science and Technology Austria)
Computational EfficiencyRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Propose and evaluate an architectural modification named ASIDE, which enhances the security of large language models (LLMs) by applying a fixed orthogonal rotation to data tokens in the token embedding layer, explicitly distinguishing instructions and data in the embedding space.
ASMIL: Attention-Stabilized Multiple Instance Learning for Whole-Slide Imaging
Linfeng Ye (University of Toronto), Konstantinos N. Plataniotis (University of Toronto)
ClassificationTransformerBiomedical Data
🎯 What it does: This paper proposes a multi-instance learning framework called ASMIL based on attention stabilization for weakly supervised diagnosis on whole slide images (WSI);
Assembling the Mind's Mosaic: Towards EEG Semantic Intent Decoding
Jiahe Li (Zhejiang University), Yang Yang (Zhejiang University)
GenerationExplainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelContrastive LearningTextMultimodalityBiomedical DataRetrieval-Augmented Generation
🎯 What it does: Proposed the Semantic Intent Decoding (SID) framework and implemented the BRAINMOSAIC deep learning architecture, which decodes EEG/SEEG signals into interpretable sets of semantic units and then generates coherent natural language sentences using LLMs; achieved cross-lingual and cross-modal brain-semantic alignment and sentence reconstruction.
ASSESS: A Semantic and Structural Evaluation Framework for Statement Similarity
Xiaoyang Liu (Shanghai Jiao Tong University), Tao Luo (Shanghai Jiao Tong University)
ClassificationTextBenchmark
🎯 What it does: Developed a framework called ASSESS for automatically evaluating the similarity of formal statements, and proposed the TransTED Similarity measure that integrates semantic transformations.
AssetFormer: Modular 3D Assets Generation with Autoregressive Transformer
Lingting Zhu (University of Hong Kong), Lequan Yu (University of Hong Kong)
GenerationData SynthesisTransformerPoint CloudMesh
🎯 What it does: Propose AssetFormer, a modular 3D asset generation framework based on autoregressive Transformer;
AssoMem: Scalable Memory QA with Multi-Signal Associative Retrieval
Kai Zhang (Worcester Polytechnic Institute), Xin Luna Dong (Worcester Polytechnic Institute)
RetrievalTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: Proposed the AssoMem framework, which achieves large-scale memory question answering by constructing an associative memory graph and fusing relevance, importance, and temporal signals.
AstaBench: Rigorous Benchmarking of AI Agents with a Scientific Research Suite
Jonathan Bragg, Daniel S Weld (Asta Team, Allen Institute for AI)
Large Language ModelTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Developed AstaBench, a complete evaluation framework containing over 2400 fine-grained scientific research tasks, a reproducible Asta environment, a unified tool interface, and an agent-baselines kit, conducting large-scale evaluations on 57 agents.
ASTGI: Adaptive Spatio-Temporal Graph Interactions for Irregular Multivariate Time Series Forecasting
Xvyuan Liu (East China Normal University), Jilin Hu (Engineering Research Center Of Blockchain Data Management)
Graph Neural NetworkTime SeriesElectronic Health Records
🎯 What it does: To enable precise prediction for irregular multivariate time series, we propose and implement the ASTGI framework.
Astra: General Interactive World Model with Autoregressive Denoising
Yixuan Zhu (Tsinghua University), Jie Zhou (Tsinghua University)
GenerationData SynthesisTransformerMixture of ExpertsVision-Language-Action ModelDiffusion modelScore-based ModelWorld ModelVideo
🎯 What it does: Built an interactive general-purpose world model Astra, which uses an autoregressive denoising framework to generate long-term, controllable videos from initial frames and action sequences.
Astraea: A Token-wise Acceleration Framework for Video Diffusion Transformers
Haosong Liu (Shanghai Jiao Tong University), Yu Feng (Shanghai Jiao Tong University)
GenerationComputational EfficiencyTransformerDiffusion modelVideo
🎯 What it does: Introduce a lightweight token-level acceleration framework ASTRAEA in video diffusion Transformer (vDiT) inference, dynamically selecting important tokens and caching computed results, combined with sparse attention and evolutionary search to achieve controllable speedup;
Asymmetric Proximal Policy Optimization: mini-critics boost LLM reasoning
Jiashun Liu (HKUST), Ling Pan (HKUST)
Computational EfficiencyTransformerLarge Language ModelReinforcement LearningPrompt EngineeringTextBenchmark
🎯 What it does: Reintroduce an asymmetric PPO framework with lightweight mini critics and non-overlapping prompt partitioning to enhance the stability and computational efficiency of large language model inference.
Asymmetric Synthetic Data Update for Domain Incremental Dataset Distillation
Minyoung Oh (Ulsan National Institute of Science and Technology), Jae-Young Sim (Ulsan National Institute of Science and Technology)
Domain AdaptationKnowledge DistillationConvolutional Neural NetworkImage
🎯 What it does: Proposed the Domain Incremental Dataset Distillation (DIDD) task and provided a complete framework for continuously distilling data from different domains on a single fixed-size synthetic dataset.
AsyncBEV: Cross-modal flow alignment in Asynchronous 3D Object Detection
Shiming Wang (Delft University of Technology), Julian F. P. Kooij (Delft University of Technology)
Object DetectionAutonomous DrivingOptical FlowImageMultimodalityPoint Cloud
🎯 What it does: Proposed the AsyncBEV module, which aligns multi-modal features by predicting BEV space flow under sensor time asynchrony, thereby enhancing the robustness of 3D object detection.
Asynchronous Denoising Diffusion Models for Aligning Text-to-Image Generation
Zijing Hu (Zhejiang University), Kun Kuang (Zhejiang University)
GenerationDiffusion modelImageText
🎯 What it does: Achieve asynchronous denoising by assigning different time steps to each pixel and using a variable concave time schedule, thereby improving text-to-image alignment.
Asynchronous Matching with Dynamic Sampling for Multimodal Dataset Distillation
Ding Qi (Tongji University), Cairong Zhao (Tongji University)
Knowledge DistillationContrastive LearningMultimodality
🎯 What it does: Propose an asynchronous matching and dynamic sampling framework (AMD) for multi-modal dataset distillation
Asynchronous Policy Gradient Aggregation for Efficient Distributed Reinforcement Learning
Alexander Tyurin (Applied AI Institute), Varvara Rudenko
Reinforcement Learning
🎯 What it does: Proposed two asynchronous aggregation policy gradient algorithms, Rennala NIGT and Malenia NIGT, to address computational and communication bottlenecks in distributed reinforcement learning, along with theoretical time complexity analysis and experimental validation.
AtC: Aggregate-then-Calibrate for Human-centered Assessment
Zejun Xie (Rutgers University), Desheng Zhang (Rutgers University)
Data-Centric LearningImageText
🎯 What it does: Proposes a two-stage framework Aggregate-then-Calibrate (AtC), which first aggregates human pairwise comparisons to obtain a consensus ranking, and then performs isometric projection calibration on model scores.
ATEX-CF: Attack-Informed Counterfactual Explanations for Graph Neural Networks
Yu Zhang (Aalborg University), Cuneyt Gurcan Akcora (University of Central Florida)
Explainability and InterpretabilityAdversarial AttackGraph Neural NetworkGraph
🎯 What it does: Investigated the integration of adversarial attacks with causal counterfactual explanations, proposing the ATEX-CF framework to generate interpretable explanations involving edge additions and deletions.
ATGen: Adversarial Reinforcement Learning for Test Case Generation
Qingyao Li (Shanghai Jiao Tong University), Weinan Zhang (Shanghai Jiao Tong University)
GenerationTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Train a test case generator ATGEN using adversarial reinforcement learning to generate high-quality test cases as it continuously encounters more challenging bugs.
ATLAS: Adaptive Transfer Scaling Laws for Multilingual Pretraining, Finetuning, and Decoding the Curse of Multilinguality
Shayne Longpre (MIT), Sayna Ebrahimi (Google DeepMind)
Computational EfficiencyRepresentation LearningData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Research and propose the Adaptive Cross-lingual Scaling Law (ATLAS), construct a 38-language mutual conversion matrix, explore the 'curse of multilingualism' in multilingual training, and provide a computational crossover point between pre-training and fine-tuning.
ATLAS: Alibaba Dataset and Benchmark for Learning-Augmented Scheduling
Zhiyun Jiang (University of Sydney), Albert Zomaya
OptimizationMeta LearningTabularBenchmark
🎯 What it does: This paper constructs the ATLAS dataset (based on over 730k real ML jobs from Alibaba PAI cluster) and provides the LASched benchmark framework to evaluate the prediction and scheduling performance of learning-enhanced scheduling.
ATLAS: Constraints-Aware Multi-Agent Collaboration for Real-World Travel Planning
Jihye Choi (University of Wisconsin-Madison), Tomas Pfister (Google Cloud)
OptimizationLarge Language ModelAgentic AITextBenchmark
🎯 What it does: Proposed the ATLAS multi-agent framework for dynamically discovering and satisfying explicit and implicit constraints in travel planning, supporting interactive search and feedback.
AtlasKV: Augmenting LLMs with Billion-Scale Knowledge Graphs in 20GB VRAM
Haoyu Huang (Hong Kong University of Science and Technology), Yangqiu Song (Huawei)
Computational EfficiencyTransformerLarge Language ModelGraph
🎯 What it does: Propose AtlasKV, a scalable method that injects knowledge graphs with billions of scale into large language models without retrieval or long context prefixes, under 20GB VRAM conditions.
ATOM: A Pretrained Neural Operator for Multitask Molecular Dynamics
Luke Thompson (University of Sydney), Andi Han (University of Sydney)
Drug DiscoveryTransformerBiomedical Data
🎯 What it does: Proposed ATOM, a pre-trained neural operator based on Transformer for multi-task molecular dynamics prediction;
Atomic HINs: Entity-Attribute Duality for Heterogeneous Graph Modeling
Shao-En Lin (National Taiwan University), Che Lin (National Taiwan University)
ClassificationOptimizationRepresentation LearningGraph Neural NetworkGraphBenchmark
🎯 What it does: This paper proposes the concept of atomic HIN and performs attribute atomization on data based on the entity-attribute duality principle, constructing the most expressive HIN; subsequently, a task-specific schema refinement method is designed, optimizing the graph structure through binary selection of node types and relationship types, and systematically searching using genetic algorithms; experiments are conducted on eight benchmark datasets using simplified RGCN (sRGCN) and more advanced HGNN to verify its effectiveness.
ATPO: ADAPTIVE TREE POLICY OPTIMIZATION FOR MULTI-TURN MEDICAL DIALOGUE
Ruike Cao (University of Science and Technology of China), Li Xiao (University of Science and Technology of China)
OptimizationDrug DiscoveryReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextBiomedical DataBenchmark
🎯 What it does: Proposed the ATPO (Adaptive Tree Policy Optimization) algorithm, which optimizes information acquisition strategies in multi-round medical dialogues through uncertainty-guided tree search.
Attack-Resistant Watermarking for AIGC Image Forensics via Diffusion-based Semantic Deflection
Qingyu Liu (Zhejiang University), Zhibo Wang (Zhejiang University)
Anomaly DetectionSafty and PrivacyDiffusion modelImage
🎯 What it does: Proposed a native watermarking framework PAI that requires no training and can be directly embedded into diffusion models for AIGC image copyright protection, ownership verification, attack detection, and semantic-level tampering localization.
Attend to the Active: Structure-Aware Dynamic Attention in LLMs for Compositional Instruction Following
Fangrui Lv (Tsinghua University), Changshui Zhang (Tsinghua University)
GenerationComputational EfficiencyTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Proposes a structure-aware dynamic attention mechanism (ATA) to dynamically identify and focus on currently active subtasks during generation, thereby suppressing attention interference from non-active subtasks and enhancing large language models' ability to follow compositional instructions.
Attention as a Compass: Efficient Exploration for Process-Supervised RL in Reasoning Models
Runze Liu (Tsinghua University), Kun Gai (Kuaishou Technology)
Reinforcement LearningTextBenchmark
🎯 What it does: Propose the AttnRL framework, which utilizes attention scores to guide process supervision reinforcement learning for efficient exploration and training in large reasoning models.
Attention Is All You Need for KV Cache in Diffusion LLMs
Quan Nguyen-Tri (MBZUAI), Zhiqiang Shen (MBZUAI)
Computational EfficiencyTransformerLarge Language ModelDiffusion modelText
🎯 What it does: For the inference phase of diffusion-based large language models (DLM), an adaptive KV cache update strategy named Elastic-Cache is designed. It utilizes attention drift and layer depth information to determine when and from which layer to refresh the cache, significantly reducing redundancy in QKV recomputation.
Attention Sinks and Compression Valleys in LLMs are Two Sides of the Same Coin
Enrique Queipo-de-Llano (University of Oxford), Ravid Shwartz-Ziv (New York University)
Representation LearningTransformerLarge Language ModelText
🎯 What it does: The paper unifies attention sinks and compression valleys as results of massive activations in residual flows, and proves their necessity for representation compression.
Attention Smoothing Is All You Need For Unlearning
Saleh Zare Zade (Wayne State University), Dongxiao Zhu (Wayne State University)
Knowledge DistillationTransformerLarge Language ModelText
🎯 What it does: Propose the Attention Smoothing Unlearning (ASU) method, which smooths the attention distribution by increasing the self-attention softmax temperature, constructs a forgetting teacher, and trains on the forgetting set using self-distillation to achieve unlearning in large language models.
Attention, Please! Revisiting Attentive Probing Through the Lens of Efficiency
Bill Psomas (Czech Technical University in Prague), Giorgos Tolias (Czech Technical University in Prague)
ClassificationComputational EfficiencyTransformerImage
🎯 What it does: This paper proposes an efficient attention aggregation method called Efficient Probing (EP) for evaluating frozen pre-trained models, achieving performance comparable to or even superior to linear probing while maintaining extremely low parameter and computational costs.
Attributing Response to Context: A Jensen–Shannon Divergence Driven Mechanistic Study of Context Attribution in Retrieval-Augmented Generation
Ruizhe Li (University of Aberdeen), Emine Yilmaz (University College London)
Explainability and InterpretabilityComputational EfficiencyTextRetrieval-Augmented Generation
🎯 What it does: Proposes an inference-time context attribution method called ARC-JSD based on Jensen-Shannon Divergence, which can quickly identify the most critical retrieved sentences for generation results in retrieval-augmented generation (RAG), and applies this method to interpret the internal mechanisms of RAG.
Attribution-Guided Decoding
Piotr Komorowski (Fraunhofer Heinrich Hertz Institute), Wojciech Samek (Fraunhofer Heinrich Hertz Institute)
Explainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: This paper proposes the Attribution-Guided Decoding (AGD) decoding strategy, which selects the word contributing the most to the user-specified ROI (e.g., instruction, knowledge head, or retrieval information) from the candidate word set using post-hoc attribution methods, thereby significantly improving LLM's instruction following and factual accuracy.
AttriCtrl: A Generalizable Framework for Controlling Semantic Attribute Intensity in Diffusion Models
Die Chen (East China Normal University), Yinda Chen (Alibaba Group)
GenerationData SynthesisDiffusion modelImageTextMultimodality
🎯 What it does: Propose a lightweight framework AttriCtrl that can control the aesthetic attribute intensity of diffusion models based on numerical values.
ATTS: Asynchronous Test-Time Scaling via Conformal Prediction
Jing Xiong (University of Hong Kong), Ngai Wong (University of Hong Kong)
Computational EfficiencyText
🎯 What it does: Proposes the ATTS (Asynchronous Test-Time Scaling) framework, which accelerates large-scale LLM inference through asynchronous inference and appropriate rejection sampling.
AttTok: Marrying Attribute Tokens with Generative Pre-trained Vision-Language Models towards Medical Image Understanding
Hualiang Wang (Hong Kong University of Science and Technology), Xiaomeng Li (Hong Kong University of Science and Technology)
ClassificationRecognitionTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringVision Language ModelContrastive LearningImageTextBiomedical Data
🎯 What it does: In medical image understanding tasks, the Attribute Tokens (AttTok) framework is proposed, which embeds clinical attributes (such as disease names, severity, etc.) into the GPTv model in the form of dedicated tokens. Visual-textual alignment and discriminative capability are enhanced through an attribute embedding repository, Attribute Cross-Attention (ACC) adapter, and Attribute Consistency Matching (ACM) loss.
AudioTrust: Benchmarking The Multifaceted Trustworthiness of Audio Large Language Models
Kai Li (Tsinghua University), Xinfeng Li (Nanyang Technological University)
Safty and PrivacyLarge Language ModelBenchmarkAudio
🎯 What it does: Built the first systematic trustworthiness evaluation framework for Audio Large Language Models (ALLM), named AudioTrust, covering six dimensions: fairness, hallucination, safety, privacy, robustness, and identity authentication, conducting large-scale evaluations using 4,420 real-world audio samples and 26 subtasks;
AudioX: A Unified Framework for Anything-to-Audio Generation
Zeyue Tian (Hong Kong University of Science and Technology), Yike Guo (Hong Kong University of Science and Technology)
GenerationTransformerDiffusion modelVideoTextMultimodalityAudio
🎯 What it does: Proposes the AudioX unified framework to achieve multi-modal (text, video, audio) to audio/music generation.
Auditing Black-Box LLM APIs with a Rank-Based Uniformity Test
Xiaoyuan Zhu (University of Southern California), Willie Neiswanger (University of Southern California)
Anomaly DetectionExplainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: To address the model replacement issue in black-box LLM APIs, the Rank-Based Uniformity Test (RUT) is proposed for model equivalence testing.
Augmented Radiance Field: A General Framework for Enhanced Gaussian Splatting
Yixin Yang (Shenzhen University), Hui Huang (Shenzhen University)
GenerationNeural Radiance FieldGaussian SplattingImage
🎯 What it does: Proposes an enhanced Gaussian kernel that utilizes perspective-related transparency to finely model specular reflections in scenes, and inserts it into existing 3D Gaussian sparse scenes through error-driven 2D Gaussian training and inverse Gaussian projection methods, forming an enhanced radiance field;
AUHead: Realistic Emotional Talking Head Generation via Action Units Control
Jiayi Lyu (University of the Chinese Academy of Sciences), Tat-Seng Chua (National University of Singapore)
GenerationTransformerLarge Language ModelDiffusion modelVideoMultimodalityChain-of-ThoughtAudio
🎯 What it does: This paper proposes a two-stage audio-driven speaker facial animation generation framework called AUHead. It first extracts fine-grained action unit (AU) sequences from speech using an audio language model (ALM), and then uses these AUs as conditional inputs to a diffusion model to generate speaker videos that are emotionally expressive, identity-preserving, and precisely synchronized.
Aurelius: Relation Aware Text-to-Audio Generation At Scale
Yuhang He (Microsoft Research), Vibhav Vineet (Microsoft Research)
GenerationData SynthesisTransformerLarge Language ModelDiffusion modelFlow-based ModelTextAudio
🎯 What it does: This paper proposes the Aurelius framework, which constructs a large-scale audio event set (AudioEventSet) and relation set (AudioRelSet) to provide data and evaluation platforms for relation-aware text-to-audio generation.
Aurora: Towards Universal Generative Multimodal Time Series Forecasting
Xingjian Wu (East China Normal University), Chenjuan Guo (East China Normal University)
GenerationTransformerVision Language ModelFlow-based ModelMultimodalityTime SeriesBenchmark
🎯 What it does: Built a cross-domain multimodal time series foundation model Aurora, supporting zero-shot prediction and generative probabilistic prediction.
Auto-RT: Automatic Jailbreak Strategy Exploration for Red-Teaming Large Language Models
yanjiang liu, Le Sun (University of Chinese Academy of Sciences)
Adversarial AttackTransformerLarge Language ModelReinforcement LearningTextBenchmark
🎯 What it does: Proposes the AUTO-RT framework, which leverages reinforcement learning to automatically explore jailbreak strategies in large language models, achieving higher exploitability and severity in vulnerability discovery.
AutoBio: A Simulation and Benchmark for Robotic Automation in Digital Biology Laboratory
Zhiqian Lan (University of Hong Kong), Ping Luo (University of Hong Kong)
Data SynthesisRobotic IntelligenceVision-Language-Action ModelGaussian SplattingVideoMultimodalityBenchmark
🎯 What it does: Studied AutoBio, a robot automation simulation framework for biological laboratories, and provided benchmark tasks with multiple difficulty levels;
AutoCode: LLMs as Problem Setters for Competitive Programming
Shang Zhou (University of California San Diego), Jingbo Shang (University of California San Diego)
GenerationAI Code AssistantTransformerLarge Language ModelAgentic AITextBenchmark
🎯 What it does: Utilize LLM to build a closed-loop multi-role framework AutoCode, automatically generating competitive programming problems and their complete test sets, and ensuring problem correctness through dual verification.
AutoCodeBench: Large Language Models are Automatic Code Benchmark Generators
Changzhi Zhou, Fengzong Lian (Hunyuan Team Tencent)
GenerationData SynthesisTransformerLarge Language ModelTextBenchmark
🎯 What it does: This study proposes a fully automated framework called AutoCodeGen, which can generate high-difficulty, multilingual, executable, and verifiable code generation test sets without requiring manual annotations, and uses these to construct a large-scale, balanced AutoCodeBench benchmark;
AutoDA-Timeseries: Automated Data Augmentation for Time Series
Zijun Dou (Tsinghua University), Dan Pei (Tsinghua University)
ClassificationAnomaly DetectionTime SeriesBenchmark
🎯 What it does: Proposes AutoDA-Timeseries, a general-purpose automatic data augmentation framework for time series that can learn and optimize time series augmentation strategies in a single-stage, end-to-end manner;
AutoDrive-R²: Incentivizing Reasoning and Self-Reflection Capacity for VLA Model in Autonomous Driving
Zhenlong Yuan (AMAP, Alibaba Group), Shuo Li (Case Western Reserve University)
Autonomous DrivingTransformerSupervised Fine-TuningReinforcement LearningVision-Language-Action ModelImage
🎯 What it does: AutoDrive-R² proposes a two-stage VLA framework, first performing supervised fine-tuning using the nuScenesR²-6K CoT dataset, then optimizing trajectory planning with GRPO combined with physics-constrained reinforcement learning.
AutoDV: An End-to-End Deep Learning Model for High-Dimensional Data Visualization
Wei Dai (Zhejiang University of Technology), Jicong Fan (Chinese University of Hong Kong)
Computational EfficiencyRepresentation LearningData-Centric LearningGraph Neural NetworkTransformerImageTabularBiomedical Data
🎯 What it does: Propose an end-to-end high-dimensional data visualization model AutoDV, which directly outputs 2D/3D embeddings using graph Transformer and invariant loss without parameter tuning or retraining.
Autoencoding-Free Context Compression for LLMs via Contextual Semantic Anchors
Xin Liu (Northeastern University), JingBo Zhu
CompressionRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: Propose a non-autoencoding context compression method called Semantic-Anchor Compression (SAC), which selects anchor words directly from the original context and uses bidirectional attention to aggregate global information into the key-value representations of these anchor words;
AutoEP: LLMs-Driven Automation of Hyperparameter Evolution for Metaheuristic Algorithms
Zhenxing Xu (National University of Defense Technology), Ji Wang (Peking University)
OptimizationHyperparameter SearchTransformerLarge Language ModelChain-of-Thought
🎯 What it does: Designed the AutoEP framework, leveraging large language models (LLM) and real-time exploratory landscape analysis (ELA) to achieve zero-training dynamic hyperparameter tuning;
AutoFigure: Generating and Refining Publication-Ready Scientific Illustrations
Minjun Zhu (Zhejiang University), Yue Zhang (Westlake University)
GenerationData SynthesisTransformerLarge Language ModelAgentic AIVision Language ModelMultimodalityBenchmark
🎯 What it does: Proposed a benchmark dataset called FigureBench specifically for generating scientific illustrations from long texts, and designed an agent framework named AUTOFIGURE based on the Reasoned Rendering paradigm, achieving end-to-end automation for generating publication-quality scientific images from text;
AutoFly: Vision-Language-Action Model for UAV Autonomous Navigation in the Wild
Xiaolou Sun (Southeast University), Hui Xiong (Hong Kong University Of Science And Technology)
Data SynthesisDepth EstimationAutonomous DrivingTransformerReinforcement LearningVision Language ModelVision-Language-Action ModelImageTextMultimodality
🎯 What it does: Designed and implemented AutoFly, an end-to-end Vision-Language-Action (VLA) model for autonomous drone navigation in unknown outdoor environments using only coarse-grained language or location information.
AutoGPS: Automated Geometry Problem Solving via Multimodal Formalization and Deductive Reasoning
Bowen Ping (Xi'an Jiaotong University), Chengyou Jia (Xi'an Jiaotong University)
Graph Neural NetworkTransformerLarge Language ModelVision Language ModelMultimodalityBenchmarkChain-of-Thought
🎯 What it does: Propose the AutoGPS framework, leveraging a multi-modal problem formalizer and deductive symbolic reasoner to achieve automatic geometric problem solving, outputting reliable and interpretable reasoning steps.
AutoLibra: Agent Metric Induction from Open-Ended Human Feedback
Hao Zhu (Stanford University), Diyi Yang (University of Pennsylvania)
Reinforcement Learning from Human FeedbackLarge Language ModelAgentic AIText
🎯 What it does: Built the AutoLibra framework, which automatically induces fine-grained evaluation metrics from open human feedback to assess and improve agent behavior.
Automata Learning and Identification of the Support of Language Models
Satwik Bhattamishra (University of Oxford), Varun Kanade (University of Oxford)
Representation LearningTransformerLarge Language ModelText
🎯 What it does: Studied how to learn and identify deterministic finite automata (DFA) under the next symbol prediction (NSP) setting, given only membership and continuation labels for positive samples and their prefixes, and proposed the L⋆nsp algorithm to extract DFA from language models.
Automated Formalization via Conceptual Retrieval-Augmented LLMs
Wangyue Lu (Northeastern University), Ge Yu (Northeastern University)
AI Code AssistantTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: Proposed and implemented the CRAMF (Concept-driven Retrieval-Augmented Mathematical Formalization) framework, enhancing the performance of LLMs in automated formalization by retrieving formal definitions of mathematical concepts.
Automated Interpretability Metrics Do Not Distinguish Trained and Random Transformers
Thomas Heap (University of Bristol), Laurence Aitchison (University of Bristol)
Explainability and InterpretabilityTransformerAuto EncoderText
🎯 What it does: Using sparse autoencoders (SAE) to evaluate whether interpretability metrics of trained Transformers can distinguish them from randomly initialized Transformers.
Automated Stateful Specialization for Adaptive Agent Systems
Myan Vu (University of Auckland), Mayank Goel (University of Auckland)
Large Language ModelAgentic AIText
🎯 What it does: Developed the ASPEC framework to realize stateful lifecycle management of expert agents, including automatically evolved discovery of expert agents, experience-based cultivation, and efficient reuse of expert knowledge across queries through a 'retain-then-escalate' control strategy.
Automatic and Structure-Aware Sparsification of Hybrid Neural ODEs with Application to Glucose Prediction
Bob Junyi Zou (Stanford University), Lu Tian (Stanford University)
Explainability and InterpretabilityComputational EfficiencyTime SeriesBiomedical DataElectronic Health RecordsOrdinary Differential Equation
🎯 What it does: Propose a graph-based gradient sparsification framework that automatically selects and optimizes states and edges in hybrid neural ODEs, improving prediction accuracy and robustness.
Automatic Dialectic Jailbreak: A Framework for Generating Effective Jailbreak Strategies
Jianghai Yu (Auburn University), Dejing Dou (Fudan University)
OptimizationAdversarial AttackText
🎯 What it does: Modeling jailbreak attacks as a Stackelberg multi-objective game and employing Hegel's dialectical Thesis-Antithesis-Synthesis cycle to automatically generate diverse and robust jailbreak strategies.
Automatic Image-Level Morphological Trait Annotation for Organismal Images
Vardaan Pahuja (Ohio State University), Yu Su (Ohio State University)
ClassificationGenerationExplainability and InterpretabilityRepresentation LearningTransformerLarge Language ModelPrompt EngineeringVision Language ModelAuto EncoderContrastive LearningImageMultimodality
🎯 What it does: Extract spatially interpretable neurons from insect images using sparse autoencoders (SAE), combine them with a multimodal language model (MLLM) to automatically generate morphological feature descriptions, and construct the BIOSCAN-TRAITS dataset.
Automatic Stage Lighting Control: Is it a Rule-Driven Process or Generative Task?
Zijian Zhao, Xiaoyu Zhang (The University of Hong Kong)
GenerationTransformerSupervised Fine-TuningGenerative Adversarial NetworkMultimodalityAudio
🎯 What it does: This paper proposes an end-to-end generative model called Skip-BART, which directly generates the hue and brightness of stage lighting from music, achieving complete automatic control from music to lighting;
Automating the Refinement of Reinforcement Learning Specifications
Tanmay Ambadkar (Pennsylvania State University), Abhinav Verma (Pennsylvania State University)
Reinforcement Learning
🎯 What it does: Propose the AUTOSPEC framework, which can automatically identify and refine coarse logical specifications in reinforcement learning tasks based on SpectRL logical specifications, thereby enhancing the feasibility of policy learning.
AutoMetrics: Approximate Human Judgments with Automatically Generated Evaluators
Michael J Ryan, Diyi Yang (Stanford University)
Explainability and InterpretabilityData-Centric LearningLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: The AutoMetrics framework automatically generates interpretable, task-specific evaluation metrics using a small amount of human feedback, and optimizes them through regression to be highly correlated with human judgments.
Autonomous Functional Play with Correspondence-Driven Trajectory Warping
William Liang (University of Pennsylvania), Dinesh Jayaraman (University of Pennsylvania)
Robotic IntelligenceTransformerReinforcement LearningDiffusion modelImageVideoMultimodality
🎯 What it does: Developed a Tether method that achieves autonomous functional play through semantic keypoint correspondence and trajectory distortion, generating a large amount of high-quality demonstration data with a small number of demonstrations.
AutoQD: Automatic Discovery of Diverse Behaviors with Quality-Diversity Optimization
Saeed Hedayatian (University of Southern California), Stefanos Nikolaidis (University of Southern California)
OptimizationReinforcement LearningBenchmark
🎯 What it does: Developed AutoQD, which automatically generates behavioral descriptors and combines them with QD algorithms to discover diverse and high-quality reinforcement learning strategies.
Autoregressive Image Generation with Randomized Parallel Decoding
Haopeng Li (Westlake University), Huan Wang (Westlake University)
GenerationTransformerImageText
🎯 What it does: Propose the ARPG framework, achieving parallel image generation in random order;
Autoregressive Models Rival Diffusion Models at ANY-ORDER Generation
Tianqi Du (Peking University), Yisen Wang (Peking University)
GenerationTransformerLarge Language ModelDiffusion modelText
🎯 What it does: Proposed the A3 framework, which enables arbitrary order and subset autoregressive modeling, and implemented two-stream attention networks and progressive training;
Autoregressive Visual Decoding from EEG Signals
Sicheng Dai (Chinese Academy of Sciences), Qiwei Ye (Beijing Academy of Artificial Intelligence)
GenerationTransformerAuto EncoderContrastive LearningImageBiomedical Data
🎯 What it does: Propose AVDE, a lightweight framework that automatically generates visual images from EEG signals through autoregressive methods;
Autoregressive-based Progressive Coding for Ultra-Low Bitrate Image Compression
Ziyuan Zhang (Tsinghua University), Han Qiu (Tsinghua University)
CompressionVision Language ModelImage
🎯 What it does: Proposed an autoregressive progressive coding (ARPC) based on the visual autoregressive model (VAR) for ultra-low bitrate image compression;
AutoSP: Unlocking Long-Context LLM Training Via Compiler-Based Sequence Parallelism
Ahan Gupta (University of Illinois Urbana-Champaign), Minjia Zhang (University of Illinois Urbana-Champaign)
OptimizationComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: Propose AutoSP, an automatic sequence parallelism and activation checkpointing optimization based on the PyTorch 2.0 compiler, to increase the trainable length of long-context LLMs.
AutoTool: Automatic Scaling of Tool-Use Capabilities in RL via Decoupled Entropy Constraints
Yirong Zeng (Harbin Institute of Technology), Ting Liu (Huawei Technologies)
TransformerSupervised Fine-TuningReinforcement LearningTextBenchmark
🎯 What it does: Propose an RL training framework that separates short and long reasoning paths and adaptively regulates entropy constraints, automatically scaling the tool usage reasoning length according to problem difficulty.