ICLR 2026 Papers — Page 28
International Conference on Learning Representations · 5356 papers
Long-Context Generalization with Sparse Attention
Pavlo Vasylenko (Instituto Superior Técnico Universidade De Lisboa), Marcos Vinicius Treviso
RetrievalComputational EfficiencyRepresentation LearningTransformerText
🎯 What it does: Replace the softmax attention in Transformers with α-entmax, and propose a learnable temperature and length-adjustable Adaptive-Scalable α-Entmax (ASEntmax) to achieve long-context generalization
Long-Document QA with Chain-of-Structured-Thought and Fine-Tuned SLMs
Zhuowen Liang (Hong Kong University of Science and Technology), Nan Tang (Hong Kong University of Science and Technology)
TransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextChain-of-Thought
🎯 What it does: Propose a framework that converts long document question answering into structured output, combining structured ideas generated by LLMs with reinforcement learning training on small models.
Long-range Modeling and Processing of Multimodal Event Sequences
Jichu Li (Renmin University of China), Quyu Kong
Computational EfficiencyRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelMultimodalityTime Series
🎯 What it does: Propose the MM-TPP framework, integrating visual, textual, temporal, and event type information into multi-modal event sequence modeling, and enhancing long-sequence processing capability through adaptive time similarity compression.
Long-tailed Test-Time Adaptation for Vision-Language Models
Xucong Wang (University Of Science And Technology Of China), Yang Wang (Suzhou Institute For Advanced Research)
Domain AdaptationVision Language ModelMultimodality
🎯 What it does: This paper proposes L-TTA, a test-time adaptation (TTA) framework for vision-language models (VLMs) on long-tail test sets, achieving dynamic performance improvement of models on unlabeled, sequentially arriving data streams.
Long-Text-to-Image Generation via Compositional Prompt Decomposition
Jen-Yuan Huang (Peking University), Yilun Du (Harvard University)
GenerationTransformerPrompt EngineeringDiffusion modelImageTextMultimodality
🎯 What it does: This paper proposes a compositional long-text-to-image generation framework called PRISM, which leverages the short-text capabilities of pre-trained text-image diffusion models. It decomposes long paragraphs into sub-components directly understandable by the model and merges the noise predictions of sub-components based on an energy benchmark at each denoising step to achieve high-fidelity generation of descriptive paragraphs.
LongHorizonUI: A Unified Framework for Robust long-horizon Task Automation of GUI Agent
Bin Kang (Chengdu Institute of Computer Applications, Chinese Academy of Sciences), Zhuotao Tian (Shenzhen Loop Area Institute)
Computational EfficiencyTransformerLarge Language ModelAgentic AIVision Language ModelMultimodalitySequentialBenchmarkChain-of-Thought
🎯 What it does: Developed the LongHorizonUI framework, leveraging multimodal large language models to enhance reliability and robustness in long-sequence GUI tasks, and proposed the LongGUIBench benchmark;
LongLive: Real-time Interactive Long Video Generation
Shuai Yang (NVIDIA), Yukang Chen (NVIDIA)
GenerationComputational EfficiencyTransformerPrompt EngineeringVideo
🎯 What it does: Propose LONGLIVE, a real-time interactive long video generation framework that supports continuous user input during the generation process, capable of producing high-quality videos up to 240 seconds in length.
LongRLVR: Long-Context Reinforcement Learning Requires Verifiable Context Rewards
Guanzheng Chen (National University Of Singapore), Lidong Bing (MiroMind Ai)
Data SynthesisTransformerSupervised Fine-TuningReinforcement LearningTextRetrieval-Augmented Generation
🎯 What it does: In long-text reasoning tasks, LongRLVR is proposed, adding a verifiable context reward to RLVR to explicitly encourage the model to first retrieve relevant evidence before providing answers.
LongWriter-Zero: Mastering Ultra-Long Text Generation via Reinforcement Learning
Yuhao Wu (Singapore University of Technology and Design), Juanzi Li (Tsinghua University)
GenerationTransformerLarge Language ModelReinforcement LearningTextChain-of-Thought
🎯 What it does: Training an LLM from scratch using reinforcement learning to achieve the long-text generation model LongWriter-Zero
Look Back to Reason Forward: Revisitable Memory for Long-Context LLM Agents
Yaorui Shi (University of Science and Technology of China), An Zhang (University of Science and Technology of China)
Reinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningAgentic AITextRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Propose ReMemR1, an LLM memory agent that integrates historical callback queries into the 'reading while memorizing' framework, achieving nonlinear reasoning and significantly improving long-text question answering performance.
Look Carefully: Adaptive Visual Reinforcements in Multimodal Large Language Models for Hallucination Mitigation
Xingyu Zhu (University of Science and Technology of China), Xiangnan He (Nanyang Technological University)
Explainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelReinforcement LearningVision Language ModelImageTextMultimodality
🎯 What it does: Proposed the AIR (Adaptive Visual Reinforcement) framework, which leverages prototype cropping and regularizable optimal transport to adaptively reinforce the focus of multimodal large language models on visual information during inference, thereby significantly reducing hallucinations.
Look-ahead Reasoning with a Learned Model in Imperfect Information Games
Ondrej Kubicek (Czech Technical University in Prague), Viliam Lisý (Czech Technical University in Prague)
Reinforcement LearningWorld Model
🎯 What it does: This paper proposes the LAMIR algorithm, which can perform forward reasoning using an abstract model learned from gameplay without requiring explicit game rules or experienced domain knowledge;
Lookahead Tree-Based Rollouts for Enhanced Trajectory-Level Exploration in Reinforcement Learning with Verifiable Rewards
Shangyu Xing (Nanjing University), Xiang Ren (University of Southern California)
Reinforcement LearningText
🎯 What it does: Propose a tree-search-based rolling strategy called LATR to enhance trajectory-level diversity in RLVR, and integrate it into GRPO and DAPO algorithms;
LookaheadKV: Fast and Accurate KV Cache Eviction by Glimpsing into the Future without Generation
Jinwoo Ahn (Samsung Research), Yongkweon Jeon (Samsung Research)
Computational EfficiencyTransformerTextBenchmark
🎯 What it does: Designed and trained a lightweight KV cache eviction framework called LookaheadKV, which can estimate future attention during the pre-inference phase by learning lookahead tokens and LoRA modules, thereby determining which KV pairs to retain.
Lookup multivariate Kolmogorov-Arnold Networks
Sergey Pozdnyakov (École Polytechnique Fédérale de Lausanne), Philippe Schwaller (École Polytechnique Fédérale de Lausanne)
ClassificationComputational EfficiencyImageTabular
🎯 What it does: In deep learning models, traditional high-dimensional linear layers are replaced with lookup multivariate Kolmogorov-Arnold networks (lmKANs), achieving a large number of trainable parameters while inference FLOPs are only twice that of linear layers; simultaneously, custom CUDA kernels are provided to accelerate inference.
LoongRL: Reinforcement Learning for Advanced Reasoning over Long Contexts
Siyuan Wang (Microsoft Research Asia), Mao Yang (Microsoft Research Asia)
Data SynthesisLarge Language ModelReinforcement LearningTextRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Training large models to achieve long-context reasoning capabilities through reinforcement learning and KeyChain data construction.
LoopFormer: Elastic-Depth Looped Transformers for Latent Reasoning via Shortcut Modulation
Ahmadreza Jeddi (University of Toronto), Babak Taati (University of Toronto)
Computational EfficiencyRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: Designed and trained a cyclic Transformer model called LoopFormer with elastic depth, supporting language modeling and inference under different computational budgets.
Loopholing Discrete Diffusion: Deterministic Bypass of the Sampling Wall
Mingyu Jo (KAIST), Sungjin Ahn (KAIST)
GenerationTransformerDiffusion modelText
🎯 What it does: Propose the Loopholing mechanism and Loopholing Discrete Diffusion Models (LDDMs), which introduce a deterministic hidden layer path into discrete diffusion models to avoid distribution information collapse caused by the sampling wall;
LoRA meets Riemannion: Muon Optimizer for Parametrization-independent Low-Rank Adapters
Vladimir Bogachev (HSE University), Maxim Rakhuba (HSE University)
OptimizationImageText
🎯 What it does: Propose a fully Riemannian LoRA framework that directly optimizes low-rank adapters on a fixed-rank matrix manifold to resolve parameterization ambiguity;
LoRA-Mixer: Coordinate Modular LoRA Experts Through Serial Attention Routing
Wenbing Li (Huazhong University of Science and Technology), Wei Yang (Huazhong University of Science and Technology)
Computational EfficiencyAI Code AssistantTransformerMixture of ExpertsTextBenchmark
🎯 What it does: Developed a hybrid expert framework called LoRA-Mixer, which routes LoRA experts to the attention projection layer, achieving efficient multi-task adaptation.
LoRA-S: An Efficient Low Rank Adaptation scheme via Sylvester equation
Jinyang ZHENG, Tong Wu (Hong Kong University of Science and Technology)
OptimizationComputational EfficiencySupervised Fine-TuningImageText
🎯 What it does: Propose the LoRA-S framework, which couples LoRA with pre-trained optimizers via horizontal enhancement theory to achieve efficient low-rank adaptation without weight decay.
LoRAGen: Structure-Aware Weight Space Learning for LoRA Generation
Hao Huang (Beijing Jiaotong University), Yong Li (Tsinghua University)
GenerationOptimizationMixture of ExpertsDiffusion modelAuto EncoderTextBenchmark
🎯 What it does: Studied the LoRA parameter generation problem, proposing the structure-aware LoRAGen method, which can directly synthesize LoRA adapters based on natural language task descriptions;
LORE: Jointly Learning The Intrinsic Dimensionality and Relative Similarity Structure from Ordinal Data
Vivek Anand (Georgia Institute of Technology), Christopher John Rozell (Georgia Institute of Technology)
Representation LearningImage
🎯 What it does: Developed the LORE framework, which jointly learns ordinal embeddings and their intrinsic dimensionality.
Lossless Vocabulary Reduction for Auto-Regressive Language Models
Daiki Chijiwa (NTT Computer and Data Science Laboratories, NTT Corporation), Susumu Takeuchi (NTT Computer and Data Science Laboratories, NTT Corporation)
GenerationCompressionRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: Proposed a theoretical framework and algorithm that can compress the vocabulary of any autoregressive language model into an arbitrary subword vocabulary while maintaining the quality of text generation; this method enables models with different vocabularies to collaborate on the same subword vocabulary, particularly for model ensembling.
Lossy Common Information in a Learnable Gray-Wyner Network
Anderson de Andrade (Simon Fraser University), Ivan V. Bajic (Simon Fraser University)
CompressionComputational EfficiencyRepresentation LearningImage
🎯 What it does: This paper proposes a learnable Gray-Wyner network that separates shared information and task-specific information through a three-channel rate compression scheme, achieving efficient encoding and distributed inference for multi-task vision models.
Lost in the Non-convex Loss Landscape: How to Fine-tune the Large Time Series Model?
Xu Zhang (Fudan University), Wei Wang (Fudan University)
Anomaly DetectionOptimizationRecurrent Neural NetworkSupervised Fine-TuningTime SeriesBenchmark
🎯 What it does: Proposes the Smoothed Full Fine-tuning (SFF) technique, which smooths the loss landscape and enhances downstream fine-tuning performance by performing linear interpolation between pre-trained LTSM and randomly initialized models.
Lost in Tokenization: Context as the Key to Unlocking Biomolecular Understanding in Scientific LLMs
Kai Zhuang (Shanghai Artificial Intelligence Laboratory), Cheng Tan (Shanghai Artificial Intelligence Laboratory)
TransformerLarge Language ModelBiomedical Data
🎯 What it does: Proposes a scientific large language model method that replaces raw biomolecular sequence inputs with high-level structured context, termed the 'context-driven' mode;
LouisKV: Efficient KV Cache Retrieval for Long Input-Output Sequences
Wenbo Wu (Peking University), Jie Zhang (Peking University)
RetrievalComputational EfficiencyTextSequential
🎯 What it does: Proposed the LouisKV framework for efficient retrieval of KV cache, significantly improving inference speed and memory utilization in scenarios with long inputs/outputs.
Low rank adaptation of chemical foundation models generate effective odorant representations
Grant D. McConachie (Boston University), Brian DePasquale (Boston University)
Representation LearningDrug DiscoveryTransformerSupervised Fine-TuningTabularBiomedical Data
🎯 What it does: What they did: Conducted a systematic evaluation of the performance of chemical foundation models in predicting odorant-receptor affinity, and proposed and trained the LORAX model to generate superior olfactory representations through LoRA fine-tuning of chemical models.
Low Rank Transformer for Multivariate Time Series Anomaly Detection and Localization
Charalampos Shimillas (University of Cyprus), Marios Polycarpou (University of Cyprus)
Anomaly DetectionExplainability and InterpretabilityTransformerTime Series
🎯 What it does: Study the learning process of Transformer in multivariate time series anomaly detection, propose Low-Rank Transformer (ALoRa-T) for anomaly detection, and design ALoRa-Loc to achieve anomaly localization and interpretability.
Low-Latency Neural LiDAR Compression with 2D Context Models
Rui Song (Hong Kong University of Science and Technology), Jun Zhang (Hong Kong University of Science and Technology)
CompressionAuto EncoderOptical FlowMultimodalityPoint Cloud
🎯 What it does: Designed a fully 2D context model called RangeCM, achieving low-latency LiDAR point cloud geometry and intensity compression.
Low-Pass Filtering Improves Behavioral Alignment of Vision Models
Max Wolff (Max Planck Institute for Intelligent Systems), Wieland Brendel (Max Planck Institute for Intelligent Systems)
Explainability and InterpretabilityConvolutional Neural NetworkTransformerImageBenchmark
🎯 What it does: Significantly improve the behavioral consistency of visual models with humans (error consistency and shape preference) by applying low-pass filtering (blurring or downsampling) to the input during testing
Low-pass Personalized Subgraph Federated Recommendation
Wooseok Sim (Sungkyunkwan University), Hogun Park (Sungkyunkwan University)
Recommendation SystemFederated LearningGraph Neural NetworkContrastive LearningGraph
🎯 What it does: Propose a low-pass spectral filtering personalized subgraph federated recommendation framework, LPSFed, to address subgraph structural imbalance and popularity bias issues.
Low-Rank Few-Shot Node Classification by Node-Level Graph Diffusion
Yancheng Wang (Arizona State University), Yingzhen Yang (Arizona State University)
ClassificationMeta LearningGraph Neural NetworkDiffusion modelAuto EncoderGraph
🎯 What it does: Propose a low-rank few-shot node classification framework called LR-FGDM, which generates synthetic support nodes and edges using a node-level graph diffusion model, and improves few-shot node classification performance by combining it with a low-rank regularized propagation classifier.
LRIM: a Physics-Based Benchmark for Provably Evaluating Long-Range Capabilities in Graph Learning
Joël Mathys, Francesco Alesiani (NEC Laboratories Europe)
Graph Neural NetworkTransformerGraphBenchmarkPhysics Related
🎯 What it does: Propose a physics-based provable long-range graph learning benchmark (LRIM) based on the Ising model, generating datasets of varying scales and long-range difficulty through controllable parameters, and converting the energy change prediction task into a node regression problem;
LS-Merge: Merging Language Models in Latent Space
Bedionita Soro (KAIST), Sung Ju Hwang (KAIST)
Knowledge DistillationRepresentation LearningTransformerAuto EncoderText
🎯 What it does: By encoding the weights of pre-trained LLMs into a low-dimensional manifold learned by a Transformer-VAE, and then performing alignment (OT) and interpolation on this manifold, model parameter fusion is achieved, supporting both homogeneous and heterogeneous model merging.
LSA: Layer-wise Sparsity Allocation for Large Language Model Pruning Based on Minimal Linear Reconstruction Error
Zhiguo Yang (University of Electronic Science and Technology of China), Jian Cheng (University of Electronic Science and Technology of China)
Computational EfficiencyKnowledge DistillationTransformerLarge Language ModelTextBenchmark
🎯 What it does: Propose a hierarchical sparse allocation method LSA based on minimum linear reconstruction error for unsupervised sparse pruning of large language models while maintaining high performance.
LucidFlux: Caption-Free Universal Image Restoration via a Large-Scale Diffusion Transformer
Song Fei (Hong Kong University of Science and Technology), Lei Zhu (Hong Kong University of Science and Technology)
RestorationTransformerVision Language ModelDiffusion modelImage
🎯 What it does: Developed a caption-free photorealistic image restoration framework called LucidFlux, leveraging the large-scale Diffusion Transformer Flux.1 to perform lossless restoration of real-world low-quality images.
LUMINA: Detecting Hallucinations in RAG System with Context–Knowledge Signals
Samuel Yeh (University of Wisconsin-Madison), Tanwi Mallick (Argonne National Laboratory)
Anomaly DetectionTransformerTextRetrieval-Augmented Generation
🎯 What it does: Developed an unsupervised hallucination detection framework called LUMINA, which evaluates the credibility of answers generated by RAG systems using two signals: external context utilization rate and internal knowledge utilization rate.
LumiTex: Towards High-Fidelity PBR Texture Generation with Illumination Context
Jingzhi Bao (Chinese University of Hong Kong), Xiaoguang Han (Chinese University of Hong Kong)
GenerationData SynthesisTransformerDiffusion modelMultimodalityMesh
🎯 What it does: Developed LumiTex, an end-to-end multi-branch framework for generating high-fidelity PBR textures and achieving seamless texture stitching;
Lumos-1: On Autoregressive Video Generation with Discrete Diffusion from a Unified Model Perspective
Hangjie Yuan (Tsinghua University), Yi Yang (Zhejiang University)
GenerationTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelDiffusion modelImageVideoTextMultimodality
🎯 What it does: Propose Lumos-1, an autoregressive video generation model based on LLM, which employs discrete diffusion for efficient generation;
LumosX: Relate Any Identities with Their Attributes for Personalized Video Generation
Jiazheng Xing (Zhejiang University), Yong Liu (DAMO Academy, Alibaba Group)
GenerationTransformerDiffusion modelVideo
🎯 What it does: This paper proposes the Lumos X framework for accurately aligning facial attributes in multi-subject videos, enabling personalized video generation.
LVTINO: LAtent Video consisTency INverse sOlver for High Definition Video Restoration
Alessio Spagnoletti (Universite Paris Cite), Marcelo Pereyra (Heriot-Watt University)
RestorationSuper ResolutionComputational EfficiencyDiffusion modelScore-based ModelVideoStochastic Differential Equation
🎯 What it does: Proposed a zero-shot or P-n-P inverse solver, L A TINO, for high-resolution video restoration, capable of combining measurement consistency and generative prior without gradient backpropagation.
LycheeDecode: Accelerating Long-Context LLM Inference via Hybrid-Head Sparse Decoding
Gang Lin (Harbin Institute of Technology), Min Zhang (Harbin Institute of Technology)
Computational EfficiencyKnowledge DistillationTransformerLarge Language ModelText
🎯 What it does: Proposed a long-context LLM inference acceleration method called LycheeDecode, which fine-grained partitions attention heads into retrieval heads and sparse heads, dynamically selects key tokens via retrieval heads to reduce KV cache access, achieving efficient inference.
Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation
Sherwin Bahmani (NVIDIA), Xuanchi Ren (NVIDIA)
GenerationData SynthesisKnowledge DistillationTransformerDiffusion modelAuto EncoderGaussian SplattingImageVideoText
🎯 What it does: Leverage a self-distillation framework to transfer the implicit 3D knowledge from video diffusion models to 3D Gaussian Splatting (3DGS), enabling the instant generation of high-quality, renderable 3D/4D scenes from single-image or single-video inputs.
M$^2$-Miner: Multi-Agent Enhanced MCTS for Mobile GUI Agent Data Mining
Rui Lv (Ant Group), Lei Zhao (Zhejiang University)
Data SynthesisData-Centric LearningLarge Language ModelReinforcement LearningAgentic AIVision-Language-Action ModelImageMultimodalityBenchmark
🎯 What it does: Proposed a fully automated mobile GUI agent data mining framework named M2-Miner, which efficiently collects intent-trajectory data by leveraging MCTS with collaborative multi-agents (InferAgent, OrchestraAgent, JudgeAgent);
M$^3$E: Continual Vision-and-Language Navigation via Mixture of Macro and Micro Experts
Yongliang Jiang (South China University of Technology), Shengfeng He (Singapore Management University)
Autonomous DrivingGraph Neural NetworkTransformerMixture of ExpertsVision Language ModelSimultaneous Localization and MappingMultimodality
🎯 What it does: Propose a hybrid macro-micro expert model (M3E) for continuous vision-language navigation, which separates global scene reasoning and local semantic alignment through dual routing, combined with dynamic momentum updates to achieve continuous learning without replay;
M3CoTBench: Benchmark Chain-of-Thought of MLLMs in Medical Image Understanding
Juntao Jiang (Zhejiang University), Shuicheng YAN
TransformerLarge Language ModelPrompt EngineeringVision Language ModelBiomedical DataBenchmarkChain-of-Thought
🎯 What it does: Proposed and constructed M3CoTBench to evaluate the Chain-of-Thought (CoT) quality of multi-modal large language models in medical image understanding, covering 1,079 images, 24 modalities, 13 task categories, and corresponding intermediate reasoning steps;
MAC-AMP: A Closed-Loop Multi-Agent Collaboration System for Multi-Objective Antimicrobial Peptide Design
Gen Zhou (Western University), Pingzhao Hu (Western University)
OptimizationExplainability and InterpretabilityDrug DiscoveryReinforcement Learning from Human FeedbackProtein Structure PredictionTransformerLarge Language ModelReinforcement LearningAgentic AIBiomedical Data
🎯 What it does: A closed-loop multi-agent collaboration system, MAC-AMP, was constructed to achieve multi-objective optimization in antimicrobial peptide (AMP) design.
Machine Unlearning under Retain–Forget Entanglement
Jingpu Cheng (National University of Singapore), CHI ZHANG
OptimizationSafty and PrivacyImageText
🎯 What it does: Address the retain-forget entanglement problem where machine learning models experience performance degradation when forgetting specific data due to the high correlation between retained and forgotten samples;
MAD-Logic: Multi-Agent Debate Enhances Symbolic Translation and Reasoning
Haocheng Yang (Peking University), Yisen Wang (Peking University)
Computational EfficiencyAgentic AIPrompt EngineeringTextChain-of-Thought
🎯 What it does: Proposes a multi-agent debate framework that conducts logical question answering using multiple symbolic languages and natural language, achieving efficient interaction through sparse communication, significantly enhancing translation and reasoning quality.
MAGE: Multi-scale Autoregressive Generation for Offline Reinforcement Learning
Chenxing Lin (Xiamen University), Cheng Wang (Xiamen University)
GenerationTransformerReinforcement LearningAuto EncoderBenchmark
🎯 What it does: Propose MAGE—a multi-scale autoregressive generation method that utilizes a quantized autoencoder and multi-scale Transformer to progressively generate offline reinforcement learning trajectories from coarse to fine, addressing long-horizon sparse reward tasks;
MAGO: Beyond Fixed Hyperparameters with Multi-Objective Pareto Optimization for Hybrid LLM Reasoning
Hongcheng Ding (Zhejiang University of Finance and Economics), LIU QINGYU
OptimizationComputational EfficiencyHyperparameter SearchTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Propose the MAGO framework, combining multi-objective optimization with hybrid reasoning and Pareto frontier maintenance to achieve dynamic weight adjustment
MAGREF: Masked Guidance for Any-Reference Video Generation with Subject Disentanglement
Yufan Deng (Peking University), Chongyang Ma (ByteDance)
GenerationData SynthesisTransformerLarge Language ModelDiffusion modelAuto EncoderVideoText
🎯 What it does: Proposed MAGREF, which can generate high-quality videos with multiple subjects and cross-categories under the condition of only being given multiple reference images and text prompts.
Making Slow Thinking Faster: Compressing LLM Chain-of-Thought via Step Entropy
Zeju Li, Qiang Xu (Chinese University of Hong Kong)
OptimizationComputational EfficiencyTransformerSupervised Fine-TuningReinforcement LearningTextBenchmarkChain-of-Thought
🎯 What it does: Propose a chain-of-thought compression method based on step entropy, trimming low-entropy steps to improve inference efficiency.
Making, Not Taking, the Best of N
Ammar Khairi (Cohere Labs), Julia Kreutzer (Cohere Labs)
GenerationData SynthesisTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper proposes an aggregation method called Fusion-of-N (FUSION), which uses large language models (LLMs) as judges to synthesize multiple candidate generations into a higher-quality final output, replacing the traditional Best-of-N (BON) selection strategy.
Mamba-3: Improved Sequence Modeling using State Space Principles
Aakash Lahoti (Carnegie Mellon University), Albert Gu (Carnegie Mellon University)
Computational EfficiencyRepresentation LearningLarge Language ModelTextSequential
🎯 What it does: Propose Mamba-3, a linear sequence layer based on state space models (SSM), significantly improving language modeling, state tracking, and inference efficiency.
MambaSL: Exploring Single-Layer Mamba for Time Series Classification
Yoo-Min Jung (Seoul National University), Leekyung Kim (Seoul National University)
ClassificationHyperparameter SearchTime SeriesBenchmark
🎯 What it does: Propose a single-layer Mamba structure (MambaSL) for time series classification, and improve input projection, time-varying characteristics, skip connections, and adaptive pooling through four hypotheses.
MambaVoiceCloning: Efficient and Expressive Text-to-Speech via State-Space Modeling and Diffusion Control
Sahil Kumar (Yeshiva University), Youshan Zhang (Chuzhou University)
GenerationTransformerDiffusion modelAudio
🎯 What it does: Propose a TTS encoder entirely based on state-space models (SSM) that does not use attention or recurrent networks during inference, maintaining linear time and bounded activation;
ManagerBench: Evaluating the Safety-Pragmatism Trade-off in Autonomous LLMs
Adi Simhi (Technion Israel Institute Of Technology), Yonatan Belinkov (Technion Israel Institute Of Technology)
GenerationData SynthesisSafty and PrivacyTransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: Proposed the MANAGERBENCH benchmark to evaluate the decision-making behavior of LLMs when safety and practicality objectives conflict;
Mango-GS: Enhancing Spatio-Temporal Consistency in Dynamic Scenes Reconstruction using Multi-Frame Node-Guided 4D Gaussian Splatting
Tingxuan Huang (Tsinghua University), Bin Wang (Tsinghua University)
GenerationTransformerGaussian SplattingVideo
🎯 What it does: Proposes a multi-frame efficient 4D Gaussian point cloud reconstruction framework called Mango-GS based on sparse control nodes, capable of real-time rendering while preserving spatial details and temporal consistency in dynamic scenes.
ManipEvalAgent: Promptable and Efficient Evaluation Framework for Robotic Manipulation Policies
Yiteng Chen (South China University of Technology), Qingyao Wu (South China University of Technology)
Robotic IntelligenceTransformerLarge Language ModelAgentic AIPrompt EngineeringVision Language ModelVideoTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Proposed ManipEvalAgent, a promptable, few-shot, multi-round, and interpretable robotic manipulation strategy evaluation framework;
Manipulation as in Simulation: Enabling Accurate Geometry Perception in Robots
Minghuan Liu (ByteDance Seed), Bingyi Kang (ByteDance Seed)
Depth EstimationDomain AdaptationRobotic IntelligenceTransformerImage
🎯 What it does: This paper proposes Camera Depth Models (CDMs), providing plug-and-play denoising and depth enhancement for commonly used depth cameras, enabling robots to obtain high-quality geometric information comparable to simulated environments in real-world scenarios, thus achieving long-term zero-shot simulation-to-reality operations.
Many Eyes, One Mind: Temporal Multi-Perspective and Progressive Distillation for Spiking Neural Networks
Kai Sun, Levin Kuhlmann (Northeastern University)
ClassificationKnowledge DistillationSpiking Neural NetworkTransformerImage
🎯 What it does: Proposes a unified knowledge distillation framework, MEOM, combining multi-perspective temporal distillation and progressive consistency training to enhance the overall accuracy and time flexibility of spiking neural networks during truncated inference.
Many-for-Many: Unify the Training of Multiple Video and Image Generation and Manipulation Tasks
Ruibin Li (ByteDance), Lei Zhang (Hong Kong Polytechnic University)
GenerationData SynthesisTransformerDiffusion modelFlow-based ModelImageVideoTextMultimodality
🎯 What it does: Proposes the Many-for-Many (MfM) framework, training a single model to perform various video and image generation and editing tasks.
MANZANO: A Simple and Scalable Unified Multimodal Model with a Hybrid Vision Tokenizer
Yanghao Li (Apple), Zhifeng Chen (Apple)
RecognitionGenerationTransformerLarge Language ModelVision Language ModelDiffusion modelMultimodality
🎯 What it does: Proposed Manzano, a unified multimodal large language model that integrates visual understanding and image generation.
Map as a Prompt: Learning Multi-Modal Spatial-Signal Foundation Models for Cross-scenario Wireless Localization
Yong Chu (Harbin Institute of Technology), Yue Yu (Pengcheng Laboratory)
Graph Neural NetworkTransformerPrompt EngineeringMultimodality
🎯 What it does: Propose SigMap, a wireless localization foundation model that learns robust signal representations using periodic adaptive masking and achieves cross-scenario adaptive localization by constructing geographic prompts through 3D map information.
Map the Flow: Revealing Hidden Pathways of Information in VideoLLMs
Minji Kim (Seoul National University), Bohyung Han (Seoul National University)
Explainability and InterpretabilityComputational EfficiencyTransformerVision Language ModelVideoText
🎯 What it does: Provide a mechanistic interpretation of the internal information flow in video large language models (VideoLLMs) for video question answering (VideoQA) tasks, revealing the phased characteristics of temporal reasoning.
Mapping Overlaps in Benchmarks through Perplexity in the Wild
Siyang Wu (University of Chicago), James Evans (University of Chicago)
Explainability and InterpretabilityLarge Language ModelTextBenchmark
🎯 What it does: Constructed a benchmark signature based on model perplexity to quantify the overlap degree between different LLM benchmarks;
Mapping Post-Training Forgetting in Language Models at Scale
Jackson Harmon (University of Tübingen), Ameya Prabhu (University of Tübingen)
Explainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextBenchmark
🎯 What it does: This paper investigates knowledge forgetting and backward transfer in language models across different post-training stages (domain-continual pre-training, instruction tuning, reasoning SFT/RL, model merging), conducting large-scale evaluations on nearly 30 model pairs and over 100 sub-benchmarks.
Mapping Semantic & Syntactic Relationships with Geometric Rotation
Michael Freenor (TELUS Digital), Lauren Alvarez (TELUS Digital)
Domain AdaptationRepresentation LearningText
🎯 What it does: Developed the RISE method based on rotational alignment, modeling sentence-level semantic-syntactic variations (negation, conditional clauses, politeness) as consistent rotations on a hypersphere, supporting cross-lingual and cross-model inference.
MAPSS: Manifold-based Assessment of Perceptual Source Separation
Amir Ivry (Technion Israel Institute of Technology), Shinji Watanabe (Carnegie Mellon University)
TransformerDiffusion modelContrastive LearningBenchmarkAudio
🎯 What it does: Proposed two novel evaluation metrics for audio source separation—Perceptual Separation (PS) and Perceptual Matching (PM)—by constructing a diffusion graph on self-supervised audio embeddings to separate and quantify two distortion patterns: leakage and self-distortion.
MARC: Memory-Augmented RL Token Compression for Efficient Video Understanding
Peiran Wu (University of Bristol), Junxiao Shen (Memories.ai Research)
CompressionTransformerReinforcement LearningContrastive LearningVideo
🎯 What it does: Proposes MARC, a token compression framework that combines visual memory retrieval with reinforcement learning distillation, capable of compressing videos to a single visual token while maintaining original inference performance.
Market Games for Generative Models: Equilibria, Welfare, and Strategic Entry
Xiukun Wei (Ohio State University), Xueru Zhang (Ohio State University)
GenerationOptimizationConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: This paper models the generative model market as a three-layer model-platform-user game, analyzing pure Nash equilibrium, user welfare, and diversity, and proposes entry strategies from the perspective of model providers.
Markovian Transformers for Informative Language Modeling
Scott W Viteri, Clark Barrett (Stanford University)
GenerationReinforcement Learning from Human FeedbackTransformerReinforcement LearningAuto EncoderTextChain-of-Thought
🎯 What it does: Designed and trained a language model under Markovian constraints, enabling it to generate answers based solely on the Chain-of-Thought (CoT) after generating the CoT, thereby making CoT a necessary pathway for answer inference.
MARL2Grid-TR: A Multi-Agent RL Benchmark in Power Grid Operations
Enrico Marchesini (Massachusetts Institute of Technology), Priya L. Donti (Massachusetts Institute of Technology)
OptimizationReinforcement LearningGraphTime SeriesBenchmark
🎯 What it does: Propose MARL2GRID-TR, the first multi-agent reinforcement learning benchmark for grid topology optimization and rescheduling;
MARS - A Foundational Map Auto-Regressor
Qi Zhang, Fuxun Yu (Microsoft)
GenerationTransformerImage
🎯 What it does: Proposes a Map Auto-Regressor (MARS) based on autoregressive Transformer, capable of end-to-end generating vectorized map elements such as roads and buildings, and supports human-machine interactive correction.
MarS-FM: Generative Modeling of Molecular Dynamics via Markov State Models
Kacper Kapuśniak (University of Oxford), Francesco Di Giovanni (Valence Labs)
Drug DiscoveryFlow-based ModelBiomedical DataPhysics Related
🎯 What it does: Proposed the MSM Emulators class of models, learning cross-state transitions through Markov state models and implementing the MARS-FM framework to generate molecular dynamics trajectories.
MARS-Sep: Multimodal-Aligned Reinforced Sound Separation
Zihan Zhang (Zhejiang University), Tao Jin (Zhejiang University)
RestorationReinforcement LearningVision-Language-Action ModelContrastive LearningMultimodality
🎯 What it does: Propose the MARS-Sep model by improving multimodal alignment audio separation through a reinforcement learning framework;
MaRS: Memory-Adaptive Routing for Reliable Capacity Expansion and Knowledge Retention
Gang Yan (Jilin University)
Knowledge DistillationRepresentation LearningTransformerContrastive LearningImageText
🎯 What it does: Proposes the MARS framework, using a frozen large pre-trained model as a stable feature encoder, and achieving continual learning through an expandable slot-based memory router and a lightweight classifier.
MARSHAL: Incentivizing Multi-Agent Reasoning via Self-Play with Strategic LLMs
Huining Yuan (Tsinghua University), Yu Wang (Tsinghua University)
Large Language ModelReinforcement LearningText
🎯 What it does: This paper proposes the MARSHAL framework, which leverages large language models (LLMs) to train multi-agent reasoning and strategy capabilities in multi-agent systems (MAS) through self-play.
MARTI: A Framework for Multi-Agent LLM Systems Reinforced Training and Inference
Kaiyan Zhang (Tsinghua University), Bowen Zhou (Tsinghua University)
OptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningAgentic AIText
🎯 What it does: Proposed and open-sourced the MARTI framework, achieving unified training of reinforcement learning and reasoning in multi-agent LLM systems, supporting centralized multi-agent interaction, distributed policy training, and asynchronous multi-round generation.
MAS$^2$: Self-Generative, Self-Configuring, Self-Rectifying Multi-Agent Systems
Kun Wang (Nanyang Technological University), Yufei Guo (Peking University)
OptimizationLarge Language ModelReinforcement LearningAgentic AITextBenchmark
🎯 What it does: Propose MAS 2, which utilizes multi-agent generation, implementation, and correction of meta-agents for recursive self-construction of multi-agent systems;
MASAM: Multimodal Adaptive Sharpness-Aware Minimization for Heterogeneous Data Fusion
Zijie Chen (Hong Kong Baptist University), Jing Qin (Lingnan University)
OptimizationMultimodalityBiomedical DataAlzheimer's Disease
🎯 What it does: Propose the MASAM framework to address the modal imbalance problem in multi-modal learning.
MaskCO: Masked Generation Drives Effective Representation Learning and Exploiting for Combinatorial Optimization
Lvda Chen (Shanghai Jiao Tong University), Junchi Yan (Shanghai Jiao Tong University)
OptimizationRepresentation LearningTransformerSupervised Fine-TuningReinforcement LearningGraphBenchmark
🎯 What it does: Proposes MaskCO, a self-supervised learning framework based on mask generation for neural combinatorial optimization;
Masked Generative Policy for Robotic Control
Lipeng Zhuang (University of Glasgow), Paul Henderson (University of Glasgow)
Robotic IntelligenceTransformerAuto EncoderSequential
🎯 What it does: Propose the Masked Generative Policy (MGP) framework, which uses a parallel masked generation Transformer to generate and dynamically refine action sequences, achieving low-latency vision-motor imitation learning.
Masked Skill Token Training for Hierarchical Off-Dynamics Transfer
Zeyu Feng (Agency for Science, Technology and Research (A*STAR)), Harold Soh (National University of Singapore)
Reinforcement LearningDiffusion modelAuto EncoderSequential
🎯 What it does: Proposes Masked Skill Token Training (MSTT), a fully offline hierarchical reinforcement learning framework capable of achieving dynamic zero-shot transfer in target environments using only observed demonstration data.
MaskInversion: Localized Embeddings via Optimization of Explainability Maps
Walid Bousselham (Tuebingen AI Center), Hilde Kuehne (Tuebingen AI Center University of Tuebingen)
RetrievalExplainability and InterpretabilityRepresentation LearningImageMultimodality
🎯 What it does: Propose the MaskInversion method, which generates local embeddings for a given mask by optimizing the interpretability maps of a frozen base model without fine-tuning the model.
MaskPro: Linear-Space Probabilistic Learning for Strict (N:M)-Sparsity on LLMs
Yan Sun (University of Sydney), Dacheng Tao (Nanyang Technological University)
Computational EfficiencyTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Propose MaskPro, a linear-space probabilistic learning framework for large language models (LLMs) to achieve strict (N:M) sparsity;
MASS: MoErging through Adaptive Subspace Selection
Donato Crisostomi (Sapienza University of Rome), Emanuele Rodolà (Sapienza University of Rome)
Computational EfficiencyTransformerMixture of ExpertsImageText
🎯 What it does: Designed a lightweight model merging method called MASS, which utilizes low-rank singular vectors from task updates and employs a projection router without training data to dynamically select subspaces and classification heads during inference.
Massive Activations are the Key to Local Detail Synthesis in Diffusion Transformers
Chaofan Gan (Shanghai Jiao Tong University), Weiyao Lin (Shanghai Jiao Tong University)
GenerationTransformerDiffusion modelImage
🎯 What it does: Systematically studied massive activations (MA) in diffusion Transformers, finding that they mainly drive local detail synthesis, and proposed a training-free self-directed strategy called Detail Guidance (DG) to significantly enhance the detail quality of generated images.
Massive Editing for Large Language Models Based on Dynamic Weight Generation
Wentao Wan (Sun Yat-sen University), Keze Wang (Sun Yat-sen University)
TransformerDiffusion modelScore-based ModelContrastive LearningTextBenchmark
🎯 What it does: Proposed a dynamic weight generation-based LLM knowledge editing method called MeG, which utilizes diffusion models to add a single dynamic neuron in specific layers for large-scale knowledge editing;
Massive Memorization with Hundreds of Trillions of Parameters for Sequential Transducer Generative Recommenders
Zhimin Chen (Meta), Wen-Yun Yang (Meta)
Recommendation SystemTransformerSequential
🎯 What it does: Proposed the VISTA two-stage framework, which first compresses users' extremely long interaction history into a small number of cached embeddings, and then applies target-aware attention to candidate items, significantly reducing online inference computational load while maintaining prediction performance.
Master Skill Learning with Policy-Grounded Synergy of LLM-based Reward Shaping and Exploring
Yanbin Chang (Shenzhen University), Jianqiang Li (Shenzhen University)
Robotic IntelligenceLarge Language ModelReinforcement Learning
🎯 What it does: Proposes the PoRSE framework, which utilizes large language models (LLM) to automatically generate task-specific reward functions and abstract functional spaces (AFS), and dynamically balances reward shaping and state exploration within the internal policy improvement (IPG) process to efficiently learn robotic skills;
Mastering Sparse CUDA Generation through Pretrained Models and Deep Reinforcement Learning
Yaoyu Wang (Chinese Academy of Sciences), Guangming Tan (Chinese Academy of Sciences)
Computational EfficiencyAI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextGraph
🎯 What it does: Proposes the SparseRL framework, leveraging pre-trained language models and deep reinforcement learning to generate high-performance CUDA code for sparse matrix operations.
MATA: A Trainable Hierarchical Automaton System for Multi-Agent Visual Reasoning
Zhixi Cai (Monash University), Hamid Rezatofighi (Monash University)
Explainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningAgentic AIVision Language ModelImageTextMultimodality
🎯 What it does: Propose the MATA system, which divides visual reasoning into a hierarchical, trainable finite state machine, using multi-agent collaboration and competition to achieve interpretable and efficient reasoning processes.
Matched Data, Better Models: Target Aligned Data Filtering with Sparse Autoencoders
Arnav Mohanty Das (University of Washington), Jeff Bilmes (University of Washington)
OptimizationRepresentation LearningData-Centric LearningTransformerVision Language ModelAuto EncoderContrastive LearningMultimodality
🎯 What it does: Propose a submodular function distribution matching (SDM) framework based on sparse autoencoders for filtering and selecting large-scale image-text data;
Matching Multiple Experts: On the Exploitability of Multi-Agent Imitation Learning
Antoine Bergerault (EPFL), Negar Mehr (UC Berkeley)
Reinforcement Learning
🎯 What it does: Studies offline imitation learning in multi-agent environments, analyzing how to learn Nash equilibrium strategies from expert demonstrations;
Matching without Group Barrier for Heterogeneous Treatment Effect Estimation
Yuguang Yan (Guangdong University of Technology), Zhifeng Hao (Shantou University)
TabularBiomedical Data
🎯 What it does: Proposed a group boundary-free matching (MOGA) method that uses all samples rather than only the target group to find nearest neighbors, thereby improving the estimation of heterogeneous treatment effects;
Math Blind: Failures in Diagram Understanding Undermine Reasoning in MLLMs
Yanpeng Sun (Singapore University of Technology and Design), Anton van den Hengel (University of Adelaide)
ClassificationRecognitionLarge Language ModelSupervised Fine-TuningPrompt EngineeringVision Language ModelImageGraphBenchmark
🎯 What it does: This study proposes a benchmark called MATHEMETRIC specifically for evaluating the mathematical diagram perception capabilities of multimodal large language models (MLLMs), and trains models on the structured graphical dataset GEOMETRIC to enhance low-level perception and high-level reasoning of diagrams.