ICML 2025 Papers — Page 18
International Conference on Machine Learning · 3257 papers
Lightweight Protocols for Distributed Private Quantile Estimation
Anders Aamand (University of Copenhagen), Rasmus Pagh (University of Copenhagen)
Safty and Privacy
🎯 What it does: Two adaptive algorithms are proposed for estimating the quantiles (especially the median) of distributed datasets under the Local Differential Privacy (LDP) and Shuffle Differential Privacy (shuffleDP) models, with sample complexities of O(log B/(ε²α²)) and O((1/α²+1/ε²)·log B), respectively.
Lightweight-Mark: Rethinking Deep Learning-Based Watermarking
Yupeng Qiu (National University of Singapore), Ee-Chien Chang (National University of Singapore)
RecognitionCompressionKnowledge DistillationConvolutional Neural NetworkImage
🎯 What it does: A lightweight deep learning watermarking model is proposed, which employs five fine-grained modules and uses a separable projection head with a decoding-guided alternative loss during the training phase, significantly reducing parameters while enhancing robustness and invisibility.
LIMEFLDL: A Local Interpretable Model-Agnostic Explanations Approach for Label Distribution Learning
Xiuyi Jia (Nanjing University of Science and Technology), Weiwei Li (Nanjing University of Aeronautics and Astronautics)
Explainability and InterpretabilityImageTabular
🎯 What it does: A local interpretable model-agnostic explanation method LIMEFLDL is proposed for label distribution learning (LDL), which can provide explanations for the complete label distribution of samples without altering the black-box model.
Limitations of measure-first protocols in quantum machine learning
Casper Gyurik (Pasqal), Vedran Dunjko (Leiden University)
Physics Related
🎯 What it does: This paper compares two quantum machine learning protocols: fully quantum processing and measurement-first learning. It proves that for specific learning tasks, the measurement-first learning protocol requires an exponential increase in sample complexity, while the fully quantum protocol only requires polynomial-level samples to complete the task.
Linear $Q$-Learning Does Not Diverge in $L^2$: Convergence Rates to a Bounded Set
Xinyu Liu (University of Virginia), Shangtong Zhang (University of Virginia)
Reinforcement Learning
🎯 What it does: This paper presents the L₂ convergence rate of linear and tabular Q-learning under the ε-softmax behavior policy without making any algorithmic changes, without the Bellman completeness assumption, or the near-optimality assumption of behavior policies, and proves that linear Q-learning does not diverge under this setting.
Linear Bandits with Partially Observable Features
Wonyoung Kim (Chung Ang University), Min-hwan Oh (Seoul National University)
OptimizationReinforcement LearningTabular
🎯 What it does: The study addresses the linear multi-armed bandit problem under the condition of observable features, proposing an algorithm named RoLF that combines feature augmentation and double robust (DR) estimation, achieving sublinear regret in the presence of unknown latent features.
Linear Contextual Bandits With Interference
Yang Xu (North Carolina State University), Rui Song (North Carolina State University)
Recommendation SystemReinforcement LearningTabular
🎯 What it does: This paper proposes interference modeling and online learning algorithms within the framework of linear contextual multi-armed bandits (LinCB) in the presence of interference, and provides theoretical analysis.
Linear convergence of Sinkhorn's algorithm for generalized static Schrödinger bridge
Rahul Choudhary (University of Wisconsin Madison), Hanbaek Lyu (University of Wisconsin Madison)
OptimizationTabular
🎯 What it does: This paper studies and proposes a general static Schrödinger bridge problem (SSB), allowing the use of any strictly increasing divergence function f, and provides its Kantorovich dual form; it subsequently proves that the general Sinkhorn algorithm (GSA) based on this dual form converges linearly under mild assumptions.
Linear Mode Connectivity between Multiple Models modulo Permutation Symmetries
Akira Ito (NTT Social Informatics Laboratories), Atsutoshi Kumagai (NTT Computer and Data Science Laboratories)
OptimizationFederated LearningConvolutional Neural NetworkImage
🎯 What it does: This study investigates the linear mode connectivity (LMC) of neural networks under permutation symmetry and proposes a straight-through estimator for multi-models (STE-MM) to find parameter arrangements that can aggregate multiple independently trained models into the same low-loss convex basin; experiments are then conducted on various model architectures.
Linear Transformers as VAR Models: Aligning Autoregressive Attention Mechanisms with Autoregressive Forecasting
Jiecheng Lu (Georgia Institute of Technology), Shihao Yang (Georgia Institute of Technology)
TransformerTime Series
🎯 What it does: This paper interprets single-layer linear attention as a dynamic VAR structure and points out the structural mismatch between multi-layer Transformers and autoregressive prediction objectives, subsequently proposing the SAMoVAR model to align the attention mechanism with the VAR model.
Linearization Turns Neural Operators into Function-Valued Gaussian Processes
Emilia Magnani (Tübingen AI Center), Philipp Hennig (Tübingen AI Center)
Gaussian SplattingTime SeriesPhysics Related
🎯 What it does: This paper proposes a model linearization-based neural operator uncertainty quantification framework (LUNO), which maps the trained neural operator to a function value Gaussian process to obtain analytical predictive uncertainty.
LineFlow: A Framework to Learn Active Control of Production Lines
Kai Müller, Tobias Windisch (University of Applied Sciences Kempten)
OptimizationReinforcement Learning
🎯 What it does: This paper presents the open-source LineFlow framework for high-precision simulation of production lines and training reinforcement learning (RL) agents for active control.
LipsNet++: Unifying Filter and Controller into a Policy Network
Xujie Song (Tsinghua University), Shengbo Eben Li (Tsinghua University)
Reinforcement Learning
🎯 What it does: Introducing Fourier filtering layers and Lipschitz control layers in deep reinforcement learning to jointly address the issues of action fluctuations caused by observation noise and policy non-smoothness.
LIVS: A Pluralistic Alignment Dataset for Inclusive Public Spaces
Rashid Mushkani (University of Montreal), Hadrien Bertrand (University of Montreal)
GenerationOptimizationTransformerLarge Language ModelSupervised Fine-TuningDiffusion modelImage
🎯 What it does: Based on community participation, a dataset called Local Intersectional Visual Spaces (LIVS) was constructed for multi-standard alignment of urban public spaces, and this dataset was used to conduct multi-standard alignment experiments on Stable Diffusion XL.
LLaVA-ReID: Selective Multi-image Questioner for Interactive Person Re-Identification
Yiding Lu (Sichuan University), Xi Peng (Sichuan University)
RecognitionRetrievalTransformerLarge Language ModelImageTextMultimodalityRetrieval-Augmented Generation
🎯 What it does: An interactive person re-identification (Inter-ReID) framework is proposed, which continuously refines descriptions through multi-turn dialogue with witnesses to achieve more accurate person retrieval.
LlavaGuard: An Open VLM-based Framework for Safeguarding Vision Datasets and Models
Lukas Helff (TU Darmstadt), Patrick Schramowski (TU Darmstadt)
Safty and PrivacyExplainability and InterpretabilityTransformerSupervised Fine-TuningVision Language ModelImageMultimodality
🎯 What it does: LlavaGuard is proposed, a visual content safety assessment framework based on visual language models (VLM), capable of providing safety ratings, categories, and reasons for images, and supporting multi-strategy adaptation.
LLM Alignment as Retriever Optimization: An Information Retrieval Perspective
Bowen Jin (University of Illinois), Sercan O Arik
RetrievalOptimizationTransformerLarge Language ModelContrastive LearningText
🎯 What it does: This paper views the alignment problem of large language models (LLM) as a retrieval optimization task in information retrieval and proposes the LARPO framework.
LLM Data Selection and Utilization via Dynamic Bi-level Optimization
Yang Yu (University of Chinese Academy of Sciences), Dacheng Tao (Nanyang Technological University)
OptimizationData-Centric LearningTransformerLarge Language ModelText
🎯 What it does: A dynamic dual-layer optimization framework and data weighting model (DWM) is proposed to enhance the efficiency of utilizing selected data during the pre-training of large language models.
LLM Enhancers for GNNs: An Analysis from the Perspective of Causal Mechanism Identification
Hang Gao (Institute of Software Chinese Academy of Sciences), Huaping Liu (Tsinghua University)
Graph Neural NetworkLarge Language ModelPrompt EngineeringGraph
🎯 What it does: This paper conducts a causal mechanism analysis of a general framework that uses large language models (LLMs) as feature enhancers before inputting them into graph neural networks (GNNs), and based on this, proposes the AT (Attention-based Transmission) module to optimize the information transfer between LLMs and GNNs.
LLM-Assisted Semantically Diverse Teammate Generation for Efficient Multi-agent Coordination
Lihe Li (Nanjing University), Yang Yu (Nanjing University)
Robotic IntelligenceTransformerLarge Language ModelReinforcement LearningAgentic AIText
🎯 What it does: Utilizing large language models (LLM) to generate descriptive and diverse collaborative behaviors, automating the construction of corresponding reward functions, thereby training semantically diverse teammate strategies, and using a multi-head network to continuously train a main control agent capable of matching different teammates.
LLM-Augmented Chemical Synthesis and Design Decision Programs
Haorui Wang (Georgia Tech), Chao Zhang (Georgia Tech)
OptimizationDrug DiscoveryTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: This paper proposes the use of large language models (LLMs) to directly generate complete multi-step synthetic routes and optimize these routes through evolutionary search (LLM-Syn-Planner), thereby achieving chemical synthesis planning and the design of synthesizable molecules.
LLM-SRBench: A New Benchmark for Scientific Equation Discovery with Large Language Models
Parshin Shojaee (Virginia Tech), Chandan K. Reddy
TransformerLarge Language ModelTextBenchmarkPhysics Related
🎯 What it does: A benchmark called LLM-SRBench has been established to evaluate the performance of large language models in the task of scientific equation discovery.
LLMs Can Reason Faster Only If We Let Them
Bilgehan Sel (Virginia Tech), Ming Jin (Virginia Tech)
OptimizationTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
🎯 What it does: This paper presents AoT-O3, an algorithm that combines supervised fine-tuning and reinforcement learning, aiming to significantly reduce the step length generated by AoT while maintaining or improving the planning accuracy of LLMs.
LLMs can see and hear without any training
Kumar Ashutosh (University of Texas at Austin), Rohit Girdhar (Meta AI)
GenerationTransformerLarge Language ModelVision Language ModelImageVideoMultimodalityAudio
🎯 What it does: A training-free multimodal iterative LLM solver named MILS is proposed, which utilizes LLM to generate candidate answers and feeds them back to LLM after evaluation by a multimodal model, iterating in a loop until convergence, completing multimodal understanding and generation tasks.
LLMs on the Line: Data Determines Loss-to-Loss Scaling Laws
Prasanna Mayilvahanan (Max Planck Institute for Intelligent Systems), Wieland Brendel (Max Planck Institute for Intelligent Systems)
TransformerLarge Language ModelText
🎯 What it does: Systematically study the loss-to-loss scaling law of LLMs, revealing that pre-training data determines the scaling trend.
LLMScan: Causal Scan for LLM Misbehavior Detection
Mengdi Zhang (Singapore Management University), Hongyu Zhang (Chongqing University)
Anomaly DetectionTransformerLarge Language ModelText
🎯 What it does: This paper proposes LLMSCAN, a causal inference-based internal monitoring method for LLMs, designed to detect model misbehaviors (lies, toxicity, jailbreaks, backdoor attacks) during response generation.
LMAct: A Benchmark for In-Context Imitation Learning with Long Multimodal Demonstrations
Anian Ruoss (Google DeepMind), Tim Genewein (Google DeepMind)
Robotic IntelligenceReinforcement Learning from Human FeedbackTransformerLarge Language ModelMultimodalityBenchmarkChain-of-Thought
🎯 What it does: This paper proposes and implements the LMAct benchmark, which evaluates the on-the-fly imitation learning capabilities of large language models in long contexts and multimodal demonstrations, covering interactive decision-making tasks such as board games, graphical games, and puzzles.
LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Models
Marwa Abdulhai (University of California), Sergey Levine (University of California)
TransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextBenchmark
🎯 What it does: This paper proposes the LMRL-Gym evaluation benchmark, which includes 8 multi-turn RL tasks (3 interactive dialogues and 5 text games), and provides an open-source framework for researchers to quickly conduct multi-turn reinforcement learning experiments with LLMs.
LOB-Bench: Benchmarking Generative AI for Finance - an Application to Limit Order Book Data
Peer Nagy (University of Oxford), Jakob Nicolaus Foerster
GenerationData SynthesisRecommendation SystemAnomaly DetectionOptimizationExplainability and InterpretabilityComputational EfficiencyGenerative Adversarial NetworkTabularTime SeriesBenchmarkFinance Related
🎯 What it does: Proposes the LOB-Bench benchmark framework to evaluate the authenticity and quality of generative AI on limit order book (LOBSTER format) data.
Local Identifying Causal Relations in the Presence of Latent Variables
Zheng Li (Beijing Technology and Business University), Zhi Geng (Beijing Technology and Business University)
OptimizationExplainability and InterpretabilityGraphBenchmark
🎯 What it does: A local method (LocICR) is proposed to locally identify causal relationships between any two variables using only observational data in the presence of latent variables (including non-ancestors, explicit ancestors, implicit ancestors, and possible ancestors).
Local Manifold Approximation and Projection for Manifold-Aware Diffusion Planning
Kyowoon Lee (Korea Advanced Institute of Science and Technology), Jaesik Choi (Korea Advanced Institute of Science and Technology)
Robotic IntelligenceReinforcement LearningDiffusion modelTabular
🎯 What it does: The LoMAP (Local Manifold Approximation and Projection) method is proposed to address the manifold deviation problem caused by guidance errors in diffusion planning.
Local Pan-privacy for Federated Analytics
Vitaly Feldman (Apple), Kunal Talwar (Apple)
Federated LearningSafty and PrivacyReinforcement Learning
🎯 What it does: The concept of local Pan-Privacy is proposed, and the privacy protection against continuous invasions of local devices is studied in the context of federated analysis; feasible algorithms are provided for basic statistical tasks (counting, histograms, averages), and it is proven that the error under the information-theoretic model cannot match that of ordinary local differential privacy.
Locality Preserving Markovian Transition for Instance Retrieval
Jifei Luo (University of Science and Technology of China), Changsheng Xu (Institute of Automation, Chinese Academy of Sciences)
RetrievalGraph Neural NetworkImageBenchmarkStochastic Differential Equation
🎯 What it does: This paper proposes a new instance retrieval re-ranking framework called LPMT, which better captures the data manifold structure and improves retrieval performance by mapping each instance as a distribution and performing thermodynamic Markov transitions in the distribution space.
LOCATE 3D: Real-World Object Localization via Self-Supervised Learning in 3D
Paul McVay (Meta), Franziska Meier (Meta)
Object DetectionRepresentation LearningRobotic IntelligenceTransformerVision Language ModelContrastive LearningPoint Cloud
🎯 What it does: Proposes the LOCATE 3D model, which utilizes 3D-JEPA self-supervised learning to achieve contextual representation of 3D point clouds, and combines a language-conditioned 3D decoder to locate 3D objects from natural language instructions.
Locate-then-edit for Multi-hop Factual Recall under Knowledge Editing
Zhuoran Zhang (Peking University), Di Wang
TransformerPrompt EngineeringText
🎯 What it does: This study investigates knowledge editing under multi-hop factual recall tasks and proposes the IFMET method to improve the locate-then-edit paradigm.
Log-Sum-Exponential Estimator for Off-Policy Evaluation and Learning
Armin Behnamnia (Sharif University of Technology), Hamid R. Rabiee (Sharif University of Technology)
Reinforcement LearningTabular
🎯 What it does: This paper proposes a nonlinear estimator based on the logarithm-sum-exponential (LSE) operator for off-policy evaluation (OPE) and off-policy learning (OPL) in offline reinforcement learning, aimed at reducing high variance and being robust to noise and heavy-tailed rewards.
Logarithmic Regret for Online KL-Regularized Reinforcement Learning
Heyang Zhao (University of California), Tong Zhang (University of Illinois Urbana-Champaign)
OptimizationReinforcement Learning
🎯 What it does: This paper proposes an online contextual game and MDP algorithm with KL regularization, and provides a logarithmic regret upper bound.
Logits are All We Need to Adapt Closed Models
Gaurush Hiranandani (Typeface), Sanmi Koyejo
GenerationTransformerLarge Language ModelText
🎯 What it does: A Plugin model based on token probability reweighting is proposed, utilizing the logit output of a closed-source LLM and a small amount of task data to achieve adaptive generation.
LOGO --- Long cOntext aliGnment via efficient preference Optimization
Zecheng Tang (Soochow University), Min Zhang (Soochow University)
GenerationOptimizationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Proposes the LOGO training strategy to achieve long context alignment, employing no-reference preference optimization and position index synthesis methods, significantly improving the quality of long text generation.
Long-Form Speech Generation with Spoken Language Models
Se Jin Park (Google DeepMind), RJ Skerry-Ryan (Google DeepMind)
GenerationBenchmarkAudio
🎯 What it does: This paper presents SpeechSSM—a text-independent speech language model based on state space models, capable of generating several minutes of continuous speech in one go.
Long-Short Alignment for Effective Long-Context Modeling in LLMs
Tianqi Du (Peking University), Yisen Wang (Peking University)
TransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: This study addresses the length generalization problem of large language models in long context modeling, proposing the concept of Long-Short Alignment and introducing the Long-Short Misalignment metric and its regularization method.
Long-Term TalkingFace Generation via Motion-Prior Conditional Diffusion Model
Fei Shen (Nanjing University of Science and Technology), Tat-Seng Chua (National University of Singapore)
GenerationData SynthesisTransformerDiffusion modelImageVideoMultimodalityAudio
🎯 What it does: Achieved the generation of long-term realistic and coherent speaker facial videos from a single face image and audio.
LongRoPE2: Near-Lossless LLM Context Window Scaling
Ning Shang (Microsoft), Mao Yang (Microsoft)
TransformerLarge Language ModelText
🎯 What it does: Expand the context window of the pre-trained LLM to 128k while maintaining the original short window performance without loss.
LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understanding
Xiaoqian Shen (King Abdullah University of Science and Technology), Vikas Chandra (Meta AI)
RecognitionCompressionTransformerLarge Language ModelVision Language ModelVideoTextMultimodality
🎯 What it does: The LongVU model is proposed, which can efficiently understand long videos within the limited context length of LLMs through spatiotemporal adaptive compression technology.
Look Twice Before You Answer: Memory-Space Visual Retracing for Hallucination Mitigation in Multimodal Large Language Models
Xin Zou (Hong Kong University of Science and Technology), Xuming Hu (Hong Kong University of Science and Technology)
GenerationRetrievalOptimizationComputational EfficiencyTransformerLarge Language ModelVision Language ModelTextMultimodalityRetrieval-Augmented Generation
🎯 What it does: This paper proposes MemVR, a decoding strategy based on visual memory re-inspection that dynamically triggers visual re-inspection during the generation process of multimodal large language models, enhancing factual consistency and reducing hallucinations.
Looking Beyond the Top-1: Transformers Determine Top Tokens in Order
Daria Lioubashevski (Hebrew University of Jerusalem), Ariel Goldstein (Hebrew University of Jerusalem)
TransformerImageTextAudio
🎯 What it does: Analyzing the computations of the Transformer at each layer after saturation events, it is found that the top-k tokens are determined layer by layer in order of their ranking;
LoRA Training Provably Converges to a Low-Rank Global Minimum Or It Fails Loudly (But it Probably Won't Fail)
Junsu Kim (Seoul National University), Ernest K. Ryu (University of California, Los Angeles)
OptimizationTransformerSupervised Fine-TuningImageText
🎯 What it does: This paper provides a theoretical analysis of the training dynamics of LoRA (Low-Rank Adaptation) fine-tuning, proving that under the conditions of restricted strong convexity and restricted smoothness, the second-order stationary points of LoRA are either low-rank, small-norm global optimal solutions or high-rank, large-norm erroneous solutions (i.e., 'fail loudly').
LoRA-Gen: Specializing Large Language Model via Online LoRA Generation
Yicheng Xiao (Tsinghua University), Ying Shan (Tencent PCG)
GenerationCompressionTransformerLarge Language ModelMixture of ExpertsText
🎯 What it does: Online generation of LoRA parameters through large cloud models and their reparameterization into edge models achieves task specialization and context compression without training.
LoRA-One: One-Step Full Gradient Could Suffice for Fine-Tuning Large Language Models, Provably and Efficiently
Yuanhe Zhang (University of Warwick), Yudong Chen (University of Wisconsin-Madison)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proves through theoretical analysis and experimental validation that the gradient updates of LoRA align with the feature subspace of a full gradient, and proposes the Spectral-init initialization and LoRA-One algorithm to achieve this alignment, thereby accelerating and enhancing the fine-tuning effect of LoRA.
Loss Functions and Operators Generated by f-Divergences
Vincent Roulet (Google DeepMind), Mathieu Blondel (Google DeepMind)
ClassificationOptimizationKnowledge DistillationTransformerSupervised Fine-TuningImageText
🎯 What it does: Proposes a convex Fenchel-Young loss based on f-divergence and the corresponding f-softargmax/softmax operations, applying them to classification and language model training.
LotteryCodec: Searching the Implicit Representation in a Random Network for Low-Complexity Image Compression
Haotian Wu (Imperial College London), Deniz Gunduz (Imperial College London)
CompressionAuto EncoderImage
🎯 What it does: This paper proposes LotteryCodec, which utilizes randomly initialized subnetworks in a network for single image compression and validates the lottery ticket hypothesis.
Low-Dimension-to-High-Dimension Generalization and Its Implications for Length Generalization
Yang Chen (Peking University), Zhouchen Lin (Peking University)
TransformerChain-of-Thought
🎯 What it does: A low-dimensional to high-dimensional generalization (LDHD) framework is proposed, explaining the essence of length generalization and exploring the model's inductive bias both theoretically and experimentally.
Low-distortion and GPU-compatible Tree Embeddings in Hyperbolic Space
Max van Spengler (University of Amsterdam), Pascal Mettes (University of Amsterdam)
OptimizationGraphBenchmark
🎯 What it does: A tree embedding method based on hyperspherical distribution optimization (HS-DTE) and a GPU-friendly floating-point extended arithmetic framework (HypFPE) are proposed to achieve low distortion and high precision in hyperbolic space tree embedding.
Low-Rank Adapting Models for Sparse Autoencoders
Matthew Chen (Massachusetts Institute of Technology), Max Tegmark (Massachusetts Institute of Technology)
OptimizationExplainability and InterpretabilityComputational EfficiencySupervised Fine-TuningAuto EncoderTextBenchmark
🎯 What it does: Using Low-Rank Adaptation (LoRA) to fine-tune language models on existing Sparse Autoencoders (SAE), significantly reducing the cross-entropy loss gap caused by the insertion of SAE and enhancing model interpretability.
Low-Rank Tensor Transitions (LoRT) for Transferable Tensor Regression
Andong Wang (RIKEN AIP), Qibin Zhao (RIKEN AIP)
RestorationDomain AdaptationComputational EfficiencyVideo
🎯 What it does: This paper proposes the Low-Rank Tensor Transitions (LoRT) framework, aimed at addressing the tensor regression problem in scenarios of insufficient samples, model/covariate shift, and decentralized data environments; a distributed version, D-LoRT, is also provided.
Low-Rank Thinning
Annabelle Michael Carrell (University of Cambridge), Lester Mackey (Microsoft Research)
OptimizationComputational EfficiencyTransformerGenerative Adversarial NetworkImageTabular
🎯 What it does: A new low-rank analysis framework is proposed to evaluate and improve sub-Gaussian thinning algorithms, thereby achieving efficient approximations for three types of machine learning tasks: dot-product attention in Transformers, gradient reordering in SGD training, and deep kernel two-sample testing.
Lower Bounds for Chain-of-Thought Reasoning in Hard-Attention Transformers
Alireza Amiri Bavandpour, Michael Hahn (Saarland University)
TransformerChain-of-Thought
🎯 What it does: This paper studies the minimum length required for chain-of-thought (CoT) in the Unconstrained Hard Attention Transformer (UHAT) and provides an unconditional linear lower bound for multi-class algorithm problems such as PARITY, multiplication, MEDIAN, and graph reachability.
LowRA: Accurate and Efficient LoRA Fine-Tuning of LLMs under 2 Bits
Zikai Zhou (Stanford University), Kunle Olukotun (Stanford University)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Proposes the LowRA framework, enabling LoRA fine-tuning to be achieved at extremely low precision with each parameter below 2 bits;
LRA-QViT: Integrating Low-Rank Approximation and Quantization for Robust and Efficient Vision Transformers
Beom Jin Kang (Seoul National University of Science and Technology), Hyun Kim (Seoul National University of Science and Technology)
Object DetectionSegmentationCompressionKnowledge DistillationTransformerImageTextMultimodalityAudio
🎯 What it does: This paper proposes a compression framework that combines low-rank approximation (RB-LRA) and quantization (WADS), aiming to significantly reduce the model size and computational requirements of Vision Transformers while maintaining accuracy.
LSCD: Lomb--Scargle Conditioned Diffusion for Time series Imputation
Elizabeth Fons (J.P. Morgan AI Research), Manuela Veloso (J.P. Morgan AI Research)
Diffusion modelScore-based ModelTime Series
🎯 What it does: This study investigates a condition diffusion model based on a differentiable Lomb-Scargle periodogram for filling in missing values in time series without interpolation.
LV-XAttn: Distributed Cross-Attention for Long Visual Inputs in Multimodal Large Language Models
Tzu-Tao Chang (University of Wisconsin-Madison), Shivaram Venkataraman (University of Wisconsin-Madison)
Computational EfficiencyTransformerLarge Language ModelVision Language ModelVideoMultimodality
🎯 What it does: A distributed and precise cross-attention mechanism (LV-XAttn) and activation recomputation technique are proposed for efficiently processing long visual inputs in multimodal large language models.
M+: Extending MemoryLLM with Scalable Long-Term Memory
Yu Wang (University of California San Diego), Zexue He (IBM Research)
TransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: Based on MemoryLLM, a scalable long-term memory mechanism and a jointly trained retriever have been added to enhance the LLM's ability to maintain long texts and long-term knowledge.
M2PDE: Compositional Generative Multiphysics and Multi-component PDE Simulation
Tao Zhang (Nuclear Power Institute of China), Tailin Wu (Westlake University)
GenerationData SynthesisOptimizationGraph Neural NetworkTransformerDiffusion modelGraphBenchmarkPhysics Related
🎯 What it does: This paper proposes M2PDE, a method that utilizes diffusion models for combined multi-physics and multi-component PDE simulations, capable of generating solutions for coupled or large-scale structures using only data from decoupled or small-scale structures for training.
M3-JEPA: Multimodal Alignment via Multi-gate MoE based on the Joint-Embedding Predictive Architecture
Hongyang Lei (Geely AI Lab), Luo Ji (Geely AI Lab)
ClassificationRetrievalTransformerMixture of ExpertsContrastive LearningImageTextMultimodalityAudio
🎯 What it does: A framework for aligning arbitrary modalities to arbitrary modalities, M3-JEPA, is proposed, utilizing a Mixture-of-Experts (MMoE) predictor to achieve cross-modal alignment in the Joint-Embedding Predictive Architecture (JEPA);
M³HF: Multi-agent Reinforcement Learning from Multi-phase Human Feedback of Mixed Quality
Ziyan Wang (King's College London), Yali Du (Carnegie Mellon University)
Robotic IntelligenceReinforcement Learning from Human FeedbackLarge Language ModelReinforcement LearningMultimodality
🎯 What it does: A framework named M3HF is proposed, which iteratively integrates multi-stage, quality-mixed human feedback during the multi-agent reinforcement learning process to enhance multi-agent collaborative behavior.
MA-LoT: Model-Collaboration Lean-based Long Chain-of-Thought Reasoning enhances Formal Theorem Proving
Ruida WANG, Tong Zhang (University of Illinois Urbana Champaign)
Large Language ModelTextChain-of-Thought
🎯 What it does: The MA-LoT framework is proposed, which splits the Lean4 formal proof task into two stages: global proof planning and fine-grained error correction, and trains a large language model (LLM) with long chain reasoning (Long CoT) capability through LoT-Transfer Learning.
Machine Learning meets Algebraic Combinatorics: A Suite of Datasets Capturing Research-level Conjecturing Ability in Pure Mathematics
Herman Chau (University of Washington), Henry Kvinge (Pacific Northwest National Laboratory)
Graph Neural NetworkTransformerLarge Language ModelPrompt EngineeringGraphTabular
🎯 What it does: This paper proposes the 'ACD Data Warehouse' aimed at combinatorial algebra, which constructs nine categories of datasets containing a large number of instances and open research questions, with the goal of inspiring machine learning models during the conjecture phase;
Machines and Mathematical Mutations: Using GNNs to Characterize Quiver Mutation Classes
Jesse He (University of California San Diego), Henry Kvinge (Pacific Northwest National Laboratory)
ClassificationExplainability and InterpretabilityGraph Neural NetworkGraph
🎯 What it does: This paper classifies the quiver mutation equivalence classes in cluster algebra using graph neural networks and extracts mathematical features through interpretability tools, thereby rediscovering and proving the complete classification of the ˜D type mutation classes.
MAGELLAN: Metacognitive predictions of learning progress guide autotelic LLM agents in large goal spaces
Loris Gaven (Inria), Pierre-Yves Oudeyer (Inria)
Meta LearningTransformerLarge Language ModelReinforcement LearningAgentic AIText
🎯 What it does: The MAGELLAN framework is proposed, enabling LLM agents to self-drive curriculum learning by predicting learning progress through metacognition in a large-scale, structured language target space.
Mahalanobis++: Improving OOD Detection via Feature Normalization
Maximilian Müller, Matthias Hein (University of Tübingen)
Anomaly DetectionConvolutional Neural NetworkImage
🎯 What it does: The Mahalanobis++ method is proposed, which normalizes features using ℓ2 normalization before calculating the Mahalanobis distance for OOD detection.
Maintaining Proportional Committees with Dynamic Candidate Sets
Chris Dong (Munich University of Technology), Jannik Peters (National University of Singapore)
Optimization
🎯 What it does: This paper studies an online algorithm for multi-winner elections with dynamically changing candidate sets, aiming to maintain proportional representation at each step while minimizing changes to the selected committee.
Make LoRA Great Again: Boosting LoRA with Adaptive Singular Values and Mixture-of-Experts Optimization Alignment
Chenghao Fan (Huazhong University of Science and Technology), Yu Cheng (Chinese University of Hong Kong)
ClassificationGenerationOptimizationSupervised Fine-TuningMixture of ExpertsImageText
🎯 What it does: The GOAT framework is proposed, which significantly enhances the performance of LoRA through a Mixture-of-Experts structure with SVD, adaptive prior initialization, and a theoretical gradient alignment scaling factor.
Making Hard Problems Easier with Custom Data Distributions and Loss Regularization: A Case Study in Modular Arithmetic
Eshika Saxena (Meta AI), Kristin E. Lauter
Transformer
🎯 What it does: The research improves the training data distribution and loss function, allowing machine learning models to achieve higher accuracy in modular arithmetic tasks (especially large-scale modular summation) and in attacks on Learning With Errors (LWE).
MapEval: A Map-Based Evaluation of Geo-Spatial Reasoning in Foundation Models
Mahir Labib Dihan (Bangladesh University of Engineering and Technology), Md Rizwan Parvez (Qatar Computing Research Institute)
TransformerLarge Language ModelVision Language ModelTextMultimodalityBenchmark
🎯 What it does: The MapEval benchmark is proposed, which evaluates the geospatial reasoning capabilities of large language models and vision-language models in three scenarios: text, API interaction, and visual maps, using 700 multiple-choice questions.
MAPLE: Many-Shot Adaptive Pseudo-Labeling for In-Context Learning
Zihan Chen (University of Virginia), Cong Shen (University of Virginia)
ClassificationData-Centric LearningGraph Neural NetworkLarge Language ModelText
🎯 What it does: This paper proposes a multi-sample adaptive pseudo-labeling framework called MAPLE, which enhances a large number of demonstrations through pseudo-labels in scenarios with scarce labels, improving the performance of large language models in multi-sample context learning.
MARGE: Improving Math Reasoning with Guided Exploration
Jingyue Gao (Tsinghua University), Jianyu Chen (Tsinghua University)
OptimizationTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: The MARGE framework is proposed, which generates high-quality and diverse training samples through guided exploration of the intermediate reasoning states of self-generated answers, thereby improving the single-step and multi-step accuracy of large language models in mathematical reasoning tasks.
MARS: Unleashing the Power of Variance Reduction for Training Large Models
Huizhuo Yuan (University of California), Quanquan Gu (University of California)
OptimizationTransformerLarge Language ModelText
🎯 What it does: A unified MARS optimization framework is proposed, combining variance reduction with adaptive preconditioning for large model training.
MAS-GPT: Training LLMs to Build LLM-based Multi-Agent Systems
Rui Ye (Shanghai Jiao Tong University), Jing Shao (Shanghai AI Laboratory)
TransformerLarge Language ModelSupervised Fine-TuningAgentic AIText
🎯 What it does: Trained MAS-GPT, a model capable of generating executable multi-agent systems (MAS) in a single LLM inference and answering multi-domain queries.
Mask-Enhanced Autoregressive Prediction: Pay Less Attention to Learn More
Xialie Zhuang (University of Chinese Academy of Sciences), Shiwei Liu (University of Oxford)
GenerationRetrievalTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Without changing the existing decoder-only structure, 10%-15% of the input tokens are randomly masked, and then standard next token prediction is directly performed, constructing the Mask-Enhanced Autoregressive Prediction (MEAP) training paradigm.
Masked Autoencoders Are Effective Tokenizers for Diffusion Models
Hao Chen (Carnegie Mellon University), Bhiksha Raj (Carnegie Mellon University)
GenerationData SynthesisTransformerDiffusion modelAuto EncoderImage
🎯 What it does: Proposes MAETok, using a masked autoencoder as a tokenizer to construct a structured latent space to enhance the generation quality of diffusion models.
Masked Generative Nested Transformers with Decode Time Scaling
Sahil Goyal (Google DeepMind), Sujoy Paul (Google DeepMind)
GenerationComputational EfficiencyTransformerDiffusion modelImageVideo
🎯 What it does: This paper proposes the MaGNeTS framework, which improves inference computation efficiency by dynamically scheduling nested Transformer models of different sizes during image/video generation and combining it with KV caching.
MaskTwins: Dual-form Complementary Masking for Domain-Adaptive Image Segmentation
Jiawen Wang (University of Science and Technology of China), Zhiwei Xiong (University of Science and Technology of China)
SegmentationDomain AdaptationImageBiomedical Data
🎯 What it does: The MaskTwins framework is proposed, which enforces consistency learning on target domain images through bidirectional complementary masks, achieving unsupervised domain adaptive semantic segmentation.
MASS: Mathematical Data Selection via Skill Graphs for Pretraining Large Language Models
Jiazheng Li (University of Connecticut), Chuxu Zhang (University of Connecticut)
TransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper proposes the MASS framework, which enhances the performance of large language models (LLMs) in mathematical reasoning tasks by efficiently selecting data through the construction of a Skill Graph.
Massive Values in Self-Attention Modules are the Key to Contextual Knowledge Understanding
Mingyu Jin (Rutgers University), Yongfeng Zhang (Rutgers University)
TransformerLarge Language ModelText
🎯 What it does: This paper systematically analyzes the 'massive values' generated by the attention module's Q and K in large language models, revealing their key role in understanding contextual knowledge.
Mastering Board Games by External and Internal Planning with Language Models
John Schultz (Google DeepMind), Nenad Tomasev (Google DeepMind)
TransformerLarge Language ModelReinforcement LearningAgentic AIText
🎯 What it does: This paper enhances the playing strength of various board games such as chess, Chess960, Connect Four, and Hex by performing multi-task pre-training and fine-tuning on large language models, enabling them to simultaneously execute board state tracking, legal move prediction, value assessment, and policy output, while integrating with external MCTS and internal search.
Mastering Massive Multi-Task Reinforcement Learning via Mixture-of-Expert Decision Transformer
Yilun Kong (Tsinghua University), Dacheng Tao (Nanyang Technological University)
TransformerReinforcement LearningMixture of ExpertsSequential
🎯 What it does: In offline multi-task reinforcement learning, researchers proposed a framework based on the Mixture-of-Expert Decision Transformer (M3DT) to achieve efficient learning for a vast number of tasks;
Mastering Multiple-Expert Routing: Realizable $H$-Consistency and Strong Guarantees for Learning to Defer
Anqi Mao (Courant Institute of Mathematical Sciences), Yutao Zhong (Google Research)
ClassificationConvolutional Neural NetworkMixture of ExpertsImage
🎯 What it does: A novel realizable H-consistent surrogate loss function for learning to defer in multi-expert learning is proposed, along with theoretical guarantees and algorithm implementations for both single-stage and double-stage scenarios.
MATH-Perturb: Benchmarking LLMs' Math Reasoning Abilities against Hard Perturbations
Kaixuan Huang (Princeton University), Mengdi Wang
Large Language ModelTextBenchmarkChain-of-Thought
🎯 What it does: A MATH-Perturb benchmark (including MATH-P-Simple and MATH-P-Hard) was constructed to evaluate the robustness of large language models against hard perturbations in mathematical reasoning tasks, and zero-shot chain-of-thought (CoT) evaluations were conducted on 18 models.
MathConstruct: Challenging LLM Reasoning with Constructive Proofs
Mislav Balunovic (ETH Zurich), Martin Vechev (ETH Zurich)
Large Language ModelTextBenchmark
🎯 What it does: A benchmark called MATHCONSTRUCT is proposed, which collects and encodes 126 constructive proof problems from high-level mathematics competitions, equipped with an automatically verifiable judge and parameterized variants;
Matrix Completion with Incomplete Side Information via Orthogonal Complement Projection
Gengshuo Chang (Harbin Institute of Technology), Lehan Zhang (Harbin Institute of Technology)
Recommendation SystemOptimizationTabular
🎯 What it does: A matrix completion model OCMC utilizing orthogonal complement projection is proposed to address the low-rank matrix completion problem when side information is incomplete.
Matryoshka Quantization
Pranav Ajit Nair, Aditya Kusupati (Google DeepMind)
CompressionOptimizationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: A multi-scale quantization method called MatQuant is proposed, which can support different precisions such as int8, int4, and int2 within the same model.
MATS: An Audio Language Model under Text-only Supervision
Wen Wang (Chinese Academy of Sciences), Xilin Chen (Chinese Academy of Sciences)
ClassificationRecognitionTransformerLarge Language ModelSupervised Fine-TuningTextMultimodalityAudio
🎯 What it does: This paper presents MATS, a multimodal large language model capable of performing various audio tasks (classification, subtitles, question answering) using only text supervision.
Maximal Update Parametrization and Zero-Shot Hyperparameter Transfer for Fourier Neural Operators
Shanda Li (Carnegie Mellon University), Yiming Yang (Carnegie Mellon University)
Computational EfficiencyHyperparameter SearchTime SeriesBenchmark
🎯 What it does: This paper proposes µ Transfer-FNO, a zero-shot hyperparameter transfer method that can directly transfer the optimal hyperparameters obtained from fine-tuning a small-scale Fourier Neural Operator (FNO) to large models with parameter counts reaching billions, without the need for re-tuning.
Maximizing Intermediate Checkpoint Value in LLM Pretraining with Bayesian Optimization
Deyuan Liu (Harbin Institute of Technology), Dianbo Sui (Harbin Institute of Technology)
OptimizationTransformerLarge Language ModelText
🎯 What it does: During the pre-training process of large language models, the authors propose a method to enhance model performance by linearly fusing adjacent intermediate checkpoints.
Maximum Coverage in Turnstile Streams with Applications to Fingerprinting Measures
Alina Ene (Boston University), Peilin Zhong (Google Research)
OptimizationComputational EfficiencyTabular
🎯 What it does: The paper presents the first universally applicable turnstile maximum coverage algorithm and applies it to risk management objectives and general fingerprinting problems. It also provides a new linear sketch for estimating the complement of the p-th frequency matrix of a vector (n^p – F_p) and achieves fast fingerprinting.
Maximum Entropy Reinforcement Learning with Diffusion Policy
Xiaoyi Dong (Institute of Automation, Chinese Academy of Sciences), Xi Sheryl Zhang (University of Chinese Academy of Sciences)
Reinforcement LearningDiffusion modelTabularOrdinary Differential Equation
🎯 What it does: This paper proposes the use of diffusion models as policy representations within the maximum entropy reinforcement learning framework and provides methods for training and probability estimation.
Maximum Total Correlation Reinforcement Learning
Bang You (Wuhan University of Technology), Oleg Arenz (Technische Universität Darmstadt)
Reinforcement LearningSequential
🎯 What it does: A reinforcement learning regularization method based on total correlation (MTC-RL) is proposed, which encourages the generation of compressible, periodic, and robust behaviors by maximizing the total correlation of state-action trajectories produced by the policy.
MCU: An Evaluation Framework for Open-Ended Game Agents
Xinyue Zheng (Beijing Institute for General Artificial Intelligence), Yitao Liang (Peking University)
Robotic IntelligenceTransformerLarge Language ModelReinforcement LearningVision Language ModelTextMultimodality
🎯 What it does: An open game agent evaluation framework named MCU was built in Minecraft, which includes 3,452 combinable atomic tasks, a task combination mechanism, and an automatic evaluation system based on visual-language models;
MDDM: Practical Message-Driven Generative Image Steganography Based on Diffusion Models
ZiHao Xu, Chuan Zhang (Beijing Institute of Technology)
GenerationData SynthesisSafty and PrivacyDiffusion modelImage
🎯 What it does: A practical message-driven generative image steganography method based on diffusion models, MDDM, is proposed. It encodes the ciphertext into the initial noise using a Cardan grille and generates the steganographic image through DDIM. The receiver can recover the ciphertext simply by performing diffusion inversion.
Measuring Diversity in Synthetic Datasets
Yuchang Zhu (Sun Yat-sen University), Yatao Bian (National University of Singapore)
ClassificationData SynthesisContrastive LearningText
🎯 What it does: A method called DCScore is proposed to evaluate the diversity of synthetic datasets from a classification perspective.