ICML 2025 Papers — Page 16
International Conference on Machine Learning · 3257 papers
Is Complex Query Answering Really Complex?
Cosimo Gregucci (University of Stuttgart), Antonio Vergari (University of Edinburgh)
Graph Neural NetworkGraphBenchmark
🎯 What it does: This paper analyzes the 'complexity' of current Complex Query Answering (CQA) benchmarks, finding that most queries can actually be simplified to simpler types (especially single-step link prediction). It proposes a more balanced benchmark (FB15k237+H, NELL995+H, ICEWS18+H) and a hybrid solver CQD-Hybrid that utilizes training links.
Is Noise Conditioning Necessary for Denoising Generative Models?
Qiao Sun (Massachusetts Institute of Technology), Kaiming He (Massachusetts Institute of Technology)
RestorationGenerationDiffusion modelImageStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Conduct systematic experiments and theoretical analysis on the impact of noise conditioning on denoising generative models, and propose a noise unconditional variant of EDM.
Is Your Model Fairly Certain? Uncertainty-Aware Fairness Evaluation for LLMs
Yinong Oliver Wang (Carnegie Mellon University), Nicholas Apostoloff
TransformerLarge Language ModelText
🎯 What it does: Proposes a fairness metric based on uncertainty, UCerF, and constructs a large-scale gender-occupation co-reference dataset, SynthBias, for fine-grained fairness evaluation of large language models.
Isolated Causal Effects of Natural Language
Victoria Lin (Carnegie Mellon University), Eli Ben-Michael (Carnegie Mellon University)
Text
🎯 What it does: A framework for estimating the causal effects of isolated text is proposed, along with evaluation metrics based on omitted variable bias.
It's My Data Too: Private ML for Datasets with Multi-User Training Examples
Arun Ganesh (Google Research), Fan Wu (University of Illinois)
Federated LearningSafty and PrivacyTextTabular
🎯 What it does: This paper studies and implements a user-level differential privacy training method for multi-user attribution datasets, proposing a fixed graph DP definition and a contribution boundary selection algorithm, and conducting experimental evaluations on real data.
IT$^3$: Idempotent Test-Time Training
Nikita Durasov (NVIDIA), Pascal Fua (EPFL)
Domain AdaptationConvolutional Neural NetworkGraph Neural NetworkImageTabular
🎯 What it does: A testing-time training (IT³) method is proposed that can achieve instantaneous adaptation to distribution shifts using only the current test sample during inference;
ITBench: Evaluating AI Agents across Diverse Real-World IT Automation Tasks
Saurabh Jha (IBM), Ruchir Puri (IBM)
TransformerLarge Language ModelAgentic AITextBenchmarkFinance Related
🎯 What it does: ITBench has been constructed, an open-source and scalable benchmarking framework for evaluating the performance of AI agents on real IT automation tasks (SRE, CISO, FinOps).
Iterative Vectors: In-Context Gradient Steering without Backpropagation
Yiting Liu (Peking University), Zhi-Hong Deng (Peking University)
ClassificationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper proposes a new gradient-free backpropagation activation vector method—Iterative Vectors (IVs), which enhances performance in In-Context Learning (ICL) by iteratively extracting and updating task-relevant meta-gradients in the attention activation space of language models.
ITFormer: Bridging Time Series and Natural Language for Multi-Modal QA with Large-Scale Multitask Dataset
Yilin wang, zhongyu wei
TransformerLarge Language ModelSupervised Fine-TuningMultimodalityTime Series
🎯 What it does: A multi-task dataset for time-series question answering (Time-Series QA) called EngineMT-QA has been constructed, and the ITFormer framework has been proposed, which connects any time-series encoder with a frozen LLM to achieve cross-modal question answering between time-series and natural language.
Jacobian Sparse Autoencoders: Sparsify Computations, Not Just Activations
Lucy Farnik (University of Bristol), Laurence Aitchison (University of Bristol)
TransformerAuto EncoderText
🎯 What it does: This paper proposes Jacobian Sparse Autoencoders (JSAE) to explicitly sparsify the computation graph of transformer MLP while maintaining activation sparsity.
Janus: Dual-Server Multi-Round Secure Aggregation with Verifiability for Federated Learning
Lang Pu (Nanjing University of Aeronautics and Astronautics), Xinyi Huang (Jinan University)
Federated LearningSafty and PrivacyComputational EfficiencyConvolutional Neural NetworkImage
🎯 What it does: A secure aggregation framework called Janus is proposed, which is based on dual servers, supports multiple rounds, and is verifiable, addressing issues of dynamic participation, verifiability, and model inconsistency attacks in federated learning (FL).
Joint Learning of Energy-based Models and their Partition Function
Michael Eli Sander, Mathieu Blondel (Google DeepMind)
ClassificationOptimizationTabular
🎯 What it does: A min-min framework is proposed for jointly learning energy models and their log normalization functions (partition function) in combinatorial large-scale discrete spaces, enabling maximum likelihood estimation (MLE) through stochastic gradient descent without using MCMC.
Joint Localization and Activation Editing for Low-Resource Fine-Tuning
Wen Lai (Technical University of Munich), Ivan Titov (University of Amsterdam)
TransformerSupervised Fine-TuningText
🎯 What it does: Proposed and implemented the JOLA method, which can jointly learn the positioning and activation editing (incremental and multiplicative) of Transformer attention heads in low-resource fine-tuning scenarios.
Joint Metric Space Embedding by Unbalanced Optimal Transport with Gromov–Wasserstein Marginal Penalization
Florian Beier (Institut für Mathematik, Technische Universität Berlin), Gabriele Steidl (Institut für Mathematik, Technische Universität Berlin)
OptimizationMultimodalityMesh
🎯 What it does: A joint metric space embedding method based on unbalanced optimal transport and Gromov-Wasserstein edge penalty is proposed, which can map two sets of heterogeneous data without correspondence into the same metric space for visualization and comparison.
Joint MoE Scaling Laws: Mixture of Experts Can Be Memory Efficient
Jan Ludziejewski (University of Warsaw), Sebastian Jaszczur (University of Warsaw)
Mixture of ExpertsText
🎯 What it does: A joint extension law suitable for sparse mixture of experts (MoE) and dense models is proposed, describing the relationship between model loss, number of parameters, number of training samples, and number of experts under a given computation or memory budget.
Joker: Joint Optimization Framework for Lightweight Kernel Machines
Junhong Zhang (Shenzhen University), Zhihui Lai (Shenzhen University)
OptimizationSupervised Fine-TuningTabular
🎯 What it does: A Joker joint optimization framework is proposed, capable of training multiple lightweight kernel machines (KRR, KLR, SVM, etc.) in one go.
Just Enough Shifts: Mitigating Over-Refusal in Aligned Language Models with Targeted Representation Fine-Tuning
Mahavir Dabas (Virginia Tech), Ruoxi Jia (Virginia Tech)
Computational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Proposes the ACTOR framework, which fine-tunes a single layer using internal activation vectors to reduce over-refusal of legitimate requests while maintaining security.
K$^2$IE: Kernel Method-based Kernel Intensity Estimators for Inhomogeneous Poisson Processes
Hideaki Kim (NTT Corporation), Akinori Fujino (NTT Corporation)
Time Series
🎯 What it does: This study proposes a kernel intensity estimator (KIE) based on least squares loss for estimating the intensity function of infinite small Poisson processes.
KABB: Knowledge-Aware Bayesian Bandits for Dynamic Expert Coordination in Multi-Agent Systems
Jusheng Zhang (Sun Yat-sen University), Keze Wang (Guangdong Key Laboratory of Big Data Analysis and Processing)
OptimizationComputational EfficiencyReinforcement LearningAgentic AITextBenchmark
🎯 What it does: A knowledge-aware Bayesian multi-armed bandit (KABB) framework is proposed for dynamically coordinating experts in multi-agent systems, enhancing task execution effectiveness while reducing computational costs.
KAN-AD: Time Series Anomaly Detection with Kolmogorov–Arnold Networks
Quan Zhou (Chinese Academy of Sciences), Jianhui li
Anomaly DetectionConvolutional Neural NetworkTime Series
🎯 What it does: Proposes KAN-AD, which utilizes Kolmogorov-Arnold networks and Fourier series for smooth modeling of time series, thereby achieving anomaly detection;
Kandinsky Conformal Prediction: Beyond Class- and Covariate-Conditional Coverage
Konstantina Bairaktari (Northeastern University), Steven Wu
ClassificationOptimizationTabular
🎯 What it does: This paper proposes Kandinsky Conformal Prediction, which provides conditional coverage guarantees for overlapping and fractional subgroups (depending on both features and labels) in terms of coverage probability, and implements algorithms for efficient quantile prediction sets during both training and testing.
KBQA-o1: Agentic Knowledge Base Question Answering with Monte Carlo Tree Search
Haoran Luo (Beijing University of Posts and Telecommunications), Anh Tuan Luu
Reinforcement LearningAgentic AIText
🎯 What it does: A KBQA method called KBQA-o1 based on Agent and MCTS is proposed, which utilizes a ReAct-style interactive toolchain to generate logical forms and execute queries;
KEA: Keeping Exploration Alive by Proactively Coordinating Exploration Strategies
Shih-Min Yang (Orebro University), Todor Stoyanov (Orebro University)
Reinforcement Learning
🎯 What it does: Proposes the KEA method, which actively coordinates SAC and novelty exploration strategies to address the exploration efficiency problem in sparse reward environments.
Kernel Quantile Embeddings and Associated Probability Metrics
Masha Naslidnyk (University College London), Krikamol Muandet (CISPA Helmholtz Center for Information Security)
Image
🎯 What it does: This paper proposes Kernel Quantization Embedding (KQE) and the corresponding Kernel Quantization Divergence (KQD), proving that it can form a probability metric under weaker kernel conditions, and provides a near-linear complexity estimator based on Gaussian measure sampling; it is then compared with methods such as MMD in two-sample testing tasks.
Kernel-based Unsupervised Embedding Alignment for Enhanced Visual Representation in Vision-language Models
Shizhan Gong (Chinese University of Hong Kong), Farzan Farnia (Chinese University of Hong Kong)
ClassificationRepresentation LearningTransformerVision Language ModelContrastive LearningImageText
🎯 What it does: Perform unsupervised alignment of the embeddings of the CLIP visual encoder and the DINOv2 visual encoder based on kernel functions, thereby enhancing CLIP's fine-grained visual perception capabilities while maintaining alignment with the text encoder.
KernelBench: Can LLMs Write Efficient GPU Kernels?
Anne Ouyang (Stanford University), Azalia Mirhoseini (Stanford University)
OptimizationComputational EfficiencyAI Code AssistantTransformerLarge Language ModelTextBenchmark
🎯 What it does: KernelBench is proposed, an open evaluation framework for automatically generating efficient GPU kernels for large language models, and it evaluates the functionality and speed of models based on 250 real PyTorch tasks.
KGMark: A Diffusion Watermark for Knowledge Graphs
Hongrui Peng (Beijing University Of Posts and Telecommunications), Guoshun Nan (Beijing University Of Posts and Telecommunications)
Recommendation SystemGraph Neural NetworkGraph
🎯 What it does: KGMark is proposed, a diffusion watermarking framework for knowledge graphs, aimed at generating robust, detectable, and transparent diffusion fingerprints for dynamic knowledge graph data.
KIND: Knowledge Integration and Diversion for Training Decomposable Models
Yucheng Xie (Southeast University), Xin Geng (Southeast University)
Domain AdaptationRepresentation LearningTransformerDiffusion modelImageMagnetic Resonance Imaging
🎯 What it does: Proposes the KIND pre-training method, which utilizes SVD structure to split model weights into class-independent learn genes and class-specific tailors, allowing the model to be flexibly restructured according to different deployment needs and cross-domain tasks.
KinDEL: DNA-Encoded Library Dataset for Kinase Inhibitors
Benson Chen (Insitro), R. Edward Watts (Insitro)
Drug DiscoveryGraph Neural NetworkTabularBiomedical Data
🎯 What it does: An 81M molecular DNA-encoded library (DEL) dataset named KinDEL has been released, along with experimental data and offline validation biophysical measurement results for two kinase targets (MAPK14 and DDR1);
Kinetic Langevin Diffusion for Crystalline Materials Generation
François R J Cornet (Technical University of Denmark), Mikkel N. Schmidt (Technical University of Denmark)
GenerationData SynthesisDiffusion modelGraphPhysics Related
🎯 What it does: A crystal material generation model based on Kinetic Langevin Diffusion (KLDM) is proposed, which utilizes the coupling of velocity variables and fractional coordinates to achieve adaptive handling of periodic translational symmetry.
Knowledge Retention in Continual Model-Based Reinforcement Learning
Haotian Fu (Brown University), George Konidaris (Brown University)
Reinforcement LearningDiffusion modelWorld ModelSequential
🎯 What it does: A continuous model-based reinforcement learning framework named DRAGO is proposed, which maintains and expands the world model through synthetic experience replay and exploration-based memory recovery.
Knowledge Swapping via Learning and Unlearning
Mingyu Xing (Hefei University of Technology), Meng Wang
ClassificationObject DetectionSegmentationTransformerSupervised Fine-TuningImage
🎯 What it does: Proposes the task of 'Knowledge Swapping', which requires selectively forgetting specified knowledge while retaining the core knowledge of the pre-trained model and learning new knowledge.
Knowledge-Guided Wasserstein Distributionally Robust Optimization
Zitao Wang (Columbia University), Nian Si (Hong Kong University of Science and Technology)
Domain AdaptationOptimizationTabular
🎯 What it does: A knowledge-guided Wasserstein distributionally robust optimization (KG-WDRO) framework is proposed to enhance the generalization performance of linear regression and binary classification in small sample transfer learning scenarios.
Kona: An Efficient Privacy-Preservation Framework for KNN Classification by Communication Optimization
Guopeng Lin (Fudan University), Tao Wei (Ant Group)
ClassificationSafty and PrivacyComputational EfficiencyTabular
🎯 What it does: This study proposes the Kona framework, which implements multi-party privacy-preserving KNN classification, addressing the issues of large communication volume and multiple rounds in traditional methods during the online phase.
KoNODE: Koopman-Driven Neural Ordinary Differential Equations with Evolving Parameters for Time Series Analysis
Hanru Bai (Fudan University), Weiyang Ding (Fudan University)
Robotic IntelligenceTime SeriesOrdinary Differential Equation
🎯 What it does: This paper proposes the KoNODE framework, which models at three hierarchical levels (observation state, ODE parameters, and Koopman linear dynamics) to enable neural ODEs to learn time-evolving parameters and capture the deep linear structure of the system.
KoopSTD: Reliable Similarity Analysis between Dynamical Systems via Approximating Koopman Spectrum with Timescale Decoupling
Shimin Zhang (Hong Kong Polytechnic University), Jibin Wu (Hong Kong Polytechnic University)
Recurrent Neural NetworkTextTime SeriesSequentialBiomedical DataMagnetic Resonance Imaging
🎯 What it does: A method for measuring dynamical similarity based on Koopman operator theory, called KoopSTD, is proposed. It utilizes time-frequency transformation and singular value decomposition to achieve multi-scale temporal decoupling, and improves the reliability of similarity measurement by controlling spectral residuals to eliminate pseudo-spectra.
KV Shifting Attention Enhances Language Modeling
Mingyu Xu (Baichuan-inc), Weipeng Chen (Baichuan-inc)
TransformerLarge Language ModelText
🎯 What it does: This paper studies and implements KV Shifting Attention, improving the key-value decoupling mechanism of the Transformer to achieve and accelerate the learning of Induction Heads in a single-layer and narrower network, thereby enhancing language modeling performance.
KVTuner: Sensitivity-Aware Layer-Wise Mixed-Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inference
Xing Li (Huawei Noah's Ark Lab), Mingxuan Yuan (Huawei Noah's Ark Lab)
OptimizationComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: The KVTuner framework is proposed to achieve hierarchical mixed-precision quantization of LLM KV caches, improving inference throughput and memory efficiency.
L-Diffusion: Laplace Diffusion for Efficient Pathology Image Segmentation
Weihan Li (Zhejiang University), Zunlei Feng (Zhejiang University)
SegmentationConvolutional Neural NetworkDiffusion modelContrastive LearningImage
🎯 What it does: A diffusion model based on the Laplace distribution (L-Diffusion) is proposed for pathological image segmentation, achieving differentiation of various tissue/cell types through pixel latent vector contrastive learning.
L3A: Label-Augmented Analytic Adaptation for Multi-Label Class Incremental Learning
Xiang Zhang (South China University of Technology), Huiping Zhuang (South China University of Technology)
ClassificationObject DetectionConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: This paper proposes a sample-free multi-label incremental learning method called L3A, which can retain learned knowledge when new categories are added.
La RoSA: Enhancing LLM Efficiency via Layerwise Rotated Sparse Activation
Kai Liu (Alibaba Group), lulu hu
Computational EfficiencyTransformerLarge Language ModelText
🎯 What it does: LaRoSA is proposed, a method for achieving activation sparsity in LLM inference quickly and stably without additional training.
Label Distribution Propagation-based Label Completion for Crowdsourcing
Tong Wu (China University of Geosciences), Chaoqun Li (China University of Geosciences)
Tabular
🎯 What it does: A label completion algorithm based on label distribution propagation, LDPLC, is proposed to alleviate the sparsity problem of crowdsourced label matrices and improve subsequent label integration effects.
LaCache: Ladder-Shaped KV Caching for Efficient Long-Context Modeling of Large Language Models
Dachuan Shi (Georgia Tech), Yingyan Celine Lin
GenerationCompressionComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: A training-free KV cache optimization framework called LaCache is proposed, which supports long-context continuous generation through a trapezoidal storage pattern and iterative compression.
LADA: Scalable Label-Specific CLIP Adapter for Continual Learning
Mao-Lin Luo (Southeast University), Min-Ling Zhang (Southeast University)
ClassificationKnowledge DistillationTransformerContrastive LearningImage
🎯 What it does: A label-specific CLIP adapter named LADA is proposed, which extends the CLIP image encoder with lightweight memory vectors to achieve parameter-free continual learning.
Ladder-Residual: Parallelism-Aware Architecture for Accelerating Large Model Inference with Communication Overlapping
Muru Zhang (University of Southern California), Tri Dao (Princeton University)
TransformerSupervised Fine-TuningText
🎯 What it does: This paper proposes and validates a Transformer architecture modification called Ladder Residual, which decouples communication and computation by utilizing the 'stale' inputs of residual networks, achieving communication overlap under Tensor Parallelism to accelerate inference of large-scale models.
LAION-C: An Out-of-Distribution Benchmark for Web-Scale Vision Models
Fanfei Li (Max Planck Institute for Intelligent Systems), Roland S. Zimmermann (Google DeepMind)
ClassificationRecognitionAnomaly DetectionConvolutional Neural NetworkTransformerSupervised Fine-TuningContrastive LearningImageBenchmark
🎯 What it does: This paper presents LAION-C, a novel OOD benchmark specifically designed for large-scale web-crawled datasets (such as LAION), which includes six types of artificially synthesized and highly challenging distortions.
LaMAGIC2: Advanced Circuit Formulations for Language Model-Based Analog Topology Generation
Chen-Chia Chang (Duke University), Xin Zhang (IBM T. J. Watson Research Center)
GenerationOptimizationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper presents LaMAGIC2, an improved circuit representation (SFCI/SFM), which utilizes language models to achieve more efficient and accurate simulation circuit topology generation.
LangDAug: Langevin Data Augmentation for Multi-Source Domain Generalization in Medical Image Segmentation
Piyush Tiwary (Indian Institute of Science), Prathosh AP
SegmentationDomain AdaptationImageBiomedical DataMagnetic Resonance ImagingStochastic Differential Equation
🎯 What it does: An enhanced method called LangDAug is proposed, which utilizes Langevin dynamics to generate intermediate samples to improve the domain generalization ability of multi-source medical image segmentation.
LangTime: A Language-Guided Unified Model for Time Series Forecasting with Proximal Policy Optimization
Wenzhe Niu (Tianjin University), Chao Hao
TransformerLarge Language ModelReinforcement LearningPrompt EngineeringTime Series
🎯 What it does: Proposes the LangTime model, which maps multi-domain time series to the LLM semantic space using natural language prompts (Temporal Comprehension Prompts) and fine-tunes through TimePPO reinforcement learning after pre-training to reduce error accumulation in autoregressive predictions.
Language Models as Implicit Tree Search
Ziliang Chen (Peng Cheng Laboratory), Liang Lin (Sun Yat-sen University)
OptimizationTransformerLarge Language ModelReinforcement LearningTextChain-of-Thought
🎯 What it does: A preference optimization method that is completely free of RL and value functions has been constructed by introducing an Implicit Tree Search (ITS) language model within the DPO framework.
Language Models May Verbatim Complete Text They Were Not Explicitly Trained On
Ken Liu, Nicolas Papernot (Google)
Adversarial AttackTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper studies the limitations of n-gram based methods in membership inference of training data for large language models, and demonstrates through experiments that the model can complete text on data that does not contain any n-grams.
Language Models over Canonical Byte-Pair Encodings
Tim Vieira (ETH Zurich), Ryan Cotterell (ETH Zurich)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This study investigates the probability distribution errors caused by non-standard encoding when using BPE tokenization in language models and proposes a method for enforcing canonicality.
Laplace Transform Based Low-Complexity Learning of Continuous Markov Semigroups
Vladimir R Kostic, Massimiliano Pontil (University College London)
Time SeriesSequentialPhysics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: This paper proposes a low-complexity learning method based on the Laplace transform for learning the spectral decomposition of the infinitesimal generator of continuous Markov semigroups from trajectory data, and provides theoretical convergence guarantees.
LapSum - One Method to Differentiate Them All: Ranking, Sorting and Top-k Selection
Łukasz Struski (Jagiellonian University), Jacek Tabor (Jagiellonian University)
OptimizationImage
🎯 What it does: Proposes the LapSum method, which uses the Laplace distribution for inversion to construct differentiable sorting, ranking, top-k, and permutations.
LaRA: Benchmarking Retrieval-Augmented Generation and Long-Context LLMs – No Silver Bullet for LC or RAG Routing
Kuan Li (Hong Kong University of Science and Technology), Minhao Cheng (Penn State University)
GenerationRetrievalTransformerLarge Language ModelTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: This paper systematically evaluates the performance of Retrieval-Augmented Generation (RAG) and Long Context Large Language Models (LC) on different tasks, model sizes, and context lengths by constructing a new long-text benchmark, LaRA.
Large Continual Instruction Assistant
Jingyang Qiao (East China Normal University), Yuan Xie (East China Normal University)
Large Language ModelSupervised Fine-TuningMultimodality
🎯 What it does: A general continuous instruction fine-tuning framework is proposed, which significantly reduces catastrophic forgetting by maintaining old knowledge while introducing new knowledge through adaptive Exponential Moving Average (EMA) weight adjustment and instruction grouping strategies.
Large Displacement Motion Transfer with Unsupervised Anytime Interpolation
Guixiang Wang (Hangzhou Dianzi University), Jianjun Li (Hangzhou Normal University)
Image TranslationGenerationTransformerOptical FlowImageVideo
🎯 What it does: A multi-scale motion transfer method based on unsupervised arbitrary moment interpolation is proposed, which can decompose large displacements between the source image and the driving video into several small displacements, thereby improving the quality of motion transfer.
Large Language Model-driven Large Neighborhood Search for Large-Scale MILP Problems
Huigen Ye (Tsinghua University), Yaoyang Cheng (Tsinghua University)
OptimizationTransformerLarge Language ModelPrompt EngineeringTabular
🎯 What it does: This paper proposes a dual-layer self-evolving LNS framework based on large language models (LLM-LNS) for solving large-scale mixed-integer linear programming problems.
Large Language Models are Demonstration Pre-Selectors for Themselves
Jiarui Jin (Shanghai Jiao Tong University), Mengyue Yang (University of Bristol)
OptimizationComputational EfficiencyLarge Language ModelPrompt EngineeringText
🎯 What it does: A new pre-selection framework called FEEDER is proposed to identify the most representative examples from training data, aiming to improve the efficiency and effectiveness of large language models (LLMs) in in-context learning (ICL).
Large Language Models to Diffusion Finetuning
Edoardo Cetin (Sakana AI), Yujin Tang (Sakana AI)
TransformerLarge Language ModelSupervised Fine-TuningDiffusion modelTextBenchmarkOrdinary Differential Equation
🎯 What it does: This paper proposes a fine-tuning method called L2D, which enables pre-trained large language models to scale computational load during inference like diffusion models, thereby improving multi-step reasoning performance.
Large Language-Geometry Model: When LLM meets Equivariance
Zongzhao Li (Renmin University of China), Wenbing Huang (Renmin University of China)
Graph Neural NetworkTransformerLarge Language ModelPrompt EngineeringGraphPhysics Related
🎯 What it does: This paper proposes the EquiLLM framework, which integrates large language models with equivariant graph neural networks to model 3D physical systems.
Larger or Smaller Reward Margins to Select Preferences for LLM Alignment?
Kexin Huang (University of Science and Technology of China), Xiang Wang
Reinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: A new alignment potential metric M AP is proposed to evaluate the quality of preference data and select the most suitable samples for LLM alignment training.
LARM: Large Auto-Regressive Model for Long-Horizon Embodied Intelligence
Zhuoling Li (University of Hong Kong), Hengshuang Zhao (University of Hong Kong)
Robotic IntelligenceTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: A lightweight autoregressive model LARM was constructed to directly output high-level actions executed in Minecraft and combined with RL training;
LASER: Attention with Exponential Transformation
Sai Surya Duvvuri (University of Texas at Austin), Inderjit S Dhillon
TransformerLarge Language ModelImageTextMultimodalityAudio
🎯 What it does: This paper proposes the LASER (Logarithm of Summed Exponential of Representations) attention mechanism, which enhances the attention gradient using exponential transformation and logarithmic summation to improve the gradient propagation issues in Transformers.
LAST SToP for Modeling Asynchronous Time Series
Shubham Gupta (Universite Laval), Lilian Bialokozowicz
Anomaly DetectionTransformerLarge Language ModelPrompt EngineeringTextTime Series
🎯 What it does: A prompt-based framework called LASTS is proposed, which utilizes large language models (LLMs) for multi-tasks such as asynchronous time series prediction, anomaly detection, and missing value imputation.
Latent Action Learning Requires Supervision in the Presence of Distractors
Alexander Nikulin (Artificial Intelligence Research Institute), Vladislav Kurenkov (Innopolis University)
Robotic IntelligenceReinforcement Learning from Human FeedbackSupervised Fine-TuningReinforcement LearningVideo
🎯 What it does: The study investigates the effects of potential action learning in the presence of action-related distractors and proposes improvement strategies and supervision mechanisms.
Latent Diffusion Planning for Imitation Learning
Amber Xie (Stanford), Chelsea Finn (Stanford)
OptimizationRobotic IntelligenceConvolutional Neural NetworkDiffusion modelAuto EncoderSequential
🎯 What it does: A latent diffusion-based imitation learning framework (Latent Diffusion Planning, LDP) is proposed, decoupling planning from inverse dynamics to predict future states in latent space and output actions through an inverse dynamics model.
Latent Imputation before Prediction: A New Computational Paradigm for De Novo Peptide Sequencing
Ye Du (Hong Kong Polytechnic University), Shujun Wang (Hong Kong Polytechnic University)
TransformerBiomedical DataBenchmark
🎯 What it does: A LIPNovo framework is proposed to first perform potential space missing fragment completion before de novo peptide sequence prediction.
Latent Mamba Operator for Partial Differential Equations
Karn Tiwari (Indian Institute of Science), Prathosh AP
Auto EncoderMeshTime SeriesPhysics RelatedOrdinary Differential Equation
🎯 What it does: A novel neural operator based on the Latent Mamba Operator, a latent space bidirectional state space model, is proposed for solving high-dimensional PDEs;
Latent Preference Coding: Aligning Large Language Models via Discrete Latent Codes
Zhuocheng Gong (Peking University), Dongyan Zhao (Peking University)
Recommendation SystemReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: This paper proposes Latent Preference Coding (LPC), which utilizes discrete latent codes for fine-grained preference modeling of large language models and seamlessly integrates it into various offline RLHF algorithms.
Latent Score-Based Reweighting for Robust Classification on Imbalanced Tabular Data
Yunze Tong (Zhejiang University), Kun Kuang (Zhejiang University)
ClassificationDiffusion modelScore-based ModelAuto EncoderTabular
🎯 What it does: This paper proposes a framework based on latent score reweighting to enhance robust classification on imbalanced tabular data.
Latent Thought Models with Variational Bayes Inference-Time Computation
Deqian Kong (University of California Los Angeles), Ying Nian Wu (University of California Los Angeles)
TransformerLarge Language ModelText
🎯 What it does: A class of language models called Latent Thought Models (LTMs) is proposed, which combines explicit latent thought vectors with a prior model structure, and achieves fast local inference and slow global learning through dual-speed optimization within a variational Bayesian framework.
Latent Variable Causal Discovery under Selection Bias
Haoyue Dai (Carnegie Mellon University), Kun Zhang (Mohamed bin Zayed University of Artificial Intelligence)
Tabular
🎯 What it does: A rank-constrained latent variable causal discovery method is proposed, which can simultaneously handle latent variables and selection bias.
Latent Variable Estimation in Bayesian Black-Litterman Models
Thomas Yuan-Lung Lin, Peter Lin (Johns Hopkins University)
Recommendation SystemOptimizationGaussian SplattingTabularTime SeriesFinance Related
🎯 What it does: This paper proposes a Bayesian Black-Litterman model without subjective investor views, which infers portfolio weights directly from asset characteristics and provides a closed-form solution or a posterior distribution that can be numerically approximated, addressing the issue of traditional models relying on human perspectives.
LAuReL: Learned Augmented Residual Layer
Gaurav Menghani (Google Research), Sanjiv Kumar (Google Research)
OptimizationComputational EfficiencyConvolutional Neural NetworkLarge Language ModelImageTextMultimodality
🎯 What it does: A learnable augmented residual layer (LAUREL) is proposed, which can replace traditional residual connections to enhance model quality and efficiency;
Layer by Layer: Uncovering Hidden Representations in Language Models
Oscar Skean (University of Kentucky), Ravid Shwartz-Ziv (New York University)
Representation LearningTransformerLarge Language ModelContrastive LearningText
🎯 What it does: A hierarchical analysis of the intermediate layer representations of large language models reveals that they outperform the final layer in various downstream tasks;
Layer-wise Alignment: Examining Safety Alignment Across Image Encoder Layers in Vision Language Models
Saketh Bachu (University of California), Amit Roy-Chowdhury
Safty and PrivacyReinforcement Learning from Human FeedbackTransformerReinforcement LearningVision Language ModelMultimodality
🎯 What it does: Systematically reveals and mitigates security vulnerabilities caused by early exits of visual language models at different image encoding layers.
Layer-wise Quantization for Quantized Optimistic Dual Averaging
Anh Duc Nguyen (National University of Singapore), Volkan Cevher (École Polytechnique Fédérale de Lausanne)
GenerationOptimizationComputational EfficiencyTransformerGenerative Adversarial NetworkImageText
🎯 What it does: A hierarchical quantization framework was developed and applied to the distributed variational inequality solver QODA, achieving acceleration in GAN training.
LBI-FL: Low-Bit Integerized Federated Learning with Temporally Dynamic Bit-Width Allocation
Li Ding (Shanghai Jiao Tong University), Hongkai Xiong (Shanghai Jiao Tong University)
Federated LearningComputational EfficiencyConvolutional Neural NetworkTransformerReinforcement LearningImage
🎯 What it does: A low-bit integer quantized federated learning framework (LBI-FL) is proposed, which significantly reduces communication and computational overhead in federated learning by quantizing weights, activations, and gradients to INT4/6/8 and dynamically allocating bit width during the training process using reinforcement learning.
LDMol: A Text-to-Molecule Diffusion Model with Structurally Informative Latent Space Surpasses AR Models
Jinho Chang (Korea Advanced Institute of Science and Technology), Jong Chul Ye (Korea Advanced Institute of Science and Technology)
GenerationData SynthesisRetrievalDrug DiscoveryTransformerDiffusion modelAuto EncoderContrastive LearningTextGraph
🎯 What it does: We propose LDMol, a latent diffusion model for text-to-molecule generation, which utilizes a structure-aware SMILES encoder and a DiT diffusion network to achieve high-quality text-conditioned molecule generation.
Lean and Mean Adaptive Optimization via Subset-Norm and Subspace-Momentum with Convergence Guarantees
Thien Hang Nguyen (Northeastern University), Huy Nguyen
OptimizationLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes two novel optimizer state compression methods: Subset-Norm (SN) significantly reduces the storage of the second moment matrix by sharing an AdaGrad-Norm style adaptive learning rate across subsets of parameters; Subspace-Momentum (SM) restricts momentum to a low-dimensional subspace while performing SGD directly in the orthogonal complement space, thereby compressing the momentum state.
LEAPS: A discrete neural sampler via locally equivariant networks
Peter Holderrieth (Massachusetts Institute of Technology), Tommi Jaakkola (Massachusetts Institute of Technology)
Reinforcement LearningGraphPhysics Related
🎯 What it does: An algorithm named LEAPS has been developed, which implements efficient sampling from discrete distributions using continuous-time Markov chains and local equivariant networks.
Learn Beneficial Noise as Graph Augmentation
Siqi Huang (Northwestern Polytechnical University), Xuelong Li (China Telecom)
OptimizationRepresentation LearningGraph Neural NetworkContrastive LearningGraph
🎯 What it does: This paper proposes a graph data augmentation method called PiNGDA based on positive incentive noise (π-noise), which utilizes a learnable noise generator to adaptively perturb graph topology and node attributes, thereby stably enhancing the effectiveness of graph contrastive learning (GCL).
Learn from Downstream and Be Yourself in Multimodal Large Language Models Fine-Tuning
Wenke Huang (Wuhan University), Mang Ye (Wuhan University)
TransformerLarge Language ModelSupervised Fine-TuningTextMultimodality
🎯 What it does: This study investigates the problem of catastrophic forgetting in downstream fine-tuning of multimodal large language models and proposes a SPIDER method based on importance difference assessment.
Learn Singularly Perturbed Solutions via Homotopy Dynamics
Chuqi CHEN, Wenrui Hao (Pennsylvania State University)
Neural Radiance FieldPhysics RelatedOrdinary Differential Equation
🎯 What it does: This paper addresses the solution of singularly perturbed partial differential equations (such as Allen-Cahn, Helmholtz, Burgers, etc.) using neural networks, proposing a training method based on homotopy dynamics that maintains the stability and accuracy of network training as the parameter ε is gradually reduced.
Learn to Vaccinate: Combining Structure Learning and Effective Vaccination for Epidemic and Outbreak Control
Sepehr Elahi (École Polytechnique Fédérale de Lausanne), Patrick Thiran (École Polytechnique Fédérale de Lausanne)
Graph
🎯 What it does: In the SIS epidemic model with an unknown contact network, a combination of network structure learning and one-time vaccination is proposed to minimize disease extinction time.
Learnable Spatial-Temporal Positional Encoding for Link Prediction
Katherine Tieu (University of Illinois Urbana-Champaign), Jingrui He (University of Illinois Urbana-Champaign)
Graph Neural NetworkGraphTime SeriesBenchmark
🎯 What it does: Proposes the L-STEP model, which utilizes learnable spatial-temporal position encoding and MLP for link prediction in temporal graphs.
Learngene Tells You How to Customize: Task-Aware Parameter Initialization at Flexible Scales
Jiaze Xu (Southeast University), Xin Geng (Southeast University)
TransformerSupervised Fine-TuningImageText
🎯 What it does: This paper proposes Task-Aware Learngene (TAL), a method for predicting parameter initialization based on task characteristics and model scale.
Learning Adaptive Lighting via Channel-Aware Guidance
Qirui Yang (Tianjin University), Jingyu Yang (Tianjin University)
Image TranslationImage HarmonizationRestorationImage
🎯 What it does: A multi-task learning adaptive lighting network LALNet is proposed, capable of simultaneously handling various lighting-related tasks such as exposure correction, image beautification, low-light enhancement, and HDR transformation.
Learning Adversarial MDPs with Stochastic Hard Constraints
Francesco Emanuele Stradi (Politecnico di Milano), Nicola Gatti (Politecnico di Milano)
OptimizationReinforcement Learning
🎯 What it does: This paper studies the online learning problem of adversarial Markov decision processes (CMDP) under random hard constraints and designs three types of algorithms: SV-OPS, S-OPS, and CV-OPS, achieving sublinear regret, per-round safety, and constant regret, respectively.
Learning Along the Arrow of Time: Hyperbolic Geometry for Backward-Compatible Representation Learning
Ngoc Bui (Yale University), Tong Zhao (Snap Inc)
Representation LearningContrastive LearningImage
🎯 What it does: This paper proposes a method for backward-compatible representation learning using hyperbolic geometry (HBCT), which allows for smooth model upgrades without the need to re-index the vector database.
Learning Attribute-Aware Hash Codes for Fine-Grained Image Retrieval via Query Optimization
Peng Wang (Nanjing University of Science and Technology), Xiu-Shen Wei (Southeast University)
RetrievalOptimizationTransformerImage
🎯 What it does: This paper proposes a fine-grained image retrieval method based on query optimization, utilizing learnable query vectors to decouple global features into attribute features, and compressing each attribute feature into binary hash bits to generate hash codes with attribute interpretability.
Learning Bayesian Nash Equilibrium in Auction Games via Approximate Best Response
Kexin Huang (University of Science and Technology of China), Xiang Wang (Independent Researcher)
OptimizationReinforcement LearningTabular
🎯 What it does: This paper proposes a learning framework based on approximate best response gradients to efficiently learn Bayesian Nash equilibria in continuous value auction games.
Learning Cascade Ranking as One Network
Yunli Wang (Kuaishou Technology), Kun Gai (Independent)
Recommendation SystemOptimizationTabularBenchmark
🎯 What it does: A unified end-to-end training framework LCRON is proposed to optimize the multi-stage collaboration and target mismatch issues in cascade ranking systems.
Learning Changes in Graphon Attachment Network Models
Xinyuan Fan (Tsinghua University), Weichi Wu (Tsinghua University)
Graph Neural NetworkGraphTime Series
🎯 What it does: A dynamic network model based on graphon subfunctions (Graphon Attachment Network Model, GAN-M) is proposed, and a new WESUM statistic is designed to learn the structural changes of the network over time (i.e., detect change points).
Learning Classifiers That Induce Markets
Yonatan Sommer (Technion - Israel Institute of Technology), Nir Rosenfeld (Technion - Israel Institute of Technology)
ClassificationOptimizationTabularFinance Related
🎯 What it does: The study investigates the scenario of using classifiers to incentivize users to purchase features in the market for positive predictions and proposes a market-aware learning framework.
Learning Compact Semantic Information for Incomplete Multi-View Missing Multi-Label Classification
Jie Wen (Harbin Institute of Technology), Chengliang Liu (Harbin Institute of Technology)
ClassificationMixture of ExpertsContrastive LearningMultimodality
🎯 What it does: This paper proposes a learning framework for incomplete multi-view multi-label classification (iM3C), aimed at simultaneously addressing the issues of missing views and missing labels.
Learning Condensed Graph via Differentiable Atom Mapping for Reaction Yield Prediction
Ankit Ghosh (Indian Institute of Technology Bombay), Abir De (Indian Institute of Technology Bombay)
Graph Neural NetworkTransformerGraph
🎯 What it does: Developed YIELDNET, which uses differentiable atomic mapping to approximate the generation of reaction concentration graphs, thereby predicting multi-step reaction yields.
Learning Configurations for Data-Driven Multi-Objective Optimization
Zhiyang Chen (Tsinghua University), Xia Yin (Tsinghua University)
OptimizationTabular
🎯 What it does: This study investigates the sample complexity of parameter configuration for multi-objective optimization algorithms, provides theoretical guarantees, and proposes a PAC learning method for Pareto volume using submodular functions. Subsequently, experimental validations are conducted on various algorithms including approximation algorithms, local search, and linear programming.
Learning Curves of Stochastic Gradient Descent in Kernel Regression
Haihan Zhang (Peking University), Cong Fang (Peking University)
Tabular
🎯 What it does: Analyzed the learning curves of single-channel SGD in kernel regression (especially inner product kernels/NTK), studying its convergence and optimality in high-dimensional (n≈d^γ) and asymptotic (n≫d) scenarios.