ICLR 2025 Papers — Page 18
International Conference on Learning Representations · 3704 papers
KAN: Kolmogorov–Arnold Networks
Ziming Liu (Massachusetts Institute of Technology), Max Tegmark (Massachusetts Institute of Technology)
Explainability and InterpretabilityComputational EfficiencyTabularSequentialPhysics Related
🎯 What it does: The Kolmogorov-Arnold Network (KAN) is proposed as an alternative to the Multi-Layer Perceptron (MLP), particularly in situations where interpretability is required.
KaSA: Knowledge-Aware Singular-Value Adaptation of Large Language Models
Fan Wang (Hong Kong University of Science and Technology), Jing Tang (Hong Kong University of Science and Technology)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes a new parameter-efficient fine-tuning method called KaSA, which utilizes knowledge-aware singular value decomposition to dynamically activate task-related model knowledge.
KBLaM: Knowledge Base augmented Language Model
Xi Wang (Johns Hopkins University), James Hensman (Microsoft Research)
TransformerLarge Language ModelSupervised Fine-TuningTextRetrieval-Augmented Generation
🎯 What it does: This study investigates a method to convert external knowledge bases into continuous key-value vectors without modifying the weights of the LLM, and injects these vectors into the LLM through rectangular attention, achieving scalable and dynamically updatable knowledge enhancement.
Kernel-based Optimally Weighted Conformal Time-Series Prediction
Jonghyeok Lee (Georgia Institute of Technology), Yao Xie (Georgia Institute of Technology)
OptimizationTime SeriesFinance Related
🎯 What it does: An adaptive weighted confidence interval method for time series, KOWCPI, is proposed, which constructs conditional confidence intervals using the reweighted Nadaraya-Watson (RNW) estimator for quantile regression.
KGARevion: An AI Agent for Knowledge-Intensive Biomedical QA
Xiaorui Su (Harvard University), Marinka Zitnik (Harvard University)
TransformerLarge Language ModelSupervised Fine-TuningAgentic AITextBiomedical DataBenchmark
🎯 What it does: This paper proposes a KG-based LLM agent—KGAREVION, which can first generate triples related to a question using LLM, then verify and correct these triples on a knowledge graph, ultimately providing reliable answers.
Kinetix: Investigating the Training of General Agents through Open-Ended Physics-Based Control Tasks
Michael Matthews (FLAIR University of Oxford), Jakob Nicolaus Foerster
Robotic IntelligenceTransformerReinforcement LearningAgentic AISequentialPhysics Related
🎯 What it does: This paper presents Kinetix—a 2D physics simulation framework based on Jax2D, which trains general RL agents using tens of millions of procedurally generated tasks, achieving zero-shot generalization and fine-tuning improvements on 74 manually designed benchmark tasks.
KinFormer: Generalizable Dynamical Symbolic Regression for Catalytic Organic Reaction Kinetics
Jindou Chen (Shanghai Jiao Tong University), Yanyan Xu (Shanghai Jiao Tong University)
TransformerTime SeriesPhysics Related
🎯 What it does: This paper presents KinFormer, a framework capable of generalizable predictions of the kinetic equations of catalytic organic reactions through conditional Transformers and Monte Carlo Tree Search.
KinPFN: Bayesian Approximation of RNA Folding Kinetics using Prior-Data Fitted Networks
Dominik Scheuer (University of Freiburg), Frank Hutter (University of Freiburg)
OptimizationComputational EfficiencyNeural Architecture SearchTransformerSequentialBiomedical Data
🎯 What it does: A deep learning framework based on the Prior Data Fitting Network (KinPFN) has been developed for the rapid approximation of the cumulative distribution function (CDF) of the first passage time (FPT) of RNA folding.
KiVA: Kid-inspired Visual Analogies for Testing Large Multimodal Models
Eunice Yiu (University of California), Kate Saenko (Boston University)
RecognitionGenerationTransformerLarge Language ModelPrompt EngineeringVision Language ModelImageMultimodalityBenchmark
🎯 What it does: A visual analogy benchmark called KiVA based on real everyday objects is proposed, and it is used to evaluate the capabilities of large multimodal models (LMM) in visual analogy reasoning, comparing their performance with that of children aged three to five and adults.
KLay: Accelerating Arithmetic Circuits for Neurosymbolic AI
Jaron Maene (KU Leuven), Pedro Zuidberg Dos Martires (Örebro University)
Computational EfficiencyGraph
🎯 What it does: This paper proposes KLAY, a data structure for arithmetic circuits based on layered indexing and hashing, which significantly accelerates the backward and forward propagation of neural symbolic AI.
kNN Attention Demystified: A Theoretical Exploration for Scalable Transformers
Themistoklis Haris (Boston University)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes a theoretical framework to explain and implement the approximation of k-NN attention, providing forward and backward computation algorithms with sub-quadratic time complexity, and validating its feasibility through experiments on random matrices, character-level Shakespeare datasets, and fine-tuning experiments with GPT-2 XL 4.
Knowing Your Target: Target-Aware Transformer Makes Better Spatio-Temporal Video Grounding
Xin Gu (University of Chinese Academy of Sciences), Libo Zhang (Institute of Software Chinese Academy of Sciences)
RecognitionObject DetectionTransformerVideoText
🎯 What it does: This paper proposes a Target-Aware Spatial-Temporal Video Grounding (TA-STVG) model that enhances spatiotemporal video localization accuracy by directly utilizing target features from both video and text to generate object queries.
Knowledge Distillation with Multi-granularity Mixture of Priors for Image Super-Resolution
Simiao Li (Huawei Noah's Ark Lab), Jie Hu (Huawei Noah's Ark Lab)
RestorationSuper ResolutionKnowledge DistillationConvolutional Neural NetworkTransformerImage
🎯 What it does: A multi-granularity mixed prior knowledge distillation (MiPKD) framework is proposed to efficiently distill prior information from teachers and students simultaneously at the feature and block levels for image super-resolution models.
Knowledge Entropy Decay during Language Model Pretraining Hinders New Knowledge Acquisition
Jiyeon Kim (Korea Advanced Institute of Science and Technology), Minjoon Seo (Korea Advanced Institute of Science and Technology)
TransformerLarge Language ModelTextBiomedical Data
🎯 What it does: This study investigates the changes in knowledge entropy during the pre-training process of large language models and its impact on the acquisition of new knowledge and forgetting in continual learning, proposing to enhance the model's plasticity by activating inactive memory vectors.
Knowledge Graph Finetuning Enhances Knowledge Manipulation in Large Language Models
Hanzhu Chen (University of Science and Technology of China), Jieping Ye (Independent Researcher)
GenerationKnowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningTextBiomedical Data
🎯 What it does: Through the knowledge graph-driven supervised fine-tuning (KG-SFT) framework, high-quality and logically coherent explanatory texts are automatically generated on existing Q&A pairs to enhance the knowledge understanding and reasoning capabilities of LLMs.
Knowledge Localization: Mission Not Accomplished? Enter Query Localization!
Yuheng Chen (Chinese Academy of Sciences), Jun Zhao (Chinese Academy of Sciences)
TransformerLarge Language ModelText
🎯 What it does: This paper re-examines the knowledge storage mechanism in large language models, finding that the Knowledge Localization (KL) hypothesis is largely invalid, and proposes the Query Localization (QL) hypothesis, which includes two components: query-knowledge neuron mapping and dynamic knowledge neuron selection; subsequently, a consistency-aware method for modifying knowledge neurons is designed based on the QL hypothesis.
Kolmogorov-Arnold Transformer
Xingyi Yang (National University of Singapore), Xinchao Wang (National University of Singapore)
ClassificationObject DetectionSegmentationTransformerImage
🎯 What it does: Replace the MLP layer in the Transformer with a Kolmogorov-Arnold Network (KAN) and propose a scalable Kolmogorov-Arnold Transformer (KAT) structure.
KooNPro: A Variance-Aware Koopman Probabilistic Model Enhanced by Neural Process for Time Series Forecasting
Ronghua Zheng (Fudan University), Weiyang Ding (Fudan University)
Anomaly DetectionTime Series
🎯 What it does: A probabilistic time series forecasting framework KooNPro is proposed, which combines the variational Koopman model with Neural Process to capture local dynamics using variance-aware continuous spectra (pseudo-spectra) and global dynamics using Neural Process.
KOR-Bench: Benchmarking Language Models on Knowledge-Orthogonal Reasoning Tasks
Kaijing Ma (Tongji University), Ge Zhang
TransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: The concept of Knowledge-Orthogonal Reasoning (KOR) is proposed, and the KOR-Bench benchmark is constructed, which includes 5 categories of rule-driven reasoning tasks (Operation, Logic, Cipher, Puzzle, Counterfactual).
Kronecker Mask and Interpretive Prompts are Language-Action Video Learners
Yang JingYi, Hui Li (University of Science and Technology of China)
ClassificationRecognitionTransformerLarge Language ModelPrompt EngineeringContrastive LearningVideo
🎯 What it does: In the video action recognition task, the CLAVER model is proposed, which realigns CLIP to action behaviors and verbs through Kronecker mask temporal attention and interpretive prompts.
L-WISE: Boosting Human Visual Category Learning Through Model-Based Image Selection and Enhancement
Morgan Bruce Talbot, Guy Gaziv (Massachusetts Institute of Technology)
ClassificationRecognitionOptimizationConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: Using a robust artificial neural network (ANN) to calculate the ground truth logit of images to predict human recognition difficulty, and utilizing gradient ascent to generate class-enhanced images, thereby improving human classification accuracy and learning speed in new category visual learning.
L3Ms — Lagrange Large Language Models
Guneet S. Dhillon (University of Oxford), Alex Smola (Boson AI)
OptimizationTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
🎯 What it does: This paper proposes a method that unifies supervised fine-tuning and alignment into a constrained optimization framework, and based on this, trains L3M (Lagrange Large Language Models) to meet application-specific constraints.
LaGeM: A Large Geometry Model for 3D Representation Learning and Diffusion
Biao Zhang (King Abdullah University of Science and Technology), Peter Wonka (King Abdullah University of Science and Technology)
GenerationCompressionRepresentation LearningTransformerDiffusion modelAuto EncoderPoint CloudMesh
🎯 What it does: A hierarchical vector set autoencoder (LaGeM) is proposed for efficient compression and reconstruction of large-scale 3D models, and cascade diffusion generation is achieved in this latent space.
Lambda-Skip Connections: the architectural component that prevents Rank Collapse
Federico Arangath Joseph (ETH Zurich), Carmen Amo Alonso (Stanford University)
TransformerText
🎯 What it does: The paper proposes lambda-skip connections to control the strength of skip connections, thereby avoiding the 'rank collapse' phenomenon in Transformers and SSMs (including LTI and selective SSM), and provides sufficient conditions and experimental validation.
LaMP: Language-Motion Pretraining for Motion Generation, Retrieval, and Captioning
Zhe Li (Huazhong University of Science and Technology), Laurence Tianruo Yang
GenerationRetrievalTransformerContrastive LearningVideoText
🎯 What it does: A language-motion pre-training model LaMP is proposed, which improves three major tasks: text-to-motion generation, motion-to-text retrieval, and motion captioning.
LaMPlace: Learning to Optimize Cross-Stage Metrics in Macro Placement
Zijie Geng (University of Science and Technology of China), Feng Wu (University of Science and Technology of China)
OptimizationGraph Neural NetworkTabularBenchmark
🎯 What it does: This paper proposes a macro placement method called LaMPlace, which utilizes a Laurent polynomial predictor to generate learnable masks (L-Mask) and directly optimizes cross-stage metrics (such as WNS, TNS, etc.) instead of intermediate proxy metrics.
LancBiO: Dynamic Lanczos-aided Bilevel Optimization via Krylov Subspace
Yan Yang (Academy of Mathematics and Systems Science Chinese Academy of Sciences University of Chinese Academy of Sciences), Ya-xiang Yuan (Academy of Mathematics and Systems Science Chinese Academy of Sciences)
OptimizationHyperparameter Search
🎯 What it does: A new bi-level optimization framework called LancBiO is proposed, which utilizes Krylov subspace and the Lanczos process to dynamically construct a low-dimensional subspace for efficiently approximating the Hessian inverse-vector product, thereby improving the accuracy of the hypergradient estimation.
Langevin Soft Actor-Critic: Efficient Exploration through Uncertainty-Driven Critic Learning
Haque Ishfaq (Mila), Doina Precup (Mila)
Robotic IntelligenceReinforcement LearningDiffusion modelSequential
🎯 What it does: A new model-free continuous control reinforcement learning algorithm LSAC is proposed, which enhances critic learning by approximating Thompson sampling through Langevin Monte Carlo sampling, thereby achieving efficient exploration and improving sample efficiency.
Language Agents Meet Causality -- Bridging LLMs and Causal World Models
John Gkountouras (University of Amsterdam), Ivan Titov (University of Edinburgh)
Robotic IntelligenceTransformerLarge Language ModelAuto EncoderWorld ModelImageText
🎯 What it does: A framework that combines Causal Representation Learning (CRL) with Large Language Models (LLM) for causal reasoning and planning in interactive environments is proposed.
Language Guided Skill Discovery
Seungeun Rho (Georgia Institute of Technology), Sehoon Ha (Georgia Institute of Technology)
Robotic IntelligenceTransformerLarge Language ModelReinforcement LearningPrompt EngineeringSequential
🎯 What it does: This paper proposes a language-guided skill discovery framework (LGSD) based on large language models (LLMs), which generates state descriptions using LLMs and measures semantic diversity with language distance, constraining the search space with language prompts to train diversified low-level skills for downstream tasks.
Language Imbalance Driven Rewarding for Multilingual Self-improving
Wen Yang (University of Chinese Academy of Sciences), Jiajun Zhang
TransformerLarge Language ModelReinforcement LearningText
🎯 What it does: A reward mechanism based on language imbalance is proposed, utilizing the performance differences of LLMs in dominant and low-resource languages to autonomously generate translation preference pairs and achieve self-improvement in multilingual settings through iterative DPO fine-tuning.
Language Model Alignment in Multilingual Trolley Problems
Zhijing Jin (University of Toronto), Bernhard Schölkopf (Max Planck Institute for Intelligent Systems)
TransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper constructs a cross-linguistic dataset for the trolley problem, MULTITP, to evaluate the moral alignment of 19 large language models across more than 100 languages.
Language Models are Advanced Anonymizers
Robin Staab (ETH Zurich), Martin Vechev (ETH Zurich)
Safty and PrivacyTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: A feedback-guided adversarial anonymization framework based on large language models (LLMs) has been developed, allowing the LLM to first infer personal attributes in the text during multiple rounds of iteration, and then another LLM actively deletes or obscures the text based on the inference results, achieving efficient and highly readable anonymization.
Language Models Are Implicitly Continuous
Samuele Marro (University of Oxford), Michael J. Wooldridge
TransformerLarge Language ModelText
🎯 What it does: This paper views large-scale language models (LLMs) as implicit continuous-time functions and proposes a Continuous Causal Transformer (CCT) framework that does not alter the original weights. The framework is experimentally validated on trained LLMs to assess the model's perception of temporal continuity and spatial interpolation.
Language Models Learn to Mislead Humans via RLHF
Jiaxin Wen (Tsinghua University), Shi Feng (George Washington University)
AI Code AssistantReinforcement Learning from Human FeedbackReinforcement LearningText
🎯 What it does: This paper empirically studies the behavior of models trained with RLHF that mislead human evaluators without being deliberately induced (referred to as U-SOPHISTRY) through experiments on two major tasks: long-form question answering and programming.
Language Models Need Inductive Biases to Count Inductively
Yingshan Chang (Carnegie Mellon University), Yonatan Bisk (Carnegie Mellon University)
Recurrent Neural NetworkTransformerSequential
🎯 What it does: This study investigates the learning and reasoning capabilities of language models in counting tasks, particularly in out-of-distribution (OOD) scenarios with inconsistent training lengths and vocabularies; it compares the performance of Transformer models with different position encodings (PE) and various RNN variants by designing multiple counting tasks (vanilla, helper token, shifted start, modular, selective, etc.).
Language models scale reliably with over-training and on downstream tasks
Samir Yitzhak Gadre (Columbia University), Ludwig Schmidt (University of Washington)
TransformerLarge Language ModelText
🎯 What it does: This study investigates the scaling laws of language models under overtraining (beyond compute-optimal) and establishes a power-law relationship between loss and the average error of downstream tasks, thereby enabling reliable predictions of large-scale, overtrained models and their downstream performance.
Language Models Trained to do Arithmetic Predict Human Risky and Intertemporal Choice
Jian-Qiao Zhu (Princeton University), Thomas L. Griffiths (Princeton University)
TransformerLarge Language ModelText
🎯 What it does: Train a GPT model with approximately 10M parameters (Arithmetic-GPT) on arithmetic problems generated from ecological distributions of probabilities and values, and then extract the model's embeddings to predict human behavior in risk decision-making and temporal discounting choices.
Language Representations Can be What Recommenders Need: Findings and Potentials
Leheng Sheng (National University of Singapore), Tat-Seng Chua (National University of Singapore)
Recommendation SystemGraph Neural NetworkTransformerLarge Language ModelContrastive LearningText
🎯 What it does: This paper explores and verifies whether the representations in pre-trained language models (LM) contain user preference information through linear mapping and the construction of the AlphaRec model, and directly uses LM representations for collaborative filtering recommendations.
Language-Assisted Feature Transformation for Anomaly Detection
EungGu Yun (SAIGE), Bryan Dongik Lee
Anomaly DetectionVision Language ModelContrastive LearningImage
🎯 What it does: The LAFT (Language-Assisted Feature Transformation) method is proposed, which utilizes the shared text-image embedding space of CLIP to transform image features through natural language, thereby achieving controllable adjustments to the 'normal' boundaries in anomaly detection.
Language-Image Models with 3D Understanding
Jang Hyun Cho (University of Texas at Austin), Marco Pavone (NVIDIA)
Object DetectionAutonomous DrivingTransformerLarge Language ModelPrompt EngineeringVision Language ModelImageMultimodalityChain-of-Thought
🎯 What it does: A large-scale multimodal language model, CUBE-LLM, has been proposed and trained to simultaneously handle 2D and 3D visual language tasks. A unified multi-scale dataset, LV3D, has been constructed, and 3D perception and reasoning have been achieved through data augmentation alone, without specialized 3D networks.
LANTERN: Accelerating Visual Autoregressive Models with Relaxed Speculative Decoding
Doohyuk Jang (Korea Advanced Institute of Science and Technology), Eunho Yang (Korea Advanced Institute of Science and Technology)
GenerationComputational EfficiencyTransformerLarge Language ModelImageText
🎯 What it does: This study explores how to combine visual autoregressive models like LlamaGen with the inference acceleration technology of EAGLE-2, achieving significant inference speed improvements by introducing a relaxation of acceptable conditions (LANTERN).
Laplace Sample Information: Data Informativeness Through a Bayesian Lens
Johannes Kaiser (Technical University of Munich), Georgios Kaissis (Technical University of Munich)
ClassificationAnomaly DetectionData-Centric LearningContrastive LearningImageTextMultimodality
🎯 What it does: A Bayesian sample information metric LSI based on Laplace approximation is proposed to evaluate the contribution of each sample in the training set to the model parameters.
Large (Vision) Language Models are Unsupervised In-Context Learners
Artyom Gadetsky (Swiss Federal Institute of Technology), Maria Brbic
TransformerLarge Language ModelReinforcement LearningVision Language ModelImageTextMultimodality
🎯 What it does: Proposes a joint reasoning framework to achieve unsupervised task adaptation.
Large Convolutional Model Tuning via Filter Subspace
Wei Chen (Purdue University), Qiang Qiu (Purdue University)
ClassificationGenerationConvolutional Neural NetworkDiffusion modelImage
🎯 What it does: This paper proposes a parameter-efficient fine-tuning method based on convolutional filter subspaces, adjusting only the filter atoms while keeping the channel mixing coefficients unchanged, thereby adapting to downstream tasks while maintaining the prior capabilities of large models.
Large Language Models are Interpretable Learners
Ruochen Wang (University of California Los Angeles), Inderjit S Dhillon
Explainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringTextMultimodalityBenchmark
🎯 What it does: Combining large language models (LLM) with symbolic programming to construct interpretable and expressive predictive models (LSP), gradually building interpretable rules through prompt learning and a decision tree-like structure;
Large Language Models Assume People are More Rational than We Really are
Ryan Liu (Princeton University), Thomas L. Griffiths (Princeton University)
TransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: This paper systematically evaluates the implicit assumptions of large language models (LLMs) regarding human decision-making behavior through two types of psychological experiments: risk selection and reverse inference. It finds that LLMs generally perceive humans as more rational than they actually are and exhibit a high similarity to humans in inferring others' preferences.
Large Language Models can Become Strong Self-Detoxifiers
Ching-Yun Ko (IBM Research), Luca Daniel (Massachusetts Institute of Technology)
TransformerLarge Language ModelText
🎯 What it does: Proposed the SASA self-discipline autoregressive sampling method, using internal representations to achieve LLM detoxification.
Large Language Models Meet Symbolic Provers for Logical Reasoning Evaluation
Chengwen Qi (Beihang University), Conghui He (Shanghai Artificial Intelligence Laboratory)
TransformerLarge Language ModelSupervised Fine-TuningTextChain-of-Thought
🎯 What it does: ProverGen framework is proposed, which combines LLM and symbolic provers to automatically generate a high-quality FOL reasoning dataset called ProverQA.
Large Language Models Often Say One Thing and Do Another
Ruoxi Xu (University of Chinese Academy of Sciences), Yingfei Sun (University of Chinese Academy of Sciences)
Large Language ModelSupervised Fine-TuningTextBenchmarkChain-of-Thought
🎯 What it does: A benchmark called 'Word and Behavior Consistency Test' (WDCT) is proposed and implemented to evaluate the consistency between the statements and actual behaviors of large language models in four domains: opinions, non-ethical values, ethical values, and theory. The causal relationship between word and behavior consistency is explored through alignment experiments.
Large Scale Knowledge Washing
Yu Wang (University of California San Diego), Julian McAuley (University of Illinois Urbana-Champaign)
TransformerLarge Language ModelText
🎯 What it does: Proposes a large-scale knowledge washing problem and designs an optimization objective on the MLP layer of LLMs to achieve seamless deletion of specified knowledge while maintaining reasoning ability.
Large-scale and Fine-grained Vision-language Pre-training for Enhanced CT Image Understanding
Zhongyi Shui (DAMO Academy, Alibaba Group), Ling Zhang (DAMO Academy, Alibaba Group)
ClassificationSegmentationGenerationTransformerVision Language ModelContrastive LearningImageTextMultimodalityComputed Tomography
🎯 What it does: This paper proposes a fine-grained visual-language pre-training model fVLM, which achieves precise alignment and pre-training of CT images and reports at the anatomical level through anatomical structure segmentation and report decomposition.
LARP: Tokenizing Videos with a Learned Autoregressive Generative Prior
Hanyu Wang (University of Maryland), Abhinav Shrivastava (University of Maryland)
GenerationData SynthesisCompressionTransformerLarge Language ModelVideo
🎯 What it does: A novel video tokenizer called LARP has been developed, combined with a lightweight AR prior model to achieve high-quality autoregressive modeling in video generation tasks.
LASER: A Neuro-Symbolic Framework for Learning Spatio-Temporal Scene Graphs with Weak Supervision
Jiani Huang (University of Pennsylvania), Ser-Nam Lim (University of Central Florida)
RecognitionObject DetectionGraph Neural NetworkLarge Language ModelVideoText
🎯 What it does: This paper proposes the LASER framework, which utilizes video subtitles as weak supervision signals to train a spatiotemporal scene graph (STSG) generation model.
LASeR: Towards Diversified and Generalizable Robot Design with Large Language Models
Junru Song (Shanghai Jiao Tong University), Feifei Wang (Renmin University of China)
Robotic IntelligenceTransformerLarge Language ModelReinforcement LearningPrompt EngineeringMultimodality
🎯 What it does: This paper studies a large language model (LLM)-driven evolutionary search framework called LASeR for the automated design of soft robots (VSR);
Lasso Bandit with Compatibility Condition on Optimal Arm
Harin Lee (Seoul National University), Min-hwan Oh (Seoul National University)
Reinforcement LearningTabular
🎯 What it does: This paper studies sparse linear contextual bandits and proposes a new Forced-Sampling + Weighted-Loss Lasso (FS-WLasso) algorithm under the premise of unknown parameter sparsity, utilizing Lasso estimation and selecting arms based on estimated values during the greedy phase.
Last Iterate Convergence of Incremental Methods as a Model of Forgetting
Xufeng Cai (University of Wisconsin Madison), Jelena Diakonikolas (University of Wisconsin Madison)
Optimization
🎯 What it does: This paper studies the final iteration convergence of incremental gradient and incremental proximal methods, establishing for the first time non-asymptotic convergence guarantees for the final iteration under general convex smooth and convex Lipschitz settings.
Last-Iterate Convergence Properties of Regret-Matching Algorithms in Games
Yang Cai (Yale University), Weiqiang Zheng (Yale University)
OptimizationReinforcement Learning from Human FeedbackTabular
🎯 What it does: This paper studies the convergence properties of the last iteration based on Regret Matching+ (RM+) and its variants in two-player zero-sum matrix games. It proves that traditional RM+, alternating RM+, and Predictive RM+ do not converge in certain games, and proposes Extragradient RM+ (ExRM+) and Smooth Predictive RM+ (SPRM+) that can guarantee last iteration convergence, further designing a restart mechanism to achieve linear convergence.
Latent Action Pretraining from Videos
Seonghyeon Ye (KAIST), Minjoon Seo (KAIST)
Robotic IntelligenceVision Language ModelVideo
🎯 What it does: An unsupervised method called 'Latent Action Pretraining (LAPA)' is proposed, which learns discretized latent actions from raw frames in videos, and then predicts these latent actions using a large visual-language model, ultimately fine-tuning them to map to real robot actions, achieving a foundational robot model without the need for manually annotated actions.
Latent Bayesian Optimization via Autoregressive Normalizing Flows
Seunghun Lee (Korea University), Hyunwoo J. Kim (KAIST)
OptimizationDrug DiscoveryFlow-based ModelSequential
🎯 What it does: In the discrete sequence optimization of molecular design, a Bayesian optimization framework based on normalized flow, NF-BO, is proposed. It utilizes reversible flow to achieve a one-to-one mapping between the input and latent space, combined with Token-level Adaptive Candidate Sampling (TACS) for efficient exploration.
Latent Radiance Fields with 3D-aware 2D Representations
Chaoyi Zhou (Clemson University), Siyu Huang (Clemson University)
GenerationData SynthesisNeural Radiance FieldAuto EncoderImage
🎯 What it does: Construct and render a light field in the two-dimensional latent space of the Variational Autoencoder (VAE) to achieve high-quality, unbounded 3D reconstruction and view synthesis of outdoor scenes.
Latent Safety-Constrained Policy Approach for Safe Offline Reinforcement Learning
Prajwal Koirala (Iowa State University), Cody Fleming (Iowa State University)
Safty and PrivacyReinforcement LearningAuto EncoderTabular
🎯 What it does: This paper proposes an offline safe reinforcement learning framework called LSPC, which maximizes cumulative rewards while ensuring safety.
Latent Space Chain-of-Embedding Enables Output-free LLM Self-Evaluation
Yiming Wang (Shanghai Jiao Tong University), Rui Wang (Shanghai Jiao Tong University)
TransformerLarge Language ModelText
🎯 What it does: This paper proposes the Chain-of-Embedding (CoE) method, achieving output-free and label-free self-evaluation of LLMs by directly utilizing all hidden states during the model inference process.
Latent-EnSF: A Latent Ensemble Score Filter for High-Dimensional Data Assimilation with Sparse Observation Data
Phillip Si (Georgia Institute of Technology), Peng Chen (Georgia Institute of Technology)
OptimizationComputational EfficiencyConvolutional Neural NetworkAuto EncoderTime SeriesStochastic Differential Equation
🎯 What it does: A new data assimilation method called Latent-EnSF is proposed, which utilizes a coupled variational autoencoder to map high-dimensional states and sparse observations to a consistent low-dimensional latent space, and performs Bayesian filtering in that space using the Ensemble Score Filter;
Law of the Weakest Link: Cross Capabilities of Large Language Models
Ming Zhong (University of Illinois Urbana-Champaign), Laurens van der Maaten (Meta)
TransformerLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: This study investigates the performance of large language models (LLMs) in cross-capabilities and constructs the CROSSEVAL benchmark, systematically defining and evaluating seven individual capabilities and their seven common cross combinations.
Lawma: The Power of Specialization for Legal Annotation
Ricardo Dominguez-Olmedo (Max Planck Institute for Intelligent Systems), Michael Livermore (University of Virginia School of Law)
ClassificationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringTextBenchmarkChain-of-Thought
🎯 What it does: A legal text classification task set, CaselawQA, consisting of 260 tasks based on data from the U.S. Supreme Court and appellate courts, was constructed, and the performance of large language models and the self-developed Lawma series models was systematically evaluated.
Layer Swapping for Zero-Shot Cross-Lingual Transfer in Large Language Models
Lucas Bandarkar (University of California, Los Angeles), Bing Liu (Meta GenAI)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: In the absence of mathematical data in the target language, we first fine-tune a 'mathematics expert' and a 'language expert' separately on English mathematical data and general instruction data in the target language, and then achieve cross-lingual zero-shot transfer through hierarchical swapping (replacing the top and bottom Transformer layers of the language expert into the mathematics expert).
LayerDAG: A Layerwise Autoregressive Diffusion Model for Directed Acyclic Graph Generation
Mufei Li (Georgia Institute of Technology), Pan Li (Georgia Institute of Technology)
GenerationData SynthesisGraph Neural NetworkDiffusion modelGraph
🎯 What it does: LayerDAG constructs a layer-wise autoregressive diffusion model to generate DAGs by treating the DAG as a hierarchical sequence of bipartite graphs.
Layerwise Recurrent Router for Mixture-of-Experts
Zihan Qiu (Alibaba Group), Jie Fu (Shanghai AI Lab)
OptimizationComputational EfficiencyRecurrent Neural NetworkTransformerSupervised Fine-TuningMixture of ExpertsTextSequential
🎯 What it does: This paper proposes a hierarchical recursive router RMoE, which utilizes GRU to share routing information between different Transformer layers, thereby improving the parameter efficiency and overall performance of Mixture-of-Experts.
Layout-your-3D: Controllable and Precise 3D Generation with 2D Blueprint
Junwei Zhou, Ming-Hsuan Yang
GenerationOptimizationDiffusion modelGaussian SplattingImage
🎯 What it does: Proposes the Layout-Your-3D framework, enabling controllable and detailed 3D scene generation based on 2D layouts.
LDAdam: Adaptive Optimization from Low-Dimensional Gradient Statistics
Thomas Robert (Institut Polytechnique de Paris), Dan Alistarh (Institute of Science and Technology Austria)
OptimizationTransformerLarge Language ModelText
🎯 What it does: An efficient memory adaptive optimizer LDAdam is proposed, which can perform Adam-level optimization in low-dimensional subspaces while continuously exploring the full parameter space during training.
Lean-STaR: Learning to Interleave Thinking and Proving
Haohan Lin (Institute for Interdisciplinary Information Sciences), Yiming Yang (Language Technologies Institute)
TransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextChain-of-Thought
🎯 What it does: By generating natural language thoughts for each step tactic in Lean interactive theorem proving to train language models, thereby enhancing automated theorem proving capabilities.
LeanAgent: Lifelong Learning for Formal Theorem Proving
Adarsh Kumarappan (California Institute of Technology), Anima Anandkumar (University of Wisconsin)
TransformerLarge Language ModelRetrieval-Augmented Generation
🎯 What it does: This paper presents LeanAgent, a lifelong learning framework for formal theorem proving that can continuously expand knowledge across multiple mathematical libraries and generate new formal proofs.
LeanQuant: Accurate and Scalable Large Language Model Quantization with Loss-error-aware Grid
Tianyi Zhang (Rice University), Anshumali Shrivastava (Rice University)
TransformerLarge Language ModelText
🎯 What it does: A low-bit quantization method named LeanQuant is proposed, which maintains higher accuracy in post-training quantization of LLMs by utilizing a loss error-aware quantization grid (non-uniform and adaptively uniform) and achieves scalable acceleration through the integration of GPU cores.
Learn hybrid prototypes for multivariate time series anomaly detection
Ke-Yuan Shen (Hebei University)
Anomaly DetectionTransformerTime Series
🎯 What it does: A multivariate time series anomaly detection model H-PAD is proposed, which integrates different scale patch prototypes and periodic prototypes;
Learn Your Reference Model for Real Good Alignment
Alexey Gorbatovski (T-Tech), Daniil Gavrilov (T-Tech)
OptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: In the offline alignment method, the reward over-optimization problem is alleviated by dynamically updating the reference model (Trust Region), and three improved methods are proposed: TR-DPO, TR-IPO, and TR-KTO.
Learn-by-interact: A Data-Centric Framework For Self-Adaptive Agents in Realistic Environments
Hongjin SU, Sercan O Arik
Data-Centric LearningRobotic IntelligenceTransformerLarge Language ModelAgentic AITextRetrieval-Augmented Generation
🎯 What it does: Proposes the LEARN-BY-INTERACT framework, enabling LLM agents to adapt in new environments without manual annotations;
Learnable Expansion of Graph Operators for Multi-Modal Feature Fusion
Dexuan Ding (Australian National University), Piotr Koniusz (Data61 CSIRO)
Anomaly DetectionGraph Neural NetworkVideoMultimodality
🎯 What it does: A learnable graph expansion fusion framework EGO based on relational graphs is proposed to fuse multi-modal, multi-representation, and multi-domain video features.
Learned Reference-based Diffusion Sampler for multi-modal distributions
Maxence Noble (Centre for Mathematical Analysis and its Applications), Alain Oliviero Durmus (Centre for Mathematical Analysis and its Applications)
OptimizationDiffusion modelMultimodalityStochastic Differential Equation
🎯 What it does: A new sampling framework is proposed - Learned Reference-based Diffusion Sampler (LRDS) - for sampling from multimodal target distributions. This framework learns a reference process that is structurally similar to the target distribution (which can be a Gaussian mixture model or an energy-based model) and performs variational optimization based on this.
Learning 3D Perception from Others' Predictions
Jinsu Yoo (Ohio State University), Wei-Lun Chao (Ohio State University)
Object DetectionAutonomous DrivingPoint Cloud
🎯 What it does: This paper proposes a framework that uses nearby vehicle detection predictions as pseudo-labels to train a new 3D object detector.
Learning a Fast Mixing Exogenous Block MDP using a Single Trajectory
Alexander Levine (University of Texas at Austin), Amy Zhang (University of Texas at Austin)
Representation LearningReinforcement LearningTabular
🎯 What it does: The STEEL algorithm is proposed, achieving unsupervised representation learning in a single continuous trajectory, non-resetting Ex-BMDP environment, and providing an upper bound on sample complexity.
Learning a Neural Solver for Parametric PDEs to Enhance Physics-Informed Methods
Lise Le Boudec (Sorbonne Université), Patrick Gallinari (Sorbonne Université)
OptimizationComputational EfficiencyTime SeriesPhysics Related
🎯 What it does: A physics-informed neural iterative solver is proposed to quickly solve parameterized partial differential equations (PDEs), achieving convergence solely through PDE parameters during inference.
Learning and aligning single-neuron invariance manifolds in visual cortex
Mohammad Bashiri (Noselab GmbH), Fabian H. Sinz (University of Göttingen)
Representation LearningImage
🎯 What it does: This paper proposes a method that combines deep neural networks, implicit neural representations (INR), and affine transformations to learn and align the continuous invariance manifold of individual neurons in the visual cortex, thereby clustering the functions of neural populations.
Learning Causal Alignment for Reliable Disease Diagnosis
Mingzhou Liu (Peking University), Yizhou Wang (Peking University)
ClassificationOptimizationConvolutional Neural NetworkGenerative Adversarial NetworkImageBiomedical DataMagnetic Resonance Imaging
🎯 What it does: A medical diagnosis framework based on causal alignment is proposed, utilizing counterfactual generation to identify the causal chain of model decisions, and employing causal alignment loss to focus the model on diagnostic factors consistent with radiologists.
Learning Chaos In A Linear Way
Xiaoyuan Cheng (University College London), Yukun Hu (University College London)
Auto EncoderTime SeriesPhysics Related
🎯 What it does: A new neural operator framework PFNN is designed and validated for learning the long-term dynamics and statistical properties of dissipative chaotic systems.
Learning Clustering-based Prototypes for Compositional Zero-Shot Learning
Hongyu Qu (Nanjing University of Science and Technology), Wenguan Wang (Zhejiang University)
ClassificationRepresentation LearningContrastive LearningImage
🎯 What it does: A clustering-based prototype mining framework called CLUSPRO is proposed to learn multi-prototype representations of attributes and objects, thereby enhancing the generalization ability of compositional zero-shot learning.
Learning Color Equivariant Representations
Yulong Yang (Princeton University), Christine Allen-Blanchette (Princeton University)
Representation LearningConvolutional Neural NetworkImage
🎯 What it does: This paper proposes a group convolutional neural network (GCNN) that is equivariant to color changes, aiming to address the sensitivity of traditional neural networks to color variations.
Learning Continually by Spectral Regularization
Alex Lewandowski (University of Alberta), Marlos C. Machado (University of Alberta)
ClassificationReinforcement LearningImage
🎯 What it does: This paper proposes a new spectral regularization method to keep the maximum singular values of each layer close to 1 in continuous learning, thereby maintaining the trainability of the network.
Learning Diagrams: A Graphical Language for Compositional Training Regimes
Mason Lary (University at Buffalo), James Fairbanks
Knowledge DistillationImageText
🎯 What it does: This paper proposes the Learning Diagrams, a graphical and composable framework for describing and constructing multi-model deep learning training processes, providing a unified semantics and an operable DSL;
Learning Distributions of Complex Fluid Simulations with Diffusion Graph Networks
Mario Lino Valencia, Nils Thuerey (Technical University of Munich)
GenerationData SynthesisOptimizationGraph Neural NetworkDiffusion modelTime Series
🎯 What it does: A diffusion model based on graph neural networks (DGN and LDGN) is proposed, which can directly learn and sample the equilibrium state distribution of fluid simulations from incomplete short time series data, thereby quickly obtaining flow field statistics.
Learning Diverse Attacks on Large Language Models for Robust Red-Teaming and Safety Tuning
Seanie Lee (Korea Advanced Institute of Science and Technology), Moksh Jain (Mila - Quebec Artificial Intelligence Institute)
Adversarial AttackTransformerLarge Language ModelReinforcement LearningPrompt EngineeringText
🎯 What it does: By using a two-stage training approach (GFlowNet fine-tuning + MLE smoothing), diverse and effective attack prompts are generated, which are then used for automated red teaming tests on various large language models, verifying the cross-model transferability of the generated prompts and the enhancement of security fine-tuning.
Learning Dynamics of Deep Matrix Factorization Beyond the Edge of Stability
Avrajit Ghosh (Michigan State University), Qing Qu (University of Michigan)
Image
🎯 What it does: This study investigates the learning dynamics of deep linear networks when the learning rate exceeds the boundary of stability (EOS), revealing the occurrence of periodic oscillations and the behavior of balanced subspaces.
Learning Dynamics of LLM Finetuning
Yi Ren (University of British Columbia), Danica J. Sutherland (University of British Columbia)
OptimizationData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
🎯 What it does: This study unifies the learning dynamics of large language models (LLMs) at different fine-tuning stages (SFT, DPO, etc.) and provides a step-by-step impact decomposition formula; based on this, it explains various phenomena (such as repetitive nonsense, confidence decay, compression effects, etc.) and proposes a simple improvement method to alleviate the compression effect by expanding the dataset during the SFT stage.
Learning Efficient Positional Encodings with Graph Neural Networks
Charilaos Kanatsoulis, Alejandro Ribeiro (University of Pennsylvania)
Graph Neural NetworkGraph
🎯 What it does: A learnable position encoding framework based on graph neural networks (PEARL) is proposed, which achieves transferable position encoding for graphs through random or basis vector initialization and statistical pooling after multiple message passing.
Learning Equivariant Non-Local Electron Density Functionals
Nicholas Gao (Munich Data Science Institute), Stephan Günnemann (Munich Data Science Institute)
Graph Neural NetworkPoint CloudPhysics Related
🎯 What it does: A non-local electronic density functional EG-XC based on SO(3) equivariant graph neural networks has been designed, which can efficiently learn non-local interactions by compressing the electronic density into a point cloud centered at the nucleus.
Learning Evolving Tools for Large Language Models
Guoxin Chen (Institute of Computing Technology, Chinese Academy of Sciences), Yasheng Wang (Tsinghua University)
TransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextBenchmark
🎯 What it does: This study investigates the adaptability of large language models in tool-variable environments and proposes the TOOLEVO framework.
Learning Fine-Grained Representations through Textual Token Disentanglement in Composed Video Retrieval
Yue WU, Shuhui Wang (Key Lab of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences)
RetrievalTransformerVision Language ModelContrastive LearningVideoTextMultimodality
🎯 What it does: A fine-grained combined video retrieval dataset FineCVR-1M containing 1,010,071 video-text triplets has been constructed, and a framework for text feature separation and cross-modal alignment (FDCA) has been proposed to perform three types of separation (retention, injection, exclusion) at the sentence and word levels, thereby enhancing the performance of fine-grained combined video retrieval.
Learning from End User Data with Shuffled Differential Privacy over Kernel Densities
Tal Wagner (Tel Aviv University)
ClassificationSafty and PrivacyContrastive LearningImageText
🎯 What it does: A kernel density estimation and classification method under the shuffled differential privacy model is proposed, supporting the one-time collection and learning of classifiers from distributed data of end users.
Learning from Imperfect Human Feedback: A Tale from Corruption-Robust Dueling
Yuwei Cheng (University of Chicago), Haifeng Xu (University of Chicago)
OptimizationReinforcement Learning from Human FeedbackReinforcement LearningTabular
🎯 What it does: In the advantage lever problem with continuous action space, the study investigates the limited imperfect human feedback (LIHF) constrained by decay scale, provides a lower bound, and proposes a robust stochastic mirror descent algorithm (RoSMID) that achieves an approximately optimal regret upper bound.
Learning from negative feedback, or positive feedback or both
Abbas Abdolmaleki (Google DeepMind), Martin Riedmiller (Google DeepMind)
OptimizationReinforcement Learning from Human FeedbackReinforcement LearningText
🎯 What it does: A preference optimization algorithm based on probabilistic inference, PMPO, is proposed, which can perform policy learning with only positive feedback, negative feedback, or a mixture of both.