ICML 2025 Papers — Page 15
International Conference on Machine Learning · 3257 papers
Importance Sampling for Nonlinear Models
Prakash Palanivelu Rajmohan, Fred Roosta (University of Queensland)
OptimizationImageTabular
🎯 What it does: This paper proposes a framework for constructing nonlinear importance sampling using nonlinear Jacobian operators, aimed at accelerating the training of nonlinear models and enhancing diagnostic capabilities.
Impossible Videos
Zechen Bai (Show Lab National University of Singapore), Mike Zheng Shou (Show Lab National University of Singapore)
GenerationData SynthesisTransformerLarge Language ModelPrompt EngineeringVideoTextBenchmark
🎯 What it does: This paper studies the concept of Impossible Videos and constructs the IPV-BENCH benchmark to evaluate the performance of video generation and understanding models on such videos.
Improved Algorithm for Deep Active Learning under Imbalance via Optimal Separation
Shyam Nuggehalli (University of Wisconsin Madison), Robert D Nowak
ClassificationData-Centric LearningConvolutional Neural NetworkReinforcement LearningImage
🎯 What it does: This paper proposes the DIRECT algorithm, which addresses the issues of class imbalance and label noise in deep learning by using one-dimensional threshold learning to actively select samples that are both uncertain and class-balanced for labeling.
Improved and Oracle-Efficient Online $\ell_1$-Multicalibration
Rohan Ghuge (Georgia Institute of Technology), Sahil Singla (Georgia Institute of Technology)
Optimization
🎯 What it does: A direct learning method is proposed for the online ℓ1-multicalibration problem, significantly improving the error upper bound and achieving an oracle-efficient algorithm (which can be efficiently implemented using offline optimization oracle).
Improved Approximations for Hard Graph Problems using Predictions
Anders Aamand (University of Copenhagen), Hao WU (University of Waterloo)
OptimizationGraph Neural NetworkGraph
🎯 What it does: This paper introduces an edge-level prediction model and designs improved approximation algorithms that utilize machine learning prediction information for various NP-hard graph problems (such as Vertex Cover, Set Cover, Maximum Independent Set, MaxCut, etc.).
Improved Coresets for Vertical Federated Learning: Regularized Linear and Logistic Regressions
Supratim Shit (Indraprastha Institute of Information Technology Delhi), Bapi Chatterjee (Indraprastha Institute of Information Technology Delhi)
Federated LearningComputational EfficiencyTabularFinance Related
🎯 What it does: This paper proposes a smaller and more efficient coreset method for regularized logistic regression and ridge regression in vertical federated learning.
Improved Discretization Complexity Analysis of Consistency Models: Variance Exploding Forward Process and Decay Discretization Scheme
Ruofeng Yang (Shanghai Jiao Tong University), Shuai Li (Shanghai Jiao Tong University)
GenerationData SynthesisDiffusion modelImageVideoMultimodalityStochastic Differential EquationOrdinary Differential EquationAudio
🎯 What it does: This paper studies the theoretical upper limit of the number of discretization steps in the training phase of consistency models, proposes a more practical VESDE forward process and EDM step size scheduling, and provides an optimal analysis of discretization complexity; it also proves that two-step sampling can further reduce the required number of steps.
Improved Expressivity of Hypergraph Neural Networks through High-Dimensional Generalized Weisfeiler-Leman Algorithms
Detian Zhang (Soochow University), Chunjiang Zhu (University of North Carolina)
ClassificationGraph Neural NetworkGraph
🎯 What it does: This paper proposes the k-dimensional generalized Weisfeiler-Leman algorithm k-GWL and constructs a hypergraph neural network k-HNN with stronger expressive power based on it.
Improved Last-Iterate Convergence of Shuffling Gradient Methods for Nonsmooth Convex Optimization
Zijian Liu (New York University), Zhengyuan Zhou (Arena Technologies)
Optimization
🎯 What it does: This paper provides an improved theoretical analysis of the last iteration convergence for two gradient methods, Random Reordering (RR) and Single-shot Random Shuffling (SS), in non-smooth (strongly convex) convex optimization, and offers new convergence upper bounds.
Improved Learning via k-DTW: A Novel Dissimilarity Measure for Curves
Amer Krivošija (TU Dortmund), Chris Schwiegelshohn
ClassificationAnomaly DetectionOptimizationTime SeriesSequential
🎯 What it does: Proposed k-DTW as a curve similarity measure, providing exact and approximate algorithms along with theoretical analysis.
Improved Lower Bounds for First-order Stochastic Non-convex Optimization under Markov Sampling
Zhenyu Sun (Northwestern University), Ermin Wei (Northwestern University)
Optimization
🎯 What it does: This paper addresses the random non-convex optimization problem under Markov sampling, providing a sample complexity lower bound for any first-order zero-respecting algorithm, and proving that this lower bound is Ω(ϵ⁻⁴) for counting state Markov chains and Ω(τϵ⁻²) for finite state Markov chains. Subsequently, a new algorithm, MaC-SAGE, is proposed, which achieves an almost matching upper bound for finite state Markov chains, demonstrating that this algorithm reaches near-optimal sample efficiency.
Improved Off-policy Reinforcement Learning in Biological Sequence Design
Hyeonah Kim (Mila Quebec AI Institute), Jinkyoo Park (KAIST)
Drug DiscoveryConvolutional Neural NetworkTransformerReinforcement LearningBiomedical Data
🎯 What it does: A δ-Conservative Search based offline reinforcement learning method is proposed to balance novelty exploration and uncertainty of the surrogate model in biological sequence design.
Improved Online Confidence Bounds for Multinomial Logistic Bandits
Joongkyu Lee (Seoul National University), Min-hwan Oh (Seoul National University)
OptimizationTabular
🎯 What it does: An improved online confidence bound and a constant time algorithm OFU-MNL++ are proposed, along with a maximum likelihood estimation (MLE) based algorithm OFU-MNL2, addressing the dependency issues on parameter norm B and maximum combination size K in the MNL bandit.
Improved Regret Analysis in Gaussian Process Bandits: Optimality for Noiseless Reward, RKHS norm, and Non-Stationary Variance
Shogo Iwazaki (MI-6 Limited), Shion Takeno (Nagoya University)
Optimization
🎯 What it does: The paper studies the Gaussian Process (GP) bandit problem, proposes a new upper bound on the posterior variance, and improves the PE and MVR algorithms using this bound. It provides optimal (or near-optimal) theoretical bounds for cumulative and simple regret under three settings: no noise, RKHS norm, and non-stationary variance.
Improved Sample Complexity for Private Nonsmooth Nonconvex Optimization
Guy Kornowski (Weizmann Institute of Science), Kunal Talwar (Apple)
OptimizationSafty and PrivacyTabular
🎯 What it does: This paper studies differential privacy (DP) optimization algorithms, proposing a method to return Goldstein stationary points for non-smooth non-convex stochastic and empirical objectives, and improves the existing sample complexity bounds.
Improved Theoretically-Grounded Evolutionary Algorithms for Subset Selection with a Linear Cost Constraint
Dan-Xuan Liu (Nanjing University), Chao Qian (Nanjing University)
OptimizationGraph
🎯 What it does: This paper provides a theoretical reanalysis of the Pareto optimization-based evolutionary algorithm POMC, proving that its approximation ratio for the subset selection problem (monotone submodular functions with linear cost constraints) can be improved to 1/2. It also introduces a novel multi-objective evolutionary algorithm EPOL, which constructs residual problems for each element and utilizes POMC to achieve a practical approximation ratio of 0.6174, demonstrating superior performance compared to existing greedy and evolutionary algorithms in experiments.
Improving Compositional Generation with Diffusion Models Using Lift Scores
Chenning Yu (University of California San Diego), Sicun Gao (University of California San Diego)
GenerationData SynthesisDiffusion modelScore-based ModelImageMultimodality
🎯 What it does: This paper proposes CompLift, a training-free rejection resampling criterion based on lift scores, aimed at enhancing the consistency and completeness of diffusion models in multi-condition combination generation.
Improving Consistency Models with Generator-Augmented Flows
Thibaut Issenhuth (Criteo AI Lab), Alain Rakotomamonjy (Criteo AI Lab)
GenerationData SynthesisOptimizationKnowledge DistillationFlow-based ModelGenerative Adversarial NetworkImageOrdinary Differential Equation
🎯 What it does: This study investigates the error differences in consistency model training and proposes a Generator-Enhanced Flow (GC) to reduce single-sample estimation errors and accelerate the training and sampling of consistency models.
Improving Continual Learning Performance and Efficiency with Auxiliary Classifiers
Filip Szatkowski (Warsaw University of Technology), Joost van de Weijer (Universitat Autonoma de Barcelona)
ClassificationComputational EfficiencyConvolutional Neural NetworkTransformerImage
🎯 What it does: The study investigates how to incorporate auxiliary classifiers into continual learning to enhance performance and efficiency.
Improving Diversity in Language Models: When Temperature Fails, Change the Loss
Alexandre Verine (Ecole Normale Superieure), benjamin negrevergne
GenerationOptimizationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper studies the impact of temperature scaling on the diversity of language models and points out that simply increasing the temperature often fails to improve recall. To address this issue, the authors propose several new loss functions (TruncR, c-Div, λ-PR) based on the precision-recall (P&R) framework, allowing the model to focus more on recall during the training phase, thereby achieving a better P&R trade-off during subsequent temperature adjustments.
Improving Flow Matching by Aligning Flow Divergence
Yuhao Huang (University of Utah), Bao Wang (University of Utah)
GenerationData SynthesisFlow-based ModelImageVideoSequential
🎯 What it does: A new method for training generative models based on flow matching, FDM, is proposed, which reduces probabilistic path errors by simultaneously matching the vector field and its divergence.
Improving Generalization in Federated Learning with Highly Heterogeneous Data via Momentum-Based Stochastic Controlled Weight Averaging
Junkang Liu (Tianjin University), Wei Feng (Xidian University)
OptimizationFederated LearningTransformerImage
🎯 What it does: This paper proposes two federated learning algorithms, FedSWA and FedMoSWA, to address the issue of insufficient model generalization performance under highly heterogeneous data.
Improving Generalization with Flat Hilbert Bayesian Inference
Tuan Truong (Movian AI), Trung Le (Monash University)
ClassificationOptimizationTransformerSupervised Fine-TuningImageBenchmark
🎯 What it does: A Flat Hilbert Bayesian Inference (FHBI) algorithm based on RKHS is proposed to enhance the generalization performance of Bayesian inference.
Improving LLM Safety Alignment with Dual-Objective Optimization
Xuandong Zhao (University of California), Dawn Song (University of California)
OptimizationSafty and PrivacyTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Achieving safe alignment of large language models through dual-objective optimization, combining forced rejection training and harmful knowledge learning to enhance the model's robustness against various jailbreak attacks.
Improving LLM Video Understanding with 16 Frames Per Second
Yixuan Li (Tsinghua University), Chao Zhang (Tsinghua University)
RecognitionCompressionOptimizationTransformerLarge Language ModelVision Language ModelVideoMultimodality
🎯 What it does: F-16 is proposed, a multimodal large language model capable of video understanding at 16 FPS, along with a variable frame rate decoding method;
Improving LLMs for Recommendation with Out-Of-Vocabulary Tokens
Ting-Ji Huang (Nanjing University), Han-Jia Ye (Nanjing University)
Recommendation SystemTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Proposes META ID, which constructs user/product IDs using OOV words obtained from clustering, and injects them into LLM for instruction-based recommendations.
Improving Memory Efficiency for Training KANs via Meta Learning
Zhangchi Zhao (Xi'an Jiaotong University), Zongben Xu (Xi'an Jiaotong University)
Computational EfficiencyMeta LearningImage
🎯 What it does: A MetaKANs framework is proposed, which generates learnable activation function weights for Kolmogorov–Arnold Networks (KAN) through a small meta-learner, significantly reducing the number of parameters and memory usage during training.
Improving Model Alignment Through Collective Intelligence of Open-Source Models
Junlin Wang (Duke University), James Zou (Stanford University)
OptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningAgentic AIText
🎯 What it does: This study proposes the Mixture of Agents Alignment (MoAA), which generates high-quality alignment data through the collaboration of multiple open-source LLMs, and applies this mixed agent framework in two stages (SFT and DPO) to enhance the model's helpfulness and safety.
Improving Multi-Class Calibration through Normalization-Aware Isotonic Techniques
Alon Arad (Tel Aviv University), Saharon Rosset (Tel Aviv University)
ClassificationOptimizationImageText
🎯 What it does: Two multi-class post-calibration methods based on isotonic regression (NA-FIR and SCIR) are proposed, which directly consider probability normalization in the calibration optimization.
Improving Multimodal Learning Balance and Sufficiency through Data Remixing
Xiaoyu Ma (Southeast University), Yongjian Deng (Beijing University of Technology)
ClassificationOptimizationData-Centric LearningConvolutional Neural NetworkMultimodalityAudio
🎯 What it does: The paper proposes a training strategy called Data Remixing, which addresses the issues of modality laziness and modality conflict in multimodal learning by decoupling multimodal data at the sample level and reorganizing it at the batch level.
Improving Out-of-Distribution Detection via Dynamic Covariance Calibration
Kaiyu Guo (University of Queensland), Mahsa Baktashmotlagh (University of Queensland)
Anomaly DetectionConvolutional Neural NetworkTransformerContrastive LearningImage
🎯 What it does: This paper proposes a method for dynamically calibrating the covariance matrix during inference to enhance OOD detection;
Improving Out-of-Distribution Detection with Markov Logic Networks
Konstantin Kirchheim (University of Magdeburg), Frank Ortmeier (University of Magdeburg)
Anomaly DetectionExplainability and InterpretabilityImage
🎯 What it does: A framework for OOD detection that utilizes Markov Logic Networks (MLN) for probabilistic reasoning of interpretable semantic constraints and integrates with existing detectors is proposed.
Improving Parallel Program Performance with LLM Optimizers via Agent-System Interfaces
Anjiang Wei (Stanford University), Alex Aiken (Stanford University)
OptimizationLarge Language ModelAgentic AI
🎯 What it does: Utilizing a large language model to automatically generate and optimize a mapper for the Legion parallel programming framework, abstracting low-level system code into a domain-specific language (DSL) through an agent-system interface, and using AutoGuide to convert execution results into actionable natural language feedback, iteratively improving the performance of the mapper.
Improving Rationality in the Reasoning Process of Language Models through Self-playing Game
Pinzheng Wang (Soochow University), Min zhang
Reinforcement LearningText
🎯 What it does: A self-play based Critic-Discernment Game is proposed, allowing reasoners to learn to maintain correct answers and correct errors when faced with helpful or misleading comments, thereby enhancing the rationality and self-correction ability of large language models during the reasoning process.
Improving Reward Model Generalization from Adversarial Process Enhanced Preferences
Zhilong Zhang (Nanjing University), Yang Yu (Nanjing University)
Robotic IntelligenceReinforcement LearningSequential
🎯 What it does: By selecting pairs of strategies with significant differences in iterative index during the adversarial imitation learning process, preference data that covers a wider range and has higher accuracy is automatically generated, thereby enhancing the generalization ability of the reward model.
Improving Soft Unification with Knowledge Graph Embedding Methods
Xuanming Cui (University of Central Florida), Ser-Nam Lim (DSO National Laboratories)
Explainability and InterpretabilityComputational EfficiencyKnowledge DistillationNeural Architecture SearchGraph Neural NetworkGraphBiomedical Data
🎯 What it does: This paper enhances the embedding structure and reasoning efficiency of the Neural Theorem Prover (NTP) by combining Knowledge Graph Embedding (KGE) methods with NTP;
Improving the Continuity of Goal-Achievement Ability via Policy Self-Regularization for Goal-Conditioned Reinforcement Learning
Xudong Gong (National University of Defense Technology), Yong Dou (National University of Defense Technology)
Robotic IntelligenceReinforcement Learning
🎯 What it does: To address the discontinuity issue in the goal achievement capability of Goal-Conditioned Reinforcement Learning (GCRL) algorithms, this paper proposes a threshold-based self-regularization method (MSR) that enhances the continuity of goal achievement by controlling the KL divergence between adjacent goal policies.
Improving the Diffusability of Autoencoders
Ivan Skorokhodov (Snap Inc), Aliaksandr Siarohin (Carnegie Mellon University)
GenerationData SynthesisDiffusion modelAuto EncoderImageVideo
🎯 What it does: Through spectral analysis of the autoencoder in latent diffusion models, it was found that there are excessive high-frequency components in the latent space, which impair the coarse-to-fine generation characteristics of the diffusion process. A method was proposed to suppress high frequencies and align the spectrum of the latent space with the RGB space using scale equivariance regularization (downsampling consistency), significantly improving generation quality.
Improving the Effective Receptive Field of Message-Passing Neural Networks
Shahaf E. Finder (Ben-Gurion University), Eran Treister (Ben-Gurion University)
OptimizationGraph Neural NetworkGraphBenchmark
🎯 What it does: A message propagation neural network based on multi-scale interaction (IM-MPNN) is proposed, which expands the effective receptive field of graph neural networks through hierarchical graph coarsening and scale mixing mechanisms.
Improving the Scaling Laws of Synthetic Data with Deliberate Practice
Reyhane Askari-Hemmat (Meta), Adriana Romero-Soriano
GenerationData SynthesisDiffusion modelImage
🎯 What it does: A framework for dynamically generating synthetic data is proposed—Deliberate Practice for Synthetic Data Generation (DP), which continuously generates the most challenging and informative samples based on the model's predictive entropy during the training process.
Improving the Statistical Efficiency of Cross-Conformal Prediction
Matteo Gasparin (University of Padova), Aaditya Ramdas (Carnegie Mellon University)
Tabular
🎯 What it does: An improved cross-conformal prediction method is proposed, utilizing exchangeable and randomized p-value combinations, resulting in a smaller prediction set while maintaining at least 1-2α coverage guarantees.
Improving the Variance of Differentially Private Randomized Experiments through Clustering
Adel Javanmard (University of Southern California), Jean Pouget-Abadie (Google Research)
Safty and PrivacyTabular
🎯 What it does: A clustering structure is introduced in random experiments to implement a differentially private mechanism (CLUSTER-DP) to reduce the variance of causal effect estimation.
Improving Transformer World Models for Data-Efficient RL
Antoine Dedieu (Google DeepMind), Kevin Patrick Murphy
Convolutional Neural NetworkRecurrent Neural NetworkTransformerReinforcement LearningWorld ModelImage
🎯 What it does: Achieved efficient model-based reinforcement learning in the Craftax-classic game by improving the Transformer world model, reaching rewards above human level.
Improving Value Estimation Critically Enhances Vanilla Policy Gradient
Tao Wang (University of California), Sicun Gao
Reinforcement LearningSequential
🎯 What it does: By significantly increasing the number of value network updates per iteration in Vanilla Policy Gradient (VPG), it is demonstrated that the accuracy of value estimation is key to improving the performance of policy gradient algorithms, allowing the improved VPG to compete with or even outperform PPO in continuous control benchmark tasks.
Improving Your Model Ranking on Chatbot Arena by Vote Rigging
Rui Min (Sea AI Lab), Min Lin (Sea AI Lab)
ClassificationRecommendation SystemLarge Language ModelText
🎯 What it does: This study investigates how to manipulate the LLM rankings in Chatbot Arena through voting, proposing a strategy based on target specificity and omnipresent voting, and systematically evaluating its effectiveness and defense mechanisms.
Improving Zero-Shot Adversarial Robustness in Vision-Language Models by Closed-form Alignment of Adversarial Path Simplices
Junhao Dong (Nanyang Technological University), Yew-Soon Ong (Nanyang Technological University)
OptimizationRepresentation LearningAdversarial AttackVision Language ModelImageMultimodalityBiomedical Data
🎯 What it does: A zero-shot adversarial fine-tuning method named AdvSimplex is proposed, which enhances the robustness of Vision-Language models by aligning simplexes along adversarial paths in a closed form.
IMTS is Worth Time $\times$ Channel Patches: Visual Masked Autoencoders for Irregular Multivariate Time Series Prediction
Zhangyi Hu (Hong Kong University of Science and Technology), Yutao Yue (Hong Kong University of Science and Technology)
Anomaly DetectionOptimizationConvolutional Neural NetworkGraph Neural NetworkAuto EncoderTabularTime SeriesElectronic Health Records
🎯 What it does: A framework called VIMTS based on visual Masked AutoEncoder is proposed for handling high missing rates and irregular sampling in multivariate time series forecasting.
In-Context Adaptation to Concept Drift for Learned Database Operations
Jiaqi Zhu (Beijing Institute of Technology), Beng Chin Ooi (National University of Singapore)
TransformerTabular
🎯 What it does: The FLAIR framework is proposed, utilizing in-context learning to achieve online adaptation of database operations (such as cardinality estimation, approximate query processing, and data analysis) in environments with concept drift;
In-Context Deep Learning via Transformer Models
Weimin Wu (Northwestern University), Han Liu (Northwestern University)
OptimizationTransformerTabular
🎯 What it does: This study investigates the ability of Transformers to simulate the gradient descent training of deep models (N-layer ReLU networks) in In-context Learning (ICL), providing explicit constructions and theoretical proofs.
In-Context Denoising with One-Layer Transformers: Connections between Attention and Associative Memory Retrieval
Matthew Smart (Flatiron Institute), Anirvan M. Sengupta (Rutgers University)
Transformer
🎯 What it does: The paper proposes the 'in-context denoising' task, linking the attention mechanism of Transformers with Dense Associative Memory networks, demonstrating that a single-layer Transformer can achieve the Bayes optimal solution for certain denoising problems.
In-Context Fine-Tuning for Time-Series Foundation Models
Matthew Faw (Georgia Institute of Technology), Abhimanyu Das (Google Research)
TransformerSupervised Fine-TuningTime Series
🎯 What it does: In the task of time series data prediction, a pre-trained model method is proposed that achieves model adaptation during inference by providing relevant time series samples as context (in-context);
In-Context Learning and Occam's Razor
Eric Elmoznino (Mila - Quebec AI Institute), Guillaume Lajoie (Mila - Quebec AI Institute)
CompressionMeta LearningTransformerLarge Language ModelSequential
🎯 What it does: Research shows that in-context learning (ICL) of sequence models is exactly equivalent to a compression method called prequential coding through the prediction of the next token, thereby minimizing both training error and model complexity during the meta-learning phase, reflecting the principle of Occam's Razor.
In-Context Learning as Conditioned Associative Memory Retrieval
Weimin Wu (Northwestern University), Han Liu (Northwestern University)
TransformerPrompt EngineeringText
🎯 What it does: This paper provides an interpretable example to explain the In-Context Learning (ICL) mechanism in forward reasoning by viewing the single-layer attention model as a conditional memory retrieval of modern Hopfield networks.
In-Context Linear Regression Demystified: Training Dynamics and Mechanistic Interpretability of Multi-Head Softmax Attention
Jianliang He (Yale University), Zhuoran Yang (Yale University)
OptimizationExplainability and InterpretabilityTransformerTabular
🎯 What it does: This study explores the training dynamics and mechanism interpretability of the multi-head softmax attention model when handling linear data, revealing unique patterns in attention weights.
In-Context Reinforcement Learning From Suboptimal Historical Data
Juncheng Dong (Duke University), Vahid Tarokh (Duke University)
TransformerReinforcement LearningSequential
🎯 What it does: Unsupervised pre-training using Transformer on a dataset of sub-optimal historical trajectories containing multiple different RL tasks, and determining the action policy for new tasks based on contextual data during testing.
iN2V: Bringing Transductive Node Embeddings to Inductive Graphs
Nicolas Lell (Ulm University), Ansgar Scherp (Ulm University)
Recommendation SystemRepresentation LearningGraph Neural NetworkGraphBenchmark
🎯 What it does: This paper proposes iN2V, a framework that adapts the traditional node2vec to an inductive learning environment on graphs without features or requiring the generation of embeddings for new nodes after training.
Incentivize without Bonus: Provably Efficient Model-based Online Multi-agent RL for Markov Games
Tong Yang (Carnegie Mellon University), Yuejie Chi (Meta AI)
Reinforcement Learning
🎯 What it does: A new model-based online multi-agent reinforcement learning algorithm, VMG, is proposed, aiming to encourage exploration by biasing the experience estimates of model parameters towards those with higher collective best response values, thereby encouraging strategies to deviate from the current equilibrium for more exploration.
Incorporating Arbitrary Matrix Group Equivariance into KANs
Lexiang Hu (Peking University), Zhouchen Lin (Peking University)
Physics Related
🎯 What it does: This paper proposes Equivariant Kolmogorov-Arnold Networks (EKAN), which integrates equivariance of arbitrary matrix groups into the KAN architecture to achieve scalable symmetry networks.
Incremental Gradient Descent with Small Epoch Counts is Surprisingly Slow on Ill-Conditioned Problems
Yujun Kim (Korea Advanced Institute of Science and Technology), Chulhee Yun (Korea Advanced Institute of Science and Technology)
Optimization
🎯 What it does: This paper studies the convergence properties of Incremental Gradient Descent (IGD) under both small and large epoch counts, providing upper and lower bounds, and revealing the significant suppression of convergence caused by non-convex components.
Independence Tests for Language Models
Sally Zhu (Stanford University), Percy Liang (Stanford University)
Large Language ModelText
🎯 What it does: This study investigates how to detect whether two language models are independently trained based on model weights, proposing two statistical testing frameworks: constrained and unconstrained, and implementing fine-grained dependency localization and model lineage tracking.
Inducing, Detecting and Characterising Neural Modules: A Pipeline for Functional Interpretability in Reinforcement Learning
Anna Soligo (Imperial College London), David Boyle (Imperial College London)
Explainability and InterpretabilityReinforcement LearningTabular
🎯 What it does: This paper proposes the incorporation of spatial distance-based sparse regularization and neuron migration into the reinforcement learning policy network to facilitate the natural emergence and interpretability of functional modules.
Inductive Gradient Adjustment for Spectral Bias in Implicit Neural Representations
Kexuan Shi (University of Electronic Science and Technology of China), Shuhang Gu (University of Electronic Science and Technology of China)
OptimizationRepresentation LearningNeural Radiance FieldImagePoint CloudMesh
🎯 What it does: To address the spectral bias problem of multilayer perceptrons in implicit neural representations (INR), a gradient adjustment method induced by the empirical neural tangent kernel (eNTK) is proposed, which adaptively adjusts the NTK spectrum during training to accelerate the convergence of high-frequency components and improve model accuracy.
Inductive Moment Matching
Linqi Zhou (Luma AI), Jiaming Song (Stanford University)
GenerationData SynthesisTransformerDiffusion modelScore-based ModelImage
🎯 What it does: Proposes the Inductive Moment Matching (IMM) method, which utilizes self-consistent interpolation to train generative models from scratch or with few steps.
InfAlign: Inference-aware language model alignment
Ananth Balashankar (Google DeepMind), Ahmad Beirami (Google Research)
Reinforcement Learning from Human FeedbackReinforcement LearningText
🎯 What it does: This study investigates how to align language models under known inference time decoding (such as Best-of-N, Worst-of-N) and proposes the InfAlign framework.
Inference-Time Alignment of Diffusion Models with Direct Noise Optimization
Zhiwei Tang (Chinese University of Hong Kong), Tsung-Hui Chang
GenerationOptimizationReinforcement LearningDiffusion modelImage
🎯 What it does: Proposes directly optimizing diffusion model noise (DNO) during the inference phase to align with continuous reward functions.
Inference-Time Decomposition of Activations (ITDA): A Scalable Approach to Interpreting Large Language Models
Patrick Leask (Durham University), Noura Al Moubayed (Durham University)
Explainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelAuto EncoderText
🎯 What it does: This paper proposes an ITDA method for sparse dictionary decomposition of large language model activations during inference, capable of generating interpretable sparse representations in a very short time.
Info-Coevolution: An Efficient Framework for Data Model Coevolution
Ziheng Qin (National University of Singapore), Yang You (National University of Singapore)
ClassificationRetrievalData-Centric LearningTransformerContrastive LearningImage
🎯 What it does: Proposes the Info-Coevolution framework, which performs sample selection online through information gain estimation and Bayesian prediction fusion, achieving no performance loss with only 68% labeling on ImageNet-1K.
InfoCons: Identifying Interpretable Critical Concepts in Point Clouds via Information Theory
Feifei Li (Fudan University), Min Yang (Fudan University)
Autonomous DrivingExplainability and InterpretabilityPoint Cloud
🎯 What it does: This paper proposes an explanation framework based on information bottleneck theory, called InfoCons, which can partition point clouds into interpretable key concepts and quantify their causal impact on model predictions.
Information Bottleneck-guided MLPs for Robust Spatial-temporal Forecasting
Min Chen (Tianjin University), Cheng Yan (Tianjin University)
Knowledge DistillationTime SeriesSequential
🎯 What it does: Proposes the Robust Space-Time Information Bottleneck (RSTIB) principle and implements it as RSTIB-MLP, achieving both robustness and efficiency in space-time prediction tasks with noise disturbances using a pure MLP structure.
InfoSAM: Fine-Tuning the Segment Anything Model from An Information-Theoretic Perspective
yuanhong zhang, Weijie Shi (Xi'an Jiaotong University)
SegmentationKnowledge DistillationSupervised Fine-TuningImageBiomedical DataAgriculture Related
🎯 What it does: InfoSAM proposes a SAM fine-tuning framework from an information-theoretic perspective, compressing domain-invariant relationships through an attention relationship module and efficiently transferring them to the student model to retain the pre-trained SAM relationships.
InfoSEM: A Deep Generative Model with Informative Priors for Gene Regulatory Network Inference
Tianyu Cui (Johnson and Johnson Innovative Medicine), Rui Liao (Johnson and Johnson Innovative Medicine)
Large Language ModelBiomedical Data
🎯 What it does: A deep generative model called InfoSEM is proposed, which utilizes text gene embeddings as priors to achieve unsupervised gene regulatory network inference.
INRFlow: Flow Matching for INRs in Ambient Space
Yuyang Wang (Apple), Miguel Ángel Bautista
GenerationData SynthesisTransformerFlow-based ModelMultimodalityPoint Cloud
🎯 What it does: A flow matching generative model called INRFlow is constructed during the same training phase for different data domains, enabling direct modeling in the environmental space and eliminating the dependency on the two-stage training of compressors.
Instance Correlation Graph-based Naive Bayes
Chengyuan Li (China University of Geosciences), Huan Zhang (Zhengzhou University)
ClassificationGraph Neural NetworkAuto EncoderTabular
🎯 What it does: Construct an Instance Correlation Graph (ICG) and generate new attributes using VGAE, then weight the new attributes, and finally use Gaussian Naive Bayes for classification on the weighted attributes.
Instance-Optimal Pure Exploration for Linear Bandits on Continuous Arms
Sho Takemori (Fujitsu Limited), Aditya Gopalan (Indian Institute of Science)
OptimizationReinforcement LearningTabular
🎯 What it does: This paper studies the linear Bandit pure exploration problem on continuous action sets and proposes a feasible instance-optimal sampling strategy algorithm called PMCA.
Instruct2See: Learning to Remove Any Obstructions Across Distributions
Junhang Li (South China University of Technology), Shengfeng He (Singapore Management University)
RestorationTransformerVision Language ModelImageMultimodality
🎯 What it does: A zero-shot obstacle removal framework called Instruct2See is proposed, capable of handling arbitrary soft and hard occlusions both within and outside the training distribution, unifying them into a soft and hard mask recovery problem.
Instruction-Following Pruning for Large Language Models
Bairu Hou (University of California Santa Barbara), Tao Lei (Apple)
TransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: In this work, the authors propose an instruction-based dynamic structured pruning method (IFPRUNING), which generates variable masks based on user prompts through a lightweight sparsity predictor, thereby activating only the parameter sub-networks most relevant to the current task during inference.
Integer Programming for Generalized Causal Bootstrap Designs
Jennifer Rogers Brennan (Google Research), Jean Pouget-Abadie (Google Research)
OptimizationTabularFinance Related
🎯 What it does: Proposes an integer programming method to calculate the worst-case copula, thereby improving the confidence interval estimation of causal bootstrap methods under various experimental designs and estimators.
Integrating Intermediate Layer Optimization and Projected Gradient Descent for Solving Inverse Problems with Diffusion Models
Yang Zheng (University of Electronic Science and Technology of China), Zhaoqiang Liu (University of Electronic Science and Technology of China)
RestorationSuper ResolutionOptimizationDiffusion modelImage
🎯 What it does: This paper proposes two methods for solving inverse problems based on diffusion models, DMILO and DMILO-PGD, which reduce memory consumption and avoid local optima through intermediate layer optimization and projected gradient descent, respectively.
Integration-free Kernels for Equivariant Gaussian Process Modelling
Tim Steinert (University of Bern), Henry Moss (Lancaster University)
Tabular
🎯 What it does: This paper proposes a transformation-invariant kernel that does not require integration, used to construct Gaussian process models that satisfy the invariance of random fields.
Interaction-Aware Gaussian Weighting for Clustered Federated Learning
Alessandro Licciardi (Polytechnic University of Turin), Marco Ciccone (Vector Institute)
Federated LearningImage
🎯 What it does: A Gaussian-weighted clustering federated learning framework, FedGWC, is proposed, which utilizes the clients' empirical loss to estimate data distribution similarity and automatically partitions homogeneous clusters. Subsequently, dedicated models are trained within each cluster to enhance personalization and overall accuracy.
Interchangeable Token Embeddings for Extendable Vocabulary and Alpha-Equivalence
İlker Işık (Middle East Technical University), Ebru Aydin Gol (Microsoft)
TransformerText
🎯 What it does: A dual-component word embedding strategy (shared learning part + random differentiation part) is proposed, allowing the model to see a limited vocabulary during training while being able to extend to a larger interchangeable vocabulary (such as atomic propositions in LTL) during inference, and demonstrating robustness that is insensitive to α-equivalence (variable renaming).
Internal Causal Mechanisms Robustly Predict Language Model Out-of-Distribution Behaviors
Jing Huang (Stanford University), Christopher Potts (Stanford University)
TransformerLarge Language ModelText
🎯 What it does: The study utilizes internal causal mechanisms to predict the correctness of language models on out-of-distribution inputs.
Interpolating Neural Network-Tensor Decomposition (INN-TD): a scalable and interpretable approach for large-scale physics-based problems
Jiachen Guo (Northwestern University), Wing Kam Liu (Northwestern University)
OptimizationExplainability and InterpretabilityComputational EfficiencyData-Centric LearningRobotic IntelligenceTime SeriesPhysics Related
🎯 What it does: This paper proposes the Interpolating Neural Network-Tensor Decomposition (INN-TD) framework to achieve high-precision function approximation and solving in large-scale, high-resolution physical problems (PDEs), covering three main tasks: data-driven training, data-free solving, and inverse optimization.
Interpreting CLIP with Hierarchical Sparse Autoencoders
Vladimir Zaigrajew (Warsaw University of Technology), Przemyslaw Biecek (University of Warsaw)
RetrievalExplainability and InterpretabilityAuto EncoderImageTextMultimodality
🎯 What it does: The Matryoshka SAE architecture is proposed, which achieves hierarchical interpretation and concept extraction of CLIP multimodal representations through multi-layer TopK sparse autoencoders, and is used for similarity search and bias analysis.
Interpreting the Repeated Token Phenomenon in Large Language Models
Itay Yona (Google DeepMind), Yossi Gandelsman (University of California Berkeley)
Explainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: This paper clarifies the root cause of the 'repeated word error' phenomenon in large language models through a mechanistic explanation approach and proposes a neural circuit-based remedy.
Intersectional Fairness in Reinforcement Learning with Large State and Constraint Spaces
Eric Eaton (University of Pennsylvania), Jessica Sorrell (Johns Hopkins University)
OptimizationReinforcement LearningGraph
🎯 What it does: This paper proposes a framework that simultaneously considers multiple objectives and cross-group fairness in reinforcement learning, aiming to maximize the minimum reward across all groups, and presents an oracle-efficient algorithm capable of handling exponential cross-group constraints in both tabular and large state spaces.
IntLoRA: Integral Low-rank Adaptation of Quantized Diffusion Models
Hang Guo (Tsinghua University), Luca Benini (ETH Zurich)
GenerationData SynthesisDiffusion modelImageText
🎯 What it does: Implement integer low-rank adaptation (IntLoRA) on quantized diffusion models, eliminating the post-processing quantization step during inference.
Introducing 3D Representation for Dense Volume-to-Volume Translation via Score Fusion
Xiyue Zhu (University of Illinois), Volodymyr Kindratenko (National Center for Supercomputing Applications)
Image TranslationSegmentationSuper ResolutionConvolutional Neural NetworkDiffusion modelScore-based ModelAuto EncoderImageVideoBiomedical DataMagnetic Resonance Imaging
🎯 What it does: This study proposes a dense volume translation model named Score-Fusion, which achieves efficient and high-quality 3D medical image translation and video super-resolution by fusing multi-directional 2D diffusion models in the score function space and learning 3D representations in a 3D network.
Invariance Makes LLM Unlearning Resilient Even to Unanticipated Downstream Fine-Tuning
Changsheng Wang (Michigan State University), Sijia Liu (IBM Research)
TransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: A memory erasure method for LLM based on invariant risk minimization (ILU) is proposed, allowing the model to maintain irrecoverable forgotten information during subsequent fine-tuning.
Invariant Deep Uplift Modeling for Incentive Assignment in Online Marketing via Probability of Necessity and Sufficiency
Zexu Sun (Renmin University of China), xiuqiang He
Recommendation SystemOptimizationTabular
🎯 What it does: This paper proposes a deep uplift model IDUM based on invariant learning and probabilistic necessary and sufficient conditions (PNS) to address the issues of distribution drift and selection bias in online marketing incentive allocation.
Inverse Bridge Matching Distillation
Nikita Gushchin (Skolkovo Institute of Science and Technology), Alexander Korotin (Skolkovo Institute of Science and Technology)
Image TranslationRestorationSuper ResolutionComputational EfficiencyKnowledge DistillationDiffusion modelImage
🎯 What it does: A universal Inverse Bridge Matching Distillation method (IBMD) is proposed, which compresses the Diffusion Bridge Model (DBM) into a few-step generator, supporting both conditional and unconditional models, and trains using only corrupted data.
Inverse Flow and Consistency Models
Yuchen Zhang (University of Texas Southwestern Medical Center), Jian Zhou (University of Texas Southwestern Medical Center)
RestorationGenerationDiffusion modelFlow-based ModelImageBiomedical DataStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Proposes the Inverse Flow framework to address the inverse generation problem without ground truth (such as denoising), and presents two algorithms: Inverse Flow Matching (IFM) and Inverse Consistency Model (ICM).
Inverse Optimization via Learning Feasible Regions
Ke Ren (Amazon), Angelos Georghiou (University of Cyprus)
OptimizationTabular
🎯 What it does: This study investigates parameterization methods for learning feasible regions in inverse optimization, proposing two loss functions (predictive loss and suboptimal loss), designing block coordinate descent and smoothing algorithms, and providing reformulations for convexification and mixed-integer linear programming.
Inverse Problem Sampling in Latent Space Using Sequential Monte Carlo
Idan Achituve (Sony Semiconductor Israel), Ethan Fetaya (Bar-Ilan University)
RestorationSuper ResolutionDiffusion modelImage
🎯 What it does: A latent diffusion model inverse problem sampling method based on Sequential Monte Carlo (SMC) called LD-SMC is proposed, which utilizes auxiliary observations to solve inverse problems in the latent space.
Inverse problems with experiment-guided AlphaFold
Sai Advaith Maddipatla, Alexander Bronstein
Protein Structure PredictionDiffusion modelBiomedical Data
🎯 What it does: This study utilizes experiment-guided AlphaFold3 to generate a set of protein conformations consistent with X-ray density maps and NMR NOE constraints.
Inverse Reinforcement Learning with Switching Rewards and History Dependency for Characterizing Animal Behaviors
Jingyang Ke (Georgia Institute of Technology), Anqi Wu (Georgia Institute of Technology)
Reinforcement LearningTime SeriesSequential
🎯 What it does: Proposes the SWIRL (Switching IRL) framework, which combines time-varying and history-dependent reward functions to characterize long-term natural behaviors of animals;
Investigating Non-Transitivity in LLM-as-a-Judge
Yi Xu (University College London), Robert Kirk (University College London)
Reinforcement Learning from Human FeedbackTransformerLarge Language ModelText
🎯 What it does: This paper studies the non-transitive preferences that arise when large language models (LLMs) act as judges, and proposes a ranking framework based on round-robin and the Bradley-Terry model, while designing an efficient SWIM (Swiss-Wise Iterative Matchmaking) tournament to reduce computational costs.
Investigating the Overlooked Hessian Structure: From CNNs to LLMs
Qian-Yuan Tang (Hong Kong Baptist University), Zeke Xie (Hong Kong University of Science and Technology)
OptimizationConvolutional Neural NetworkTransformerLarge Language ModelImageText
🎯 What it does: This paper first discovers and verifies that the Hessian spectrum in trained convolutional networks and large language models exhibits a simple power-law structure, providing a maximum entropy theoretical explanation and deriving a low-dimensional robust subspace; it further explores the relationship between this structure and phenomena such as optimization, generalization, and over-parameterization.
IRBridge: Solving Image Restoration Bridge with Pre-trained Generative Diffusion Models
Hanting Wang (Zhejiang University), Zhou Zhao (Zhejiang University)
RestorationDiffusion modelImageStochastic Differential Equation
🎯 What it does: A framework named IRBridge is proposed, which utilizes pre-trained generative diffusion models (such as Stable Diffusion v1-5) and image restoration bridge models (IR-SDE, GOUB) coupled through a transfer equation to achieve various image restoration tasks (de-raining, de-hazing, de-snowing, de-droplet, low-light enhancement, image inpainting) without the need to retrain the bridge model for each type of degradation.
Is Best-of-N the Best of Them? Coverage, Scaling, and Optimality in Inference-Time Alignment
Audrey Huang (University of Illinois at Urbana-Champaign), Dylan J Foster
OptimizationComputational EfficiencyReinforcement Learning from Human FeedbackReinforcement LearningText
🎯 What it does: This paper proposes a theoretical framework for inference-time alignment, analyzing the issue of reward overfitting that commonly arises with the Best-of-N sampling method when the reward model is imperfect. Based on χ² regularization, a new inference-time pessimistic algorithm, InferenceTimePessimism, is designed to achieve optimal regret and scalable performance.