ICML 2025 Papers — Page 33
International Conference on Machine Learning · 3257 papers
When and How Does CLIP Enable Domain and Compositional Generalization?
Elias Kempf (University of Freiburg), Thomas Brox (University of Freiburg)
Domain AdaptationRepresentation LearningTransformerVision Language ModelAuto EncoderContrastive LearningImageTextMultimodality
🎯 What it does: Systematically investigates the performance of CLIP in domain transfer and compositional generalization by controlling the training data distribution; explores the impact of domain diversity, class exposure, and shared intermediate representations on generalization through structured experiments.
When Bad Data Leads to Good Models
Kenneth Li (Harvard University), Martin Wattenberg (Harvard University)
GenerationAdversarial AttackData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: The study investigates the impact of incorporating toxic data during the pre-training phase of large language models on the effectiveness of post-training alignment. It systematically evaluates the relationship between the proportion of toxic data and the model's toxicity recognition and controllability, and employs post-training methods such as prompts and Inference-Time Interventions (ITI) to reduce generated toxicity.
When can in-context learning generalize out of task distribution?
Chase Goddard (Princeton University), David J. Schwab (Graduate Center CUNY)
TransformerLarge Language ModelTabular
🎯 What it does: This paper studies whether the Transformer can generalize to out-of-distribution tasks when the diversity of pre-training task distributions changes. By sampling linear regression tasks on high-dimensional spherical sections at different angles ϕ and training a GPT-2 style Transformer for in-context learning (ICL), a transition from 'specialized' solutions to 'general' solutions was observed; this transition was further validated in classification and nonlinear regression tasks.
When Can Proxies Improve the Sample Complexity of Preference Learning?
Yuchen Zhu (University College London), Alexander Nicholas D'Amour
Recommendation SystemOptimizationReinforcement LearningBiomedical Data
🎯 What it does: This paper proposes several sufficient conditions and proves that when these conditions are met, using proxy preference data can significantly reduce the sample complexity of learning the true preference strategy, and presents a two-stage learning algorithm.
When Data-Free Knowledge Distillation Meets Non-Transferable Teacher: Escaping Out-of-Distribution Trap is All You Need
Ziming Hong (Sydney AI Centre), Tongliang Liu (Sydney AI Centre)
Knowledge DistillationAdversarial AttackGenerative Adversarial NetworkImage
🎯 What it does: This study investigates the OOD 'trap' effect that occurs during the Data-Free Knowledge Distillation (DFKD) process when the teacher model is Non-Transferable Learning (NTL), and proposes the ATEsc method based on the differences in adversarial robustness to mitigate this effect.
When Diffusion Models Memorize: Inductive Biases in Probability Flow of Minimum-Norm Shallow Neural Nets
Chen Zeno (Technion), Daniel Soudry (Technion)
GenerationData SynthesisDiffusion modelMultimodalityOrdinary Differential Equation
🎯 What it does: Analyzed the probability flow ODE and fractional flow ODE of shallow ReLU networks under minimum L2 normalization, studied their convergence behavior on orthogonal and obtuse simplex training sets, and verified theoretical predictions through simulations.
When Do LLMs Help With Node Classification? A Comprehensive Analysis
Xixi Wu (Chinese University of Hong Kong), Hong Cheng (Chinese University of Hong Kong)
ClassificationGraph Neural NetworkTransformerLarge Language ModelPrompt EngineeringTextGraphBenchmark
🎯 What it does: The LLMNodeBed benchmark was constructed, systematically comparing 14 multi-domain graph datasets, 8 types of LLM baseline methods, and 8 classical methods. Over 2,700 models were trained and evaluated under three learning modes: semi-supervised, supervised, and zero-shot, exploring the effectiveness and bottlenecks of LLM in node classification.
When do neural networks learn world models?
Tianren Zhang (Tsinghua University), Feng Chen (Tsinghua University)
World ModelTime SeriesSequentialPhysics Related
🎯 What it does: The study investigates whether neural networks can learn the latent variables (world models) of data generation through low-complexity biases in a multi-task learning environment.
When Dynamic Data Selection Meets Data Augmentation: Achieving Enhanced Training Acceleration
Suorong Yang (Nanjing University), Dongzhan Zhou (Shanghai Artificial Intelligence Laboratory)
Computational EfficiencyData-Centric LearningTransformerContrastive LearningImage
🎯 What it does: This paper proposes an online training framework that unifies dynamic data selection and data augmentation, significantly reducing the amount of training data and computational costs while maintaining or even improving model performance.
When Every Millisecond Counts: Real-Time Anomaly Detection via the Multimodal Asynchronous Hybrid Network
Dong Xiao (Peking University), Yonghong Tian (Peking University)
Anomaly DetectionAutonomous DrivingConvolutional Neural NetworkGraph Neural NetworkImageVideoMultimodality
🎯 What it does: A multi-modal asynchronous hybrid network is designed to achieve real-time anomaly detection in autonomous driving environments, focusing on reducing response time while maintaining high detection accuracy.
When Maximum Entropy Misleads Policy Optimization
Ruipeng Zhang (University of California San Diego), Sicun Gao (University of California San Diego)
Autonomous DrivingOptimizationRobotic IntelligenceReinforcement LearningSequential
🎯 What it does: This paper studies the reasons why Maximum Entropy Reinforcement Learning (MaxEnt RL) may lead to misleading policy optimization in complex control tasks, and proposes the 'Entropy Bifurcation Extension' method to demonstrate the existence of this misleading effect. It further experimentally verifies the reasons for the poor performance of Soft Actor-Critic (SAC) in real control environments (such as high-speed vehicles, quadcopters, and quadruped robots) and explores the mitigating effects of adaptive entropy regulation.
When Model Knowledge meets Diffusion Model: Diffusion-assisted Data-free Image Synthesis with Alignment of Domain and Class
Yujin Kim (Korea University), Suhyun Kim (Kyung Hee University)
GenerationData SynthesisKnowledge DistillationDiffusion modelImage
🎯 What it does: A data-independent image synthesis method DDIS is proposed, which utilizes the internal knowledge of text-to-image diffusion models and pre-trained models to generate image samples that approximate the training set distribution.
When to Forget? Complexity Trade-offs in Machine Unlearning
Martin Van Waerebeke (INRIA Paris), Kevin Scaman (INRIA Paris)
Safty and PrivacyComputational EfficiencyImageTabular
🎯 What it does: This paper studies the 'forgetting' problem in machine learning, proposes and analyzes the computational complexity of unlearning, presents three working stages (ineffective, inefficient, effective) under different privacy and forgetting ratios, and evaluates the efficiency comparison between the optimal unlearning algorithm and retraining from scratch by minimizing the maximum computational time framework.
When to retrain a machine learning model
Florence Regol (Block), Thomas Markovich (Block)
Anomaly DetectionOptimizationTime Series
🎯 What it does: This paper addresses the decision problem of when to retrain a model when performance declines due to distribution drift over time, and provides a cost-based formalization.
When Will It Fail?: Anomaly to Prompt for Forecasting Future Anomalies in Time Series
Min-Yeong Park (Kyung Hee University), Gyeong-Moon Park (Korea University)
Anomaly DetectionTransformerTime Series
🎯 What it does: A time series anomaly prediction framework A2P is proposed, which can locate anomaly points in future signals.
When, Where and Why to Average Weights?
Niccolò Ajroldi (Max Planck Institute for Intelligent Systems), Jonas Geiping (Max Planck Institute for Intelligent Systems)
OptimizationTransformerSupervised Fine-TuningImageTextBenchmark
🎯 What it does: Evaluated and validated the effects of weight averaging (LAWA, EMA) on accelerating training and improving generalization on large deep learning benchmarks.
Where is the Truth? The Risk of Getting Confounded in a Continual World
Florian Peter Busch (TU Darmstadt), Martin Mundt (University of Bremen)
ClassificationRecognitionConvolutional Neural NetworkImage
🎯 What it does: The study investigates the impact of confounding factors on model performance in continuous learning scenarios, proposing the ConCon dataset and evaluating various CL methods.
Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems
Shaokun Zhang (Pennsylvania State University), Qingyun Wu (AG2AI, Inc.)
TransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Research on automation failure attribution, identifying agents and steps that lead to task failures in multi-agent systems.
Which Attention Heads Matter for In-Context Learning?
Kayo Yin (University of California Berkeley), Jacob Steinhardt (University of California Berkeley)
TransformerLarge Language ModelText
🎯 What it does: This paper conducts a systematic analysis of 12 different scales of decoder-only language models (Pythia, GPT-2, Llama-2) to compare the roles of two types of attention heads—induction heads and function vector (FV) heads—in few-shot in-context learning (ICL).
Whitened CLIP as a Likelihood Surrogate of Images and Captions
Roy Betser (Technion Israel Institute of Technology), Guy Gilboa (Technion Israel Institute of Technology)
GenerationDomain AdaptationAnomaly DetectionTransformerVision Language ModelContrastive LearningImageText
🎯 What it does: Whitened CLIP is proposed, transforming the CLIP embedding space into an approximately standard normal isotropic distribution through an invertible linear whitening, allowing for direct estimation of the log-likelihood of images and texts using the Euclidean norm;
Whoever Started the interference Should End It: Guiding Data-Free Model Merging via Task Vectors
Runxi Cheng (Tsinghua University), Chun Yuan (Tsinghua University)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: A data-free, scale-free model merging method called WUDI-Merging is proposed.
Why Has Predicting Downstream Capabilities of Frontier AI Models with Scale Remained Elusive?
Rylan Schaeffer (Stanford University), Sanmi Koyejo (Stanford University)
Large Language ModelTextBenchmark
🎯 What it does: Analyzed and quantified how the conversion process from negative log-likelihood to final evaluation metrics in multiple-choice question answering benchmarks leads to unpredictability in model performance as scale changes, highlighting that fluctuations in the probability quality of incorrect options are the main reason.
Why Is Spatial Reasoning Hard for VLMs? An Attention Mechanism Perspective on Focus Areas
Shiqi Chen (City University of Hong Kong), Manling Li (Northwestern University)
TransformerVision Language ModelImage
🎯 What it does: By analyzing the attention distribution of visual language models (VLM) in spatial reasoning tasks, a decoding method called ADAPTVIS is proposed, which adaptively adjusts the attention distribution based on model confidence, significantly improving spatial reasoning performance.
Widening the Network Mitigates the Impact of Data Heterogeneity on FedAvg
Like Jian (Beihang University), Dong Liu (Beihang University)
Federated LearningConvolutional Neural NetworkImage
🎯 What it does: This paper analyzes and proves that increasing the network width in FedAvg can weaken the impact of data heterogeneity on convergence and generalization, and that FedAvg is equivalent to centralized gradient descent in the case of infinitely wide networks.
WikiBigEdit: Understanding the Limits of Lifelong Knowledge Editing in LLMs
Lukas Thede (University of Tübingen), Thomas Hartvigsen (University of Virginia)
TransformerTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: Proposes the WikiBigEdit benchmark, providing over 500,000 real-time knowledge editing question-answer data based on Wikidata, supporting large-scale lifelong knowledge editing research.
WildChat-50M: A Deep Dive Into the Role of Synthetic Data in Post-Training
Benjamin Feuer (New York University), Chinmay Hegde (New York University)
GenerationData SynthesisTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper constructs the WILDCHAT-50M large synthetic chat dataset and utilizes it for supervised fine-tuning (SFT) experiments on LLMs, evaluating the impact of different generative models (DGM) on post-training performance.
WILTing Trees: Interpreting the Distance Between MPNN Embeddings
Masahiro Negishi (Imperial College), Pascal Welke (Lancaster University)
Explainability and InterpretabilityRepresentation LearningGraph Neural NetworkGraph
🎯 What it does: This study explores the embedding distances learned by Message Passing Neural Networks (MPNN) in practical tasks and approximates them as interpretable graph distances using the interpretable Weisfeiler-Leman Tree (WILT);
Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scale
Rogerio Bonatti (Microsoft), Zheng Hui
TransformerLarge Language ModelAgentic AITextMultimodalityBenchmarkChain-of-Thought
🎯 What it does: Introducing Windows Agent Arena, a scalable multimodal agent benchmark built on a real Windows 11 environment, containing 154 tasks across 11 categories including office, browsing, system, programming, and media.
Winner-takes-all for Multivariate Probabilistic Time Series Forecasting
Adrien Cortes (Institut Polytechnique de Paris), Victor Letzelter (Valeo)
Recurrent Neural NetworkTime SeriesFinance Related
🎯 What it does: The TimeMCL method is proposed, which utilizes multi-head neural networks and Winner-Takes-All (WTA) loss to generate multiple feasible future prediction paths with a single forward pass.
WMAdapter: Adding WaterMark Control to Latent Diffusion Models
Hai Ci (Show Lab National University of Singapore), Mike Zheng Shou (Show Lab National University of Singapore)
GenerationData SynthesisDiffusion modelImage
🎯 What it does: This paper proposes WMAdapter, a plugin that can non-invasively embed any watermark during the generation process of latent diffusion models.
WMarkGPT: Watermarked Image Understanding via Multimodal Large Language Models
Songbai Tan (Shenzhen University), Fei Yu (Guangdong Laboratory of Artificial Intelligence and Digital Economy)
RecognitionObject DetectionSegmentationTransformerLarge Language ModelVision Language ModelImageMultimodality
🎯 What it does: WMarkGPT is proposed, a multimodal large language model specifically designed for watermark image understanding, capable of locating, describing watermarks, and assessing their visibility using only watermark images;
Wolfpack Adversarial Attack for Robust Multi-Agent Reinforcement Learning
Sunwoo Lee (Ulsan National Institute of Science and Technology), Seungyul Han (Ulsan National Institute of Science and Technology)
Adversarial AttackTransformerReinforcement LearningTabular
🎯 What it does: This paper proposes the Wolfpack adversarial attack strategy and its corresponding WALL robust training framework to enhance the reliability of multi-agent reinforcement learning in the face of coordinated attacks.
WOMD-Reasoning: A Large-Scale Dataset for Interaction Reasoning in Driving
Yiheng Li (University of California Berkeley), Wei Zhan
Autonomous DrivingTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelVideoTextMultimodalityChain-of-Thought
🎯 What it does: A large-scale interactive reasoning dataset, WOMD-Reasoning, has been constructed, and Motion-LLaVA has been fine-tuned based on it to enhance the understanding and prediction of traffic rules and human intention-driven interactions.
World Model Implanting for Test-time Adaptation of Embodied Agents
Minjong Yoo (Sungkyunkwan University), Honguk Woo (Sungkyunkwan University)
Domain AdaptationRobotic IntelligenceMeta LearningTransformerLarge Language ModelWorld ModelTextRetrieval-Augmented Generation
🎯 What it does: The WorMI framework is proposed, which enables entity agents to achieve cross-domain adaptation by combining and aligning domain-specific world models with LLM reasoning models during testing.
WorldSimBench: Towards Video Generation Models as World Simulators
Yiran Qin (Sun Yat-sen University), Ruimao Zhang (Guangdong Key Laboratory of Big Data Analysis and Processing)
GenerationAutonomous DrivingRobotic IntelligenceLarge Language ModelSupervised Fine-TuningVideoTextMultimodalityBenchmark
🎯 What it does: A benchmark framework called WorldSimBench is proposed to evaluate the performance of video generation models (S3-level world simulators) in three types of embodied scenarios (Open-Ended Embodied Environment, Autonomous Driving, Robot Manipulation).
Wrapped Gaussian on the manifold of Symmetric Positive Definite Matrices
Thibault de Surrel (PSL University Paris-Dauphine), Florian Yger (INSA Rouen-Normandy)
ClassificationTabular
🎯 What it does: This paper defines a non-isotropic wrapped Gaussian distribution on the Riemannian manifold of symmetric positive definite (SPD) matrices, provides methods for its density, sampling, and maximum likelihood estimation, and applies it to the classification of SPD data, proposing new Ho-WDA and He-WDA discriminant analysis models.
Wyckoff Transformer: Generation of Symmetric Crystals
Nikita Kazeev (Institute for Functional Intelligent Materials University of Singapore), Kedar Hippalgaonkar (Nanyang Technological University)
GenerationTransformerTabularPhysics Related
🎯 What it does: This paper proposes a crystal generation model called WyFormer based on Wyckoff positions, which can directly generate symmetric and stable crystal structures under given space group conditions.
WyckoffDiff -- A Generative Diffusion Model for Crystal Symmetry
Filip Ekström Kelvinius (Linköping University), Fredrik Lindsten (Linköping University)
GenerationData SynthesisGraph Neural NetworkDiffusion modelGraph
🎯 What it does: This paper proposes a discrete diffusion model based on Wyckoff positions, called WyckoffDiff, which can generate prototype structures that satisfy crystal symmetry constraints.
X-Hacking: The Threat of Misguided AutoML
Rahul Sharma (Deutsches Forschungszentrum für Künstliche Intelligenz GmbH), David Antony Selby (Deutsches Forschungszentrum für Künstliche Intelligenz GmbH)
OptimizationExplainability and InterpretabilityAuto EncoderTabular
🎯 What it does: This study proposes the concept of 'X-hacking' and explores how to use AutoML to find models that maintain predictive performance while providing pre-defined explanations (such as SHAP values) within model diversity (Rashomon phenomenon), which can intentionally or unintentionally mislead model interpretations.
X-Transfer Attacks: Towards Super Transferable Adversarial Attacks on CLIP
Hanxun Huang (University of Melbourne), James Bailey (University of Melbourne)
Adversarial AttackTransformerVision Language ModelContrastive LearningImageMultimodality
🎯 What it does: Aiming at CLIP and large-scale visual language models, the X-Transfer attack method is proposed to generate universal adversarial perturbations that can achieve cross-data, cross-domain, cross-model, and cross-task capabilities.
XAttention: Block Sparse Attention with Antidiagonal Scoring
Ruyi Xu (Tsinghua University), Song Han (Massachusetts Institute of Technology)
TransformerVideoTextBenchmark
🎯 What it does: A pluggable block sparse attention framework called XAttention is proposed to accelerate long-context Transformer inference.
XAttnMark: Learning Robust Audio Watermarking with Cross-Attention
Yixin Liu (Lehigh University), Andrea Fanelli (Dolby Laboratories)
Data SynthesisAdversarial AttackTransformerAudio
🎯 What it does: The XAttnMark audio watermarking framework is proposed, achieving reliable watermark detection and accurate message attribution while being resistant to various audio editing and generative attacks.
xLSTM 7B: A Recurrent LLM for Fast and Efficient Inference
Maximilian Beck (NXAI GmbH), Sepp Hochreiter (Johannes Kepler University)
Recurrent Neural NetworkLarge Language ModelText
🎯 What it does: Developed and trained a 7B parameter xLSTM language model, focusing on efficient inference.
You Always Recognize Me (YARM): Robust Texture Synthesis Against Multi-View Corruption
Weihang Ran (University of Tokyo), Yinqiang Zheng (University of Tokyo)
RecognitionData SynthesisOptimizationConvolutional Neural NetworkNeural Radiance FieldImage
🎯 What it does: Using multi-view 3D reconstruction to obtain voxel representation, robust texture optimization is applied to the color channel, significantly improving model recognition accuracy under various image degradation conditions.
You Get What You Give: Reciprocally Fair Federated Learning
Aniket Murhekar (University of Illinois), Ruta Mehta (University of Illinois)
Federated LearningTabularBiomedical Data
🎯 What it does: This paper proposes a budget-balanced payment mechanism M Shap based on the Shapley value to address the free-riding problem caused by self-interested agents in federated learning, and proves that this mechanism has a Nash equilibrium when satisfying revenue concavity and cost convexity.
Zebra: In-Context Generative Pretraining for Solving Parametric PDEs
Louis Serrano (Sorbonne Université), Patrick Gallinari (Sorbonne Université)
GenerationOptimizationComputational EfficiencyTransformerAuto EncoderTime SeriesSequentialPhysics RelatedOrdinary Differential Equation
🎯 What it does: A generative framework called Zebra is proposed, based on VQ-VAE discretization and autoregressive Transformer, to quickly solve parameterized PDEs through context learning without gradient adaptation.
ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoning
Bill Yuchen Lin (University of Washington), Yejin Choi (Stanford University)
TransformerLarge Language ModelPrompt EngineeringTextBenchmarkChain-of-Thought
🎯 What it does: The ZebraLogic framework is proposed, which constructs adjustable complexity evaluation benchmarks based on logic grid puzzles and CSP.
Zero Shot Generalization of Vision-Based RL Without Data Augmentation
Sumeet Batra (University of Southern California), Gaurav S. Sukhatme
Reinforcement LearningAuto EncoderImage
🎯 What it does: A method called ALDA is proposed, which jointly learns decoupled latent representations and associative memory to achieve zero-shot generalization in visual RL without using data augmentation.
Zero-Inflated Bandits
Haoyu Wei (University of California San Diego), Rui Song (Amazon)
OptimizationReinforcement LearningTabularFinance Related
🎯 What it does: This study investigates the application of zero-inflated distributions in multi-armed and contextual bandits, proposing algorithms based on UCB and TS, along with theoretical and experimental validation.
Zero-Shot Adaptation of Parameter-Efficient Fine-Tuning in Diffusion Models
Farzad Farhadzadeh (Qualcomm Technologies), Fatih Porikli (Qualcomm Technologies)
GenerationDomain AdaptationDiffusion modelImage
🎯 What it does: This paper proposes the ProLoRA method, achieving cross-model transfer of LoRA without training or data.
Zero-Shot Cyclic Peptide Design via Composable Geometric Constraints
Dapeng Jiang (Tsinghua University), Yang Liu (Tsinghua University)
GenerationDrug DiscoveryDiffusion modelSequentialBiomedical Data
🎯 What it does: The CP-Composer framework is proposed, which achieves zero-shot cyclic peptide design through composable geometric constraints.
Zero-Shot Generalization of GNNs over Distinct Attribute Domains
Yangyi Shen (Stanford University), Bruno Ribeiro (Purdue University)
Graph Neural NetworkGraph
🎯 What it does: The STAGE model is proposed, which achieves zero-shot generalization of GNNs in unknown attribute domains by mapping node attributes to statistically dependent edge embeddings.
Zero-shot Meta-learning for Tabular Prediction Tasks with Adversarially Pre-trained Transformer
Yulun Wu (University of California), Doron L Bergman
Meta LearningTransformerTabular
🎯 What it does: Designed and trained the Adversarially Pre-trained Transformer (APT), achieving zero-shot meta-learning for tabular prediction tasks without pre-training on real data, and eliminated the limitation on the number of classification labels through mixture blocks.
Zero-Shot Offline Imitation Learning via Optimal Transport
Thomas Rupf (University of Tübingen), Georg Martius (University of Tübingen)
OptimizationRobotic IntelligenceReinforcement LearningWorld ModelSequential
🎯 What it does: This study proposes a zero-shot offline imitation learning method based on Optimal Transport (ZILOT), which directly matches the policy with the occupancy distribution of a single example trajectory, avoiding the short-sighted behavior caused by traditional goal abstraction.
ZeroFlow: Overcoming Catastrophic Forgetting is Easier than You Think
Tao Feng (Tsinghua University), Jie Tang (Tsinghua University)
OptimizationImageBenchmark
🎯 What it does: Designed and implemented the ZeroFlow benchmark to evaluate the effectiveness of methods relying solely on forward propagation (zero-order/forward mode) in catastrophic forgetting scenarios.
ZipAR: Parallel Autoregressive Image Generation through Spatial Locality
Yefei He (Zhejiang University), Bohan Zhuang (Zhejiang University)
GenerationTransformerImage
🎯 What it does: Developed ZipAR, a training-free, pluggable parallel decoding framework to accelerate the generation process of autoregressive visual generation models.
µnit Scaling: Simple and Scalable FP8 LLM Training
Saaketh Narayan (Databricks Mosaic Research), Davis Blalock (Databricks Mosaic Research)
TransformerLarge Language ModelText
🎯 What it does: Proposed the µnit Scaling method, enabling large-scale LLMs to be trained under FP8 computation and supporting hyperparameter transfer without dynamic scaling.