ICML 2024 Papers — Page 7
International Conference on Machine Learning · 2610 papers
Diffusion Rejection Sampling
Byeonghu Na (Korea Advanced Institute of Science and Technology), Il-chul Moon
GenerationData SynthesisKnowledge DistillationDiffusion modelImage
🎯 What it does: This paper proposes Diffusion Rejection Sampling (DiffRS), which corrects the sampling errors of pre-trained diffusion models by performing rejection sampling on the transition kernel of the reverse diffusion process at each time step, and introduces an acceptance probability estimation based on a time-dependent discriminator along with a reinitialization mechanism.
Diffusion Tempering Improves Parameter Estimation with Probabilistic Integrators for Ordinary Differential Equations
Jonas Beck (Hertie Institute for AI in Brain Health University of Tübingen), Philipp Berens (Hertie Institute for AI in Brain Health University of Tübingen)
OptimizationTime SeriesStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: This paper proposes a regularization technique called Diffusion Tempering, aimed at improving the gradient optimization process for ODE parameter estimation in the probabilistic numerical integrator (Fenrir); by gradually reducing the diffusion hyperparameter κ during the optimization process, it achieves a transition from smooth loss to a more accurate ODE solution approximation, significantly enhancing the reliability of parameter estimation.
Diffusion-based Missing-view Generation With the Application on Incomplete Multi-view Clustering
Jie Wen (Harbin Institute of Technology), Yong Xu
GenerationDiffusion modelImageMultimodality
🎯 What it does: This paper studies a missing view generation method based on diffusion models, DMVG, and applies it to incomplete multi-view clustering.
Diffusive Gibbs Sampling
Wenlin Chen (University of Cambridge), David Barber (University College London)
Diffusion modelScore-based Model
🎯 What it does: A new multimodal distribution sampling method called Diffusive Gibbs Sampling (DiGS) is proposed, which performs Gibbs sampling between the original target distribution and its Gaussian convolution distribution, and uses Metropolis-within-Gibbs initialization during the denoising sampling phase.
DiJiang: Efficient Large Language Models through Compact Kernelization
Hanting Chen (Huawei Noah's Ark Lab), Yunhe Wang (Huawei Noah's Ark Lab)
Computational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: The DiJiang method is proposed, transforming the pre-trained Transformer into a linear complexity attention model through frequency domain kernelization and weighted quasi-Monte Carlo sampling, significantly reducing training costs and improving inference speed.
DiNADO: Norm-Disentangled Neurally-Decomposed Oracles for Controlling Language Models
Sidi Lu (University of California), Nanyun Peng (University of California)
GenerationTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: This paper presents DiNADO, an improved Neurally-Decomposed Oracle (NADO) framework that enables controllable text generation by splitting global norms and stepwise reward values.
DiracDiffusion: Denoising and Incremental Reconstruction with Assured Data-Consistency
Zalan Fabian (University of Southern California), Mahdi Soltanolkotabi (University of Southern California)
RestorationDiffusion modelScore-based ModelImageStochastic Differential Equation
🎯 What it does: A framework for solving inverse problems based on a stochastic degradation process is proposed, utilizing reverse diffusion to reconstruct clean images and achieve early stopping and data consistency control;
Directly Denoising Diffusion Models
Dan Zhang (Clemson University), Feng Luo (Clemson University)
RestorationGenerationConvolutional Neural NetworkDiffusion modelImageOrdinary Differential Equation
🎯 What it does: A Direct Denoising Diffusion Model (DDDM) is proposed, achieving high-quality image generation with few steps (1 step, 2 steps) by using the target estimated from the previous round during training and leveraging the noise from the previous step during sampling.
Dirichlet Flow Matching with Applications to DNA Sequence Design
Hannes Stark (Massachusetts Institute of Technology), Tommi Jaakkola (Massachusetts Institute of Technology)
GenerationData SynthesisOptimizationKnowledge DistillationFlow-based ModelSequentialBiomedical Data
🎯 What it does: A flow matching framework based on the Dirichlet distribution is proposed and implemented for generating discrete sequences, particularly DNA gene sequences.
DisCo-Diff: Enhancing Continuous Diffusion Models with Discrete Latents
Yilun Xu (NVIDIA), Karsten Kreis (NVIDIA)
GenerationData SynthesisTransformerDiffusion modelImage
🎯 What it does: A learnable discrete latent variable is added based on the diffusion model, obtained through an encoder, and jointly trained with continuous latent variables, followed by using an autoregressive model to learn the distribution of discrete latent variables;
Discounted Adaptive Online Learning: Towards Better Regularization
Zhiyu Zhang (Harvard University), Heng Yang (Harvard University)
OptimizationReinforcement LearningTime SeriesSequential
🎯 What it does: In a non-stationary environment, the adaptive FTRL algorithm is proposed to solve convex optimization using discounted online learning, and it is applied to online synthetic prediction.
Discovering Bias in Latent Space: An Unsupervised Debiasing Approach
Dyah Adila (University of Wisconsin Madison), Bernie Wang (Amazon Web Services)
TransformerText
🎯 What it does: An unsupervised STEERFAIR method is proposed, which reduces the model's sensitivity to the order of answers by identifying bias directions in the internal representation space of the Transformer and intervening in attention activations during inference.
Discovering Environments with XRM
Mohammad Pezeshki (Meta), David Lopez-Paz (Meta)
Domain AdaptationImage
🎯 What it does: The XRM algorithm is proposed, which utilizes two 'twin' networks trained on randomly partitioned training subsets. By leveraging held-out error label flipping and a cross-error formula, it automatically assigns environments to samples, achieving environment discovery without human labels and promoting OOD generalization.
Discovering Features with Synergistic Interactions in Multiple Views
Chohee Kim (Chung Ang University), Changhee Lee
Biomedical Data
🎯 What it does: This paper proposes a multi-view feature selection framework, SynFS, which can simultaneously discover feature subsets with both synergistic and non-synergistic interaction information, thereby capturing the interactions between different views more comprehensively.
Discovering Mixtures of Structural Causal Models from Time Series Data
Sumanth Varambally (University of California San Diego), Rose Yu (University of California San Diego)
Time SeriesFinance Related
🎯 What it does: Using the variational inference framework MCD to simultaneously learn multiple structural causal models and the corresponding mixed components in time series data, thereby achieving the discovery of mixed causal models.
Discovering Multiple Solutions from a Single Task in Offline Reinforcement Learning
Takayuki Osa (University of Tokyo), Tatsuya Harada (University of Tokyo)
Reinforcement LearningTabular
🎯 What it does: This paper proposes a method for discovering multiple feasible solutions for a single task in offline reinforcement learning through unsupervised learning of latent skill representations, utilizing an EM-style coordinate ascent algorithm.
Discovering Symmetry Breaking in Physical Systems with Relaxed Group Convolution
Rui Wang (Massachusetts Institute of Technology), Tess Smidt
Convolutional Neural NetworkSupervised Fine-TuningTabularTime SeriesPhysics Related
🎯 What it does: Utilize relaxed group convolution to learn and discover symmetry breaking in physical systems.
DISCRET: Synthesizing Faithful Explanations For Treatment Effect Estimation
Yinjun Wu (Peking University), Eric Wong (University of Pennsylvania)
Explainability and InterpretabilityReinforcement LearningImageTextTabular
🎯 What it does: Proposes the DISCRET framework, which generates trustworthy explanations for each sample using logical rules and estimates individual treatment effects (ITE) accordingly.
Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution
Aaron Lou (Stanford University), Stefano Ermon (Stanford University)
GenerationData SynthesisTransformerDiffusion modelScore-based ModelText
🎯 What it does: The Score Entropy Discrete Diffusion model (SEDD) is proposed, which achieves the diffusion generation of discrete data by learning the probability ratios of discrete distributions.
Discrete Latent Perspective Learning for Segmentation and Detection
Deyi Ji (University of Science and Technology of China), Jieping Ye (Alibaba Group)
Object DetectionSegmentationTransformerSupervised Fine-TuningImage
🎯 What it does: This paper proposes a discrete latent perspective learning framework (DLPL) based on single-view images to enhance semantic consistency in segmentation and detection tasks across different perspectives.
Disentangled 3D Scene Generation with Layout Learning
Dave Epstein (University of California Berkeley), Aleksander Holynski (Google Research)
GenerationData SynthesisDiffusion modelScore-based ModelNeural Radiance FieldImageText
🎯 What it does: Using layout learning techniques, generate and decompose 3D scenes into independent objects under unsupervised conditions;
Disentangled Continual Graph Neural Architecture Search with Invariant Modular Supernet
Zeyang Zhang (Tsinghua University), Wenwu Zhu (Tsinghua University)
OptimizationNeural Architecture SearchGraph Neural NetworkGraph
🎯 What it does: The GASIM method is proposed, utilizing a modular hypernetwork, graph factor routers, and an invariance search mechanism to achieve continuous graph neural network architecture search, adapting to emerging new graph tasks and preventing forgetting.
Disentangled Graph Self-supervised Learning for Out-of-Distribution Generalization
Haoyang Li (Tsinghua University), Wenwu Zhu (Tsinghua University)
Domain AdaptationRepresentation LearningGraph Neural NetworkContrastive LearningGraphBenchmark
🎯 What it does: A self-supervised out-of-distribution (OOD) generalization framework OOP-GCL is proposed, which achieves the decoupling of graph representations through a multi-channel graph encoder and factor-level contrastive learning, enabling cross-distribution generalization in an unlabeled environment.
Disentanglement Learning via Topology
Nikita Balabin (Skolkovo Institute of Science and Technology), Serguei Barannikov (CNRS IMJ Paris Cite University)
GenerationRepresentation LearningAuto EncoderGenerative Adversarial NetworkImage
🎯 What it does: The TopDis method is proposed, which promotes unsupervised separable representation learning by incorporating a differentiable multi-scale topological loss RTD into VAE training.
Disguised Copyright Infringement of Latent Diffusion Models
Yiwei Lu (University of Waterloo), Yaoliang Yu (University of Waterloo)
GenerationData SynthesisAdversarial AttackDiffusion modelImage
🎯 What it does: This study investigates how to use steganography to hide copyrighted images in the training data of potential diffusion models (LDM) and demonstrates that these hidden images can lead to the model replicating copyrighted content during inference, thereby achieving indirect copyright infringement.
Disparate Impact on Group Accuracy of Linearization for Private Inference
Saswat Das (University of Virginia), Ferdinando Fioretto (University of Virginia)
ClassificationRecognitionSafty and PrivacyKnowledge DistillationConvolutional Neural NetworkImage
🎯 What it does: The study investigates the impact of ReLU linearization on model fairness when implementing privacy inference in neural networks and proposes a mitigation strategy that incorporates fairness regularization during the fine-tuning phase.
Dissecting Multimodality in VideoQA Transformer Models by Impairing Modality Fusion
Ishaan Singh Rawal (Agency for Science Technology and Research), Cheston Tan (Agency for Science Technology and Research)
TransformerVideoTextMultimodality
🎯 What it does: The QUAG tool is designed for non-parametric evaluation of multimodal fusion in the VideoQA Transformer model, and QUAG-attention is proposed as a replacement for self-attention to achieve efficient inference.
Distilling Morphology-Conditioned Hypernetworks for Efficient Universal Morphology Control
Zheng Xiong (University of Oxford), Shimon Whiteson (University of Oxford)
Computational EfficiencyKnowledge DistillationRobotic IntelligenceTransformerReinforcement LearningAgentic AISequential
🎯 What it does: The HyperDistill method is proposed, which combines morphological conditional supernetworks with policy distillation, achieving high efficiency of MLP during inference while maintaining the performance of Transformers in general robotic control models.
DistiLLM: Towards Streamlined Distillation for Large Language Models
Jongwoo Ko (KAIST), Se-Young Yun (KAIST)
CompressionComputational EfficiencyKnowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Achieving compression of large language models through knowledge distillation.
Distinguishing the Knowable from the Unknowable with Language Models
Gustaf Ahdritz (Harvard University), Benjamin L. Edelman (Harvard University)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This study investigates how to distinguish between cognitive uncertainty and random uncertainty in the outputs of language models, using larger models as reference to label the uncertainty of smaller models.
Distributed Bilevel Optimization with Communication Compression
Yutong He (Peking University), Kun Yuan (Peking University)
OptimizationHyperparameter SearchImage
🎯 What it does: This study investigates the communication compression problem in bi-level optimization within a distributed environment and proposes a series of compression algorithms based on the SOBA framework.
Distributed High-Dimensional Quantile Regression: Estimation Efficiency and Support Recovery
Caixing Wang (Shanghai University of Finance and Economics), Ziliang Shen (Shanghai University of Finance and Economics)
OptimizationTabular
🎯 What it does: This study investigates the estimation efficiency and support recovery of distributed high-dimensional quantile regression, proposing a distributed algorithm based on double smoothing.
Distribution Alignment Optimization through Neural Collapse for Long-tailed Classification
Jintong Gao (Jilin University), Hongyuan Zha (Chinese University of Hong Kong)
ClassificationOptimizationConvolutional Neural NetworkImage
🎯 What it does: By matching the distribution of the model's final layer features with the distribution of the Equiangular Tight Frame (ETF) structure, a Distribution Alignment Optimization (DisA) regularization method is proposed to promote the convergence of neural networks to the Neural Collapse (NC) state in long-tail classification.
Distributional Bellman Operators over Mean Embeddings
Li Kevin Wenliang (Google DeepMind), Mark Rowland (Google DeepMind)
Reinforcement Learning
🎯 What it does: A distributed reinforcement learning framework based on finite-dimensional mean embedding has been developed, introducing the 'Sketch Bellman Operator' along with its corresponding dynamic programming (Sketch-DP) and temporal difference (Sketch-TD) algorithms, which update entirely within the sketch space.
Distributionally Robust Data Valuation
Xiaoqiang Lin (National University of Singapore), Bryan Kian Hsiang Low (National University of Singapore)
Data-Centric LearningConvolutional Neural NetworkNeural Radiance FieldImageTabular
🎯 What it does: A distributionally robust data value assessment method that does not rely on a fixed validation set is proposed, utilizing distributionally robust generalization error (DRGE) and model deviation to measure the contribution of each sample.
DITTO: Diffusion Inference-Time T-Optimization for Music Generation
Zachary Novack (University of California San Diego), Nicholas J. Bryan (Adobe Research)
GenerationOptimizationDiffusion modelAudio
🎯 What it does: This paper presents DITTO, a method that achieves various fine-grained controls (such as illustration, extension, repetition, intensity, melody, and structure) for a pre-trained text-to-music diffusion model during the inference phase by optimizing the initial noise latent vector and using any differentiable feature matching loss without additional training.
Ditto: Quantization-aware Secure Inference of Transformers upon MPC
Haoqi Wu (Ant Group), Lei Wang (Ant Group)
Computational EfficiencyKnowledge DistillationTransformerLarge Language ModelText
🎯 What it does: The Ditto framework is proposed to achieve secure Transformer inference with quantized perception, significantly reducing computational and communication overhead by combining MPC technology.
Diversified Batch Selection for Training Acceleration
Feng Hong (Shanghai Jiao Tong University), Yanfeng Wang (Shanghai AI Laboratory)
OptimizationComputational EfficiencyImageMultimodality
🎯 What it does: A no-reference model batch selection method called DivBS is proposed, which accelerates training by optimizing the overall orthogonal representation of the subset.
Diving into Underwater: Segment Anything Model Guided Underwater Salient Instance Segmentation and A Large-scale Dataset
Shijie Lian (Hainan University), Runmin Cong (Shandong University)
Object DetectionSegmentationTransformerSupervised Fine-TuningImage
🎯 What it does: Constructed the first underwater salient instance segmentation dataset USIS10K and proposed the USIS-SAM model based on SAM.
DMTG: One-Shot Differentiable Multi-Task Grouping
Yuan Gao (Wuhan University), Gui-Song Xia (Wuhan University)
Convolutional Neural NetworkTransformerImage
🎯 What it does: A differentiable multi-task grouping (DMTG) method is proposed, which simultaneously completes task grouping and model learning during the training process;
DNA-SE: Towards Deep Neural-Nets Assisted Semiparametric Estimation
Qinshuo Liu (University of Hong Kong), Zhonghua Liu (Columbia University)
OptimizationTabular
🎯 What it does: This paper proposes the DNA SE algorithm, which utilizes deep neural networks as numerical solvers to transform the semiparametric estimation problem into a bi-level optimization problem, achieving synchronous estimation of parameters and integral equations.
DNCs Require More Planning Steps
Yara Shamshoum (Technion Israel Institute of Technology), Assaf Schuster (Technion Israel Institute of Technology)
Graph Neural NetworkReinforcement LearningGraph
🎯 What it does: This paper studies the impact of planning budget on algorithm learning and generalization in Differentiable Neural Computer (DNC), and proposes three improvement schemes: adaptive planning budget, temperature reweighting, and stochastic planning budget regularization.
Do Efficient Transformers Really Save Computation?
Kai Yang (Peking University), Liwei Wang (Peking University)
OptimizationComputational EfficiencyTransformerSequential
🎯 What it does: Analyzed and evaluated the expressive power and computational complexity of Sparse Transformer and Linear Transformer in dynamic programming inference tasks, exploring whether they truly achieve efficiency improvements.
Do Language Models Exhibit the Same Cognitive Biases in Problem Solving as Human Learners?
Andreas Opedal (ETH Zurich), Mrinmaya Sachan (ETH Zurich)
TransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: This paper investigates whether large language models (LLMs) exhibit cognitive biases similar to those of children during the process of solving arithmetic math puzzles, systematically designing and evaluating consistency bias, transfer vs. comparison bias, and carry effect.
Do Large Code Models Understand Programming Concepts? Counterfactual Analysis for Code Predicates
Ashish Hooda (University of Wisconsin Madison), Somesh Jha (University of Wisconsin Madison)
AI Code AssistantLarge Language ModelText
🎯 What it does: The CACP (Counterfactual Analysis for Programming Concept Predicates) framework is proposed, which evaluates whether large-scale code language models understand programming concepts by performing controllable minimal variations (counterfactuals) on the code.
Do Large Language Models Perform the Way People Expect? Measuring the Human Generalization Function
Keyon Vafa (Harvard University), Sendhil Mullainathan (Massachusetts Institute of Technology)
TransformerLarge Language ModelText
🎯 What it does: The study investigates how humans infer the capabilities of LLMs through minimal interactions on other questions, constructing 19,000 pieces of human inductive data to evaluate the consistency between LLMs and human expectations.
Do Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanations
Yanda Chen (Columbia University), Kathleen McKeown (Columbia University)
Explainability and InterpretabilityTransformerLarge Language ModelTextChain-of-Thought
🎯 What it does: This paper studies whether natural language explanations generated by large language models (LLMs) can help humans build correct mental models and proposes an evaluation framework for 'counterfactual simulability';
Do Topological Characteristics Help in Knowledge Distillation?
Jungeun Kim (Yonsei University), Jae-Hun Jung (POSTECH)
Knowledge DistillationImage
🎯 What it does: A knowledge distillation method based on global topological features, TopKD, is proposed, which utilizes the persistent homology information of the feature space from the teacher network to guide the student network's learning.
Do Transformer World Models Give Better Policy Gradients?
Michel Ma (Mila), Pierre-Luc Bacon (Mila)
OptimizationTransformerReinforcement LearningWorld ModelSequential
🎯 What it does: This paper proposes an action-conditioned world model (AWM) that improves gradient-based policy optimization by mapping the world model solely as a sequence of actions.
Does Label Smoothing Help Deep Partial Label Learning?
Xiuwen Gong (University of Technology Sydney), Guandong Xu (Polytechnic University)
ClassificationOptimizationConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: This paper studies the role of label smoothing in deep partial label learning (PLL), proposes theoretical risk bounds, derives the optimal smoothing rate, and presents the LS-PLL algorithm based on this theory.
DOGE: Domain Reweighting with Generalization Estimation
Simin Fan (École Polytechnique Fédérale de Lausanne), Martin Jaggi (École Polytechnique Fédérale de Lausanne)
Domain AdaptationOptimizationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: The DOGE method is proposed, which first trains a small proxy model to obtain domain weights, and then uses these weights to resample during the pre-training of a large model, thereby enhancing the generalization ability of LLM in the target domain.
Domain Generalisation via Imprecise Learning
Anurag Singh (Saarland University), Krikamol Muandet (University of Oxford)
Domain AdaptationOptimizationTabular
🎯 What it does: A domain generalization framework that maintains uncertainty in generalization strategies during the training phase (Imprecise Domain Generalisation) is proposed, along with a corresponding Imprecise Risk Optimisation (IRO) algorithm.
Domain-wise Data Acquisition to Improve Performance under Distribution Shift
Yue He (Tsinghua University), Peng Cui (Tsinghua University)
Domain AdaptationSupervised Fine-TuningImage
🎯 What it does: This paper proposes a data collection framework based on Domain-wise Active Acquisition (DAA) to enhance the generalization performance of models in the target test domain by dynamically allocating a limited sampling budget across multiple domains in the context of distribution shift.
Don't be so Negative! Score-based Generative Modeling with Oracle-assisted Guidance
Saeid Naderiparizi (University of British Columbia), Frank Wood (University of British Columbia)
GenerationData SynthesisKnowledge DistillationDiffusion modelScore-based ModelImage
🎯 What it does: This paper studies a method for improving score-based diffusion models under the presence of oracle constraints.
Don’t Label Twice: Quantity Beats Quality when Comparing Binary Classifiers on a Budget
Florian E. Dorner (Max Planck Institute for Intelligent Systems), Moritz Hardt (Max Planck Institute for Intelligent Systems)
ClassificationOptimization
🎯 What it does: The study investigates how to optimally allocate labels under a limited labeling budget to compare the accuracy of binary classifiers.
Don't trust your eyes: on the (un)reliability of feature visualizations
Robert Geirhos (Google DeepMind), Been Kim (Google DeepMind)
Explainability and InterpretabilityConvolutional Neural NetworkImage
🎯 What it does: The reliability of feature visualization is studied, proposing the deceivable 'fooling circuit' and 'silent units', along with an empirical sanity check and theoretical proof;
DoRA: Weight-Decomposed Low-Rank Adaptation
Shih-yang Liu, Min-Hung Chen (NVIDIA)
TransformerLarge Language ModelSupervised Fine-TuningVision Language ModelMultimodality
🎯 What it does: The DoRA method is proposed, which splits the pre-trained weights into two parts: magnitude and direction, applying low-rank updates only to the direction using LoRA, thereby enhancing the learning capability of PEFT while maintaining no inference overhead.
DoraemonGPT: Toward Understanding Dynamic Scenes with Large Language Models (Exemplified as A Video Agent)
Zongxin Yang (Zhejiang University), Yi Yang (Zhejiang University)
Object DetectionObject TrackingSegmentationRetrievalTransformerLarge Language ModelAgentic AIVideoText
🎯 What it does: This paper proposes DoraemonGPT, a multi-tool video agent that utilizes large language models (LLM) and symbolic memory for spatial-temporal reasoning, knowledge retrieval, and task planning in dynamic videos.
Double Momentum Method for Lower-Level Constrained Bilevel Optimization
Wanli Shi (Jilin University), Bin Gu (Jilin University)
OptimizationMeta LearningTabular
🎯 What it does: A dual momentum single-loop single time-scale algorithm is proposed for solving bilevel optimization problems with lower-level constraints;
Double Stochasticity Gazes Faster: Snap-Shot Decentralized Stochastic Gradient Tracking Methods
Hao Di (Xi'an Jiaotong University), Ivor Tsang
OptimizationTabular
🎯 What it does: This paper proposes two novel decentralized stochastic gradient descent algorithms based on snap-shot gradient tracking: SS DSGT and ASS DSGT, and provides an analysis of their iterative complexity under general communication networks.
Double Variance Reduction: A Smoothing Trick for Composite Optimization Problems without First-Order Gradient
Hao Di (Xi'an Jiaotong University), Ivor Tsang
OptimizationGaussian SplattingTabular
🎯 What it does: For the problem of strongly convex finite sums with non-smooth regularization, a dual variance reduction method ZPDVR is proposed, which only uses stochastic gradient estimates and requires only one zero-order gradient query per step, achieving linear convergence.
Double-Step Alternating Extragradient with Increasing Timescale Separation for Finding Local Minimax Points: Provable Improvements
Kyuwon Kim (KAIST), Donghwan Kim (KAIST)
OptimizationOrdinary Differential Equation
🎯 What it does: A dual-step alternating limit gradient method (Alt2-EG) is proposed, which seeks non-strict local minimax points through dual-step alternating updates and increased time scale separation.
Doubly Robust Causal Effect Estimation under Networked Interference via Targeted Learning
Weilin Chen (Guangdong University of Technology), Zhifeng Hao (Shantou University)
Graph Neural NetworkGraph
🎯 What it does: A dual robust causal effect estimation method TNet is proposed for network interference scenarios, utilizing the targeted learning (TMLE) framework to achieve collaborative optimization of the entire network's potential outcomes and propensity scores.
DPN: Decoupling Partition and Navigation for Neural Solvers of Min-max Vehicle Routing Problems
Zhi Zheng (Southern University of Science and Technology), Ke Tang (Southern University of Science and Technology)
OptimizationTransformerReinforcement Learning
🎯 What it does: A reinforcement learning-based sequential planning framework named DPN is proposed to solve the min-max vehicle routing problem, with the core idea of decoupling customer allocation (partition) and route navigation (navigation) and modeling them separately.
DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Training
Zhongkai Hao (Tsinghua University), Jun Zhu (Tsinghua University)
OptimizationComputational EfficiencyData-Centric LearningTransformerTime SeriesSequentialBenchmarkPhysics Related
🎯 What it does: Designed and implemented DPOT - a self-regressive denoising pre-trained Fourier attention transformer, used to learn universal solvers from PDE data of various dimensions, scales, and geometries.
DPZero: Private Fine-Tuning of Language Models without Backpropagation
Liang Zhang (ETH Zurich), Niao He
OptimizationSafty and PrivacyTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper studies and implements a zero-order fine-tuning method with differential privacy without using backpropagation, enabling efficient and private fine-tuning on large language models.
Dr. Strategy: Model-Based Generalist Agents with Strategic Dreaming
Hany Hamed (KAIST), Sungjin Ahn (KAIST)
Robotic IntelligenceReinforcement LearningWorld ModelImage
🎯 What it does: A model-based general agent, Dr. Strategy, is proposed, utilizing potential landmarks and a divide-and-conquer 'strategic dream' to enhance sample efficiency in exploration and goal achievement.
DRCT: Diffusion Reconstruction Contrastive Training towards Universal Detection of Diffusion Generated Images
Baoying Chen (Alibaba Group), Rui Yang (Shenzhen Campus of Sun Yat-sen University)
Object DetectionGenerationDiffusion modelContrastive LearningImage
🎯 What it does: This paper proposes a general framework called Diffusion Reconstruction Contrastive Training (DRCT) to enhance the generalization ability of detectors for images generated by diffusion models.
DRED: Zero-Shot Transfer in Reinforcement Learning via Data-Regularised Environment Design
Samuel Garcin (University of Edinburgh), Stefano V Albrecht
Reinforcement LearningAuto EncoderSequentialBenchmark
🎯 What it does: This study investigates the zero-shot transfer capability of RL agents under different sampling strategies and proposes the DRED framework, which combines adaptive sampling with level generation to reduce the mutual information between internal representations and level identities, thereby enhancing generalization performance.
Drug Discovery with Dynamic Goal-aware Fragments
Seul Lee (Korea Advanced Institute of Science and Technology), Sung Ju Hwang (Korea Advanced Institute of Science and Technology)
Drug DiscoveryGraph Neural NetworkReinforcement LearningGraph
🎯 What it does: A drug discovery generation framework based on target-aware fragments, GEAM, is proposed, which includes three main modules: fragment extraction, assembly, and modification.
DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoning
Siyuan Guo (Jilin University), Jun Wang (University College London)
TransformerLarge Language ModelTextTabularTime SeriesRetrieval-Augmented Generation
🎯 What it does: By combining case-based reasoning (CBR) with large language models, we automate the generation and training of machine learning models that meet task requirements.
DSD-DA: Distillation-based Source Debiasing for Domain Adaptive Object Detection
Yongchao Feng (Beihang University), Yunhong Wang (Beihang University)
Object DetectionDomain AdaptationKnowledge DistillationGenerative Adversarial NetworkImage
🎯 What it does: This paper proposes a source domain debiasing framework based on knowledge distillation (DSD), combined with a Target-Relevant Object Localization Network (TROLN) and a Consistency Enhancement Strategy (DCE), to improve the performance of unsupervised domain adaptation object detection models in the target domain.
DsDm: Model-Aware Dataset Selection with Datamodels
Logan Engstrom (Massachusetts Institute of Technology), Aleksander Madry (Massachusetts Institute of Technology)
OptimizationData-Centric LearningTextBenchmark
🎯 What it does: A data model-based training data selection method DSDM is proposed, which directly optimizes the model performance of the target task.
Dual Operating Modes of In-Context Learning
Ziqian Lin (University of Wisconsin Madison), Kangwook Lee (University of Wisconsin Madison)
TransformerLarge Language ModelText
🎯 What it does: A pre-trained data probability model based on Gaussian mixture models is proposed, analyzing its impact on in-context learning (ICL) dual modes (task retrieval and task learning), and using this model to explain the observed phenomena of 'early rise' and 'bounded effects of biased label ICL', followed by experimental validation on various large language models.
DUPLEX: Dual GAT for Complex Embedding of Directed Graphs
Zhaoru Ke (Ant Group), Haipeng Zhang (ShanghaiTech University)
ClassificationRepresentation LearningGraph Neural NetworkGraph
🎯 What it does: This paper proposes a complex embedding framework for directed graphs called DUPLEX, which utilizes a dual GAT encoder to generate complex embeddings consisting of amplitude and phase components. It reconstructs the Hermitian adjacency matrix through two non-parametric decoders in a self-supervised manner, achieving high-quality representations for low-degree nodes, generalizability to new nodes, and task-independent embeddings.
Dynamic Anisotropic Smoothing for Noisy Derivative-Free Optimization
Sam Reifenstein (NTT Research Inc), Yoshihisa Yamamoto (NTT Research Inc)
OptimizationStochastic Differential Equation
🎯 What it does: A dynamic anisotropic smoothing (DAS) algorithm is proposed for gradient-free optimization in environments with noise and unavailable gradients, and it is applied to parameter tuning of combinatorial optimization heuristic solvers.
Dynamic Byzantine-Robust Learning: Adapting to Switching Byzantine Workers
Ron Dorfman (Technion), Kfir Yehuda Levy
OptimizationFederated LearningConvolutional Neural NetworkImage
🎯 What it does: This paper presents DynaBRO, a robust optimization algorithm that maintains convergence in a distributed learning environment with Byzantine workers whose identities change over time.
Dynamic Correlation Clustering in Sublinear Update Time
Vincent Cohen-Addad (Google Research), Nikos Parotsidis (Google Research)
GraphTabular
🎯 What it does: A relevant clustering algorithm that maintains constant approximation in dynamic node streams is proposed, with an update time of polynomial logarithmic order;
Dynamic Evaluation of Large Language Models by Meta Probing Agents
Kaijie Zhu (Microsoft Research), Xing Xie (Microsoft Research)
Large Language ModelAgentic AIPrompt EngineeringTextBenchmark
🎯 What it does: This paper proposes Meta Probing Agents (MPA) — a dynamic evaluation protocol based on agents that automatically generates new assessment questions and judgments through LLM agents, achieving a fine-grained evaluation of the multidimensional cognitive abilities of large models.
Dynamic Facility Location in High Dimensional Euclidean Spaces
Sayan Bhattacharya (University of Warwick), Yubo Zhang (Peking University)
OptimizationReinforcement LearningTabular
🎯 What it does: This paper proposes an algorithm for fully dynamic facility location problems in high-dimensional Euclidean space, which can maintain an approximately optimal facility set during the insertion/deletion of points.
Dynamic Memory Compression: Retrofitting LLMs for Accelerated Inference
Piotr Nawrot (University of Edinburgh), Edoardo Ponti
CompressionComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: A dynamic memory compression (DMC) mechanism has been designed and implemented, which can adaptively compress the KV cache during LLM inference, significantly improving memory utilization and inference throughput.
Dynamic Metric Embedding into lp Space
Kiarash Banihashem (University of Maryland), Max Springer (University of Maryland)
Graph
🎯 What it does: The paper presents the first low-distortion embedding algorithm that maps weighted undirected graphs to ℓp space in a weight-increasing (decremental) dynamic environment.
Dynamic Spectral Clustering with Provable Approximation Guarantee
Steinar Laenen (University of Edinburgh), He Sun (University of Edinburgh)
Graph Neural NetworkGraph
🎯 What it does: This paper studies clustering algorithms for dynamic evolving graphs and proposes a dynamic spectral clustering algorithm that can effectively approximate the clustering structure in the final state of the graph.
Dynamic Survival Analysis with Controlled Latent States
Linus Bleistein (Inria Paris), Agathe Guilloux (Centre de Recherche des Cordeliers)
Time SeriesSequentialStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Using Controlled Differential Equations (CDE) to model individual-specific counting process intensity, two estimators are proposed: one based on Neural Controlled Differential Equations (NCDE) and the other based on signature (CoxSig).
DynSyn: Dynamical Synergistic Representation for Efficient Learning and Control in Overactuated Embodied Systems
Kaibo He (Tsinghua University), Yanan Sui (Tsinghua University)
OptimizationRobotic IntelligenceReinforcement Learning
🎯 What it does: This paper proposes a dynamic collaborative representation (DynSyn) algorithm based on dynamic structure for efficient learning and control of high-dimensional over-actuated systems.
DySLIM: Dynamics Stable Learning by Invariant Measure for Chaotic Systems
Yair Schiff (Cornell Tech), Leonardo Zepeda-Núñez (Google Research)
Time SeriesPhysics RelatedOrdinary Differential Equation
🎯 What it does: A new framework DySLIM is proposed, which regularizes the learning dynamics model to approximate the true dynamics while preserving invariant measures, thus achieving the dual goals of short-term tracking and long-term statistical stability.
E$^2$GAN: Efficient Training of Efficient GANs for Image-to-Image Translation
Yifan Gong (Snap Inc), Jian Ren (Snap Inc)
Image TranslationComputational EfficiencyKnowledge DistillationTransformerDiffusion modelGenerative Adversarial NetworkImage
🎯 What it does: This paper develops EGAN2, which uses diffusion models to generate paired data for distillation. By implementing three main strategies: base model + LoRA + data clustering, it achieves real-time, low-latency image translation on mobile devices while significantly reducing training costs and storage requirements.
EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty
Yuhui Li (Peking University), Hongyang Zhang (University of Waterloo)
TransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: A new EAGLE framework is proposed, which achieves lossless explicit sampling acceleration by performing autoregressive prediction at the feature layer and incorporating a one-step ahead token as input.
Early Time Classification with Accumulated Accuracy Gap Control
Liran Ringel (Technion Israel Institute of Technology), Yaniv Romano (Technion Israel Institute of Technology)
ClassificationOptimizationTextTime Series
🎯 What it does: This paper proposes a statistical framework that utilizes data-driven stopping rules to achieve controlled accuracy differences in early time classification.
Easing Concept Bleeding in Diffusion via Entity Localization and Anchoring
Jiewei Zhang (Hong Kong Polytechnic University), Xiao-Ming Wu (Hong Kong Polytechnic University)
GenerationData SynthesisDiffusion modelImageText
🎯 What it does: This paper proposes an entity localization and anchoring (ELA) method that does not require training. It utilizes the cross-attention map of the diffusion model to locate concept leakage (entity bleeding) in multi-entity generation during the inference phase and constrains the latent representation with local loss, allowing entities to focus on the expected areas separately, thus avoiding overlap, loss, and attribute binding errors.
eCeLLM: Generalizing Large Language Models for E-commerce from Large-scale, High-quality Instruction Data
Bo Peng (Ohio State University), Xia Ning (Ohio State University)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper constructs a large-scale, real, and high-quality annotated e-commerce instruction dataset called ECInstruct, and trains a series of general e-commerce LLMs (eCeLLM) through instruction tuning on it.
ED-Copilot: Reduce Emergency Department Wait Time with Language Model Diagnostic Assistance
Liwen Sun (Carnegie Mellon University), Chenyan Xiong (Carnegie Mellon University)
Recommendation SystemOptimizationTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTabularBiomedical DataElectronic Health RecordsBenchmark
🎯 What it does: This paper develops ED-Copilot, an emergency diagnostic assistance system based on a pre-trained medical language model, which can dynamically recommend the most informative test groups based on patient triage information and completed laboratory tests, and predict the risk of critical illness and prolonged emergency department stay, thereby significantly reducing the waiting time for emergency patients.
EDISON: Enhanced Dictionary-Induced Tensorized Incomplete Multi-View Clustering with Gaussian Error Rank Minimization
Zhibin Gu (Beijing Jiaotong University), Songhe Feng (Beijing Jiaotong University)
OptimizationMultimodality
🎯 What it does: This paper proposes a model named EDISON, which utilizes three innovations: enhanced dictionary representation, Gaussian Error Rank, and hyper-anchor graph Laplacian regularization to address the issues of missing data, low-rank representation, and local geometric information loss in incomplete multi-view clustering.
Editing Partially Observable Networks via Graph Diffusion Models
Puja Trivedi (University of Michigan), Danai Koutra (University of Michigan)
Graph Neural NetworkDiffusion modelGraph
🎯 What it does: Proposes a subgraph diffusion model (SGDM) that supports three types of editing tasks: denoising, expansion, and style transfer for a single partially observable network.
EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelism
Yanxi Chen (Alibaba Group), Jingren Zhou (Alibaba Group)
Computational EfficiencyTransformerLarge Language ModelText
🎯 What it does: The EE-LLM framework is proposed, supporting early exit large language models for large-scale training and inference, and achieving 3D parallelism on Megatron-LM.
Effective Federated Graph Matching
Yang Zhou (Auburn University), Wei-Shinn Ku (Sony AI)
OptimizationFederated LearningSafty and PrivacyGraph Neural NetworkGraph
🎯 What it does: An unsupervised federated graph matching algorithm UFGM is proposed, which generates pseudo training data using graph features and achieves privacy-preserving graph node matching through separated trust region optimization.
Effects of Exponential Gaussian Distribution on (Double Sampling) Randomized Smoothing
Youwei Shu (Shenzhen International Graduate School, Tsinghua University), Bo Li (University of Illinois Urbana-Champaign)
ClassificationOptimizationImage
🎯 What it does: This study investigates the impact of Exponential Gaussian Distribution (ESG and EGG) on Random Smoothing (RS) and Double Sampling Random Smoothing (DSRS), and provides the analytical radius formula for ESG as well as the effect of EGG in mitigating the curse of dimensionality in DSRS.
Efficient Adaptation in Mixed-Motive Environments via Hierarchical Opponent Modeling and Planning
Yizhe Huang (Peking University), Xue Feng (Peking University)
Robotic IntelligenceReinforcement LearningAgentic AISequential
🎯 What it does: A hierarchical opponent modeling and planning algorithm HOP is proposed to achieve rapid adaptation to unknown opponents in mixed-motivation environments.
Efficient Algorithms for Empirical Group Distributionally Robust Optimization and Beyond
Dingzhi Yu (Nanjing University), Lijun Zhang (Nanjing University)
OptimizationTabular
🎯 What it does: Two efficient algorithms are proposed and implemented: ALEG for solving the Empirical Group Distributionally Robust Optimization (GDRO) problem, and ALEM for solving the Empirical Min-Max Risk Optimization (MERO) problem.
Efficient Algorithms for Sum-Of-Minimum Optimization
Lisang Ding (University of California), Wotao Yin (Alibaba US)
OptimizationTabular
🎯 What it does: A general 'sum-of-minimum' optimization model is proposed, and an efficient solving method based on k-means++ initialization and the Lloyd algorithm is designed for this model.