ICML 2023 Papers — Page 5
International Conference on Machine Learning · 1828 papers
Differentially Private Hierarchical Clustering with Provable Approximation Guarantees
Jacob Imola (University of California San Diego), Vahab Mirrokni (Google)
OptimizationSafty and PrivacyGraph
🎯 What it does: A hierarchical clustering algorithm under differential privacy is proposed, along with provable approximation guarantees.
Differentially Private Optimization on Large Model at Small Cost
Zhiqi Bu (Amazon Web Services), George Karypis (University of Minnesota)
OptimizationSafty and PrivacyComputational EfficiencyTransformerLarge Language ModelImageText
🎯 What it does: The Book-Keeping (BK) algorithm is proposed, achieving nearly the same time and memory overhead as non-private training through a single round of backpropagation in differential privacy training.
Differentially Private Sharpness-Aware Training
Jinseong Park (Seoul National University), Jaewook Lee (Seoul National University)
OptimizationSafty and PrivacyTransformerSupervised Fine-TuningImage
🎯 What it does: This paper studies the effects of finding flat minima in differential privacy training and proposes a sharpness-aware training method, DP-SAT, that does not increase privacy overhead or computational cost.
Differentially Private Stochastic Convex Optimization under a Quantile Loss Function
Du Chen (Nanyang Technological University), Geoffrey A. Chua (Nanyang Technological University)
OptimizationSafty and PrivacyTabular
🎯 What it does: This paper studies differential privacy stochastic convex optimization under quantized loss functions, proposing a convolution smoothing method and designing a private algorithm based on DP-SGD and target perturbation.
Diffusion Based Representation Learning
Sarthak Mittal (Mila), Arash Mehrjou (Max Planck Institute for Intelligent Systems)
Representation LearningDiffusion modelScore-based ModelAuto EncoderContrastive LearningImage
🎯 What it does: A framework for unsupervised representation learning based on continuous-time diffusion models (DRL) is proposed, which achieves conditional denoising of data and learns latent representations that can be directly used for downstream tasks by incorporating latent encoding into the denoising score matching objective.
Diffusion Models are Minimax Optimal Distribution Estimators
Kazusato Oko (University of Tokyo), Taiji Suzuki (University of Tokyo)
GenerationData SynthesisDiffusion modelScore-based Model
🎯 What it does: Analyzes the statistical learning theory of diffusion models (score-based generative models) as distribution estimators, and proves that they can achieve approximately optimal estimation rates in terms of total variation and Wasserstein distance in Besov spaces;
Diffusion Models as Artists: Are we Closing the Gap between Humans and Machines?
Victor Boutin (Toulouse Institute), Thomas Serre (Brown University)
GenerationDiffusion modelContrastive LearningImage
🎯 What it does: This study investigates a one-shot drawing task, comparing the performance of humans and various generative models (VAE, GAN, diffusion models) in terms of drawing diversity and recognizability, and proposes originality metrics and generalization curves for more granular evaluation.
Diffusion Models for Black-Box Optimization
Siddarth Krishnamoorthy (University of California Los Angeles), Aditya Grover (University of California Los Angeles)
OptimizationDiffusion modelStochastic Differential Equation
🎯 What it does: A method for offline black-box optimization based on conditional diffusion models is proposed—DDOM, which directly learns a one-to-many mapping from function values to input space, achieving offline optimization.
Dimension-independent Certified Neural Network Watermarks via Mollifier Smoothing
Jiaxiang Ren (Auburn University), Da Yan (University of Alabama at Birmingham)
ClassificationComputational EfficiencyAdversarial AttackConvolutional Neural NetworkImageOrdinary Differential Equation
🎯 What it does: A watermark proof method based on mollifier smoothing is proposed, which provides a dimension-independent security radius against lₚ norm watermark removal attacks in high-dimensional parameter spaces for any 1≤p≤∞.
Dimensionality Reduction for General KDE Mode Finding
Xinyu Luo (Purdue University), Cas Widdershoven (Institute of Software, Chinese Academy of Sciences)
ImageText
🎯 What it does: This paper proposes a general random dimensionality reduction method for efficiently finding the modes of kernel density estimation (KDE) in high-dimensional spaces, applicable to a wide range of relative distance smoothing kernels (such as Gaussian, logistic, S-shaped, generalized Gaussian, etc.).
Dink-Net: Neural Clustering on Large Graphs
Yue Liu (National University of Defense Technology), Stan Z. Li (Westlake University)
Graph Neural NetworkContrastive LearningGraph
🎯 What it does: A scalable deep graph clustering method Dink-Net is proposed, achieving a unified framework for self-supervised node discrimination and neural clustering modules.
Direct Parameterization of Lipschitz-Bounded Deep Networks
Ruigang Wang (University of Sydney), Ian Manchester
OptimizationConvolutional Neural NetworkImage
🎯 What it does: A new direct parameterization method is proposed, allowing deep neural networks to automatically satisfy the ℓ2 Lipschitz upper bound during training, and enabling learning through standard gradient optimization.
Directed Chain Generative Adversarial Networks
Ming Min (University of California), Tomoyuki Ichiba (University of California)
GenerationData SynthesisGenerative Adversarial NetworkMultimodalityTime SeriesFinance RelatedStochastic Differential Equation
🎯 What it does: A generative adversarial network based on directed chain stochastic differential equations (DC-SDE) is proposed—DC-GAN, to synthesize multimodal time series data.
Dirichlet Diffusion Score Model for Biological Sequence Generation
Pavel Avdeyev (University of Texas Southwestern Medical Center), Jian Zhou (University of Texas Southwestern Medical Center)
GenerationData SynthesisTransformerDiffusion modelScore-based ModelSequentialBiomedical DataStochastic Differential Equation
🎯 What it does: A continuous-time fractional model for diffusion on probability simplices is proposed—the Dirichlet Diffusion Fractional Model (DDSM), which is applied to discrete data generation, especially for biological sequences (such as human promoters) and constrained data (Sudoku) generation and solving.
DiscoBAX - Discovery of optimal intervention sets in genomic experiment design
Clare Lyle (University of Oxford), Patrick Schwab (GlaxoSmithKline)
OptimizationDrug DiscoveryBiomedical Data
🎯 What it does: A method called DiscoBAX based on Bayesian Algorithm Execution (BAX) is proposed for efficiently searching intervention sets in genomic experiments that can significantly alter targeted phenotypes and cover various biological mechanisms.
Discover and Cure: Concept-aware Mitigation of Spurious Correlation
Shirley Wu (Stanford University), James Zou (Stanford University)
ClassificationExplainability and InterpretabilityContrastive LearningImage
🎯 What it does: Proposes the DISC method, which uses interpretable concepts to identify and eliminate spurious correlations in image classification.
Discover-Then-Rank Unlabeled Support Vectors in the Dual Space for Multi-Class Active Learning
Dayou Yu (Rochester Institute of Technology), Qi Yu (Rochester Institute of Technology)
ClassificationOptimizationTabular
🎯 What it does: A framework for active learning is proposed, starting from the dual space of sparse kernel maximum margin predictors, which first discovers and then ranks unlabelled support vectors.
Discovering Object-Centric Generalized Value Functions From Pixels
Somjit Nath (Ecole de technologie superieure), Samira Ebrahimi Kahou (Mila-Quebec AI Institute)
Robotic IntelligenceReinforcement LearningImage
🎯 What it does: This paper proposes an end-to-end OC-GVFs method that automatically discovers object features from pixels using Slot Attention, and then maps these features to cumulants to learn a Generalized Value Function (GVF), thereby providing a rich and interpretable representation for control policies.
Discrete Continuous Optimization Framework for Simultaneous Clustering and Training in Mixture Models
Parth Vipul Sangani (Indian Institute of Technology Bombay), Abir De
OptimizationImage
🎯 What it does: This paper proposes the PRESTO framework, which jointly optimizes model parameters and data partitioning to learn mixed models.
Discrete Key-Value Bottleneck
Frederik Träuble (MPI for Intelligent Systems), Bernhard Schölkopf (MPI for Intelligent Systems)
ClassificationRepresentation LearningConvolutional Neural NetworkTransformerContrastive LearningImage
🎯 What it does: This paper proposes a Discrete Key-Value Bottleneck architecture that establishes a separate representation layer between the frozen keys and the updatable values in the pre-trained encoder and downstream tasks, thereby enabling local context updates in non-static training flows (such as class-incremental learning) and significantly alleviating catastrophic forgetting.
Disentangled Generative Models for Robust Prediction of System Dynamics
Stathi Fotiadis (Imperial College London), Anil Anthony Bharath (Imperial College London)
GenerationData SynthesisRecurrent Neural NetworkAuto EncoderImageTime SeriesOrdinary Differential Equation
🎯 What it does: Constructed a decoupled variational autoencoder under the supervision of dynamical system parameters, achieving long-term and robust predictions of system states.
Disentangled Multi-Fidelity Deep Bayesian Active Learning
Dongxia Wu (University of California San Diego), Rose Yu (University of California San Diego)
TabularPhysics Related
🎯 What it does: A multi-fidelity deep Bayesian active learning framework D-MFDAL is proposed to learn high-fidelity simulation models from multi-fidelity data.
Disentangled Multiplex Graph Representation Learning
Yujie Mo (University of Electronic Science and Technology of China), Xiaofeng Zhu (University of Electronic Science and Technology of China)
Representation LearningGraph Neural NetworkContrastive LearningGraph
🎯 What it does: Proposes an unsupervised multi-graph representation learning framework that explicitly separates public information from private information and enhances the complementarity of private information through contrastive learning.
Dissecting the Effects of SGD Noise in Distinct Regimes of Deep Learning
Antonio Sclocchi (Ecole Polytechnique Federale de Lausanne), Matthieu Wyart (Ecole Polytechnique Federale de Lausanne)
OptimizationConvolutional Neural NetworkImage
🎯 What it does: This paper studies the impact of noise in stochastic gradient descent (SGD) on the generalization performance of deep networks under different training regimes (determined by the initialization scale α) and different training set sizes P. It systematically constructs a (α,T) phase diagram and reveals whether noise is beneficial or detrimental under different regimes; it also proves through analytical expressions and the perceptron model that noise delays training convergence, leads to weight increase, and introduces interpretable scalability laws.
Distance Weighted Supervised Learning for Offline Interaction Data
Joey Hejna (Stanford University), Dorsa Sadigh (Stanford University)
Robotic IntelligenceReinforcement LearningImageTabular
🎯 What it does: This paper proposes an offline goal-oriented learning method called Distance Weighted Supervised Learning (DWSL), which utilizes supervised learning to estimate the distance distribution between states and constructs an optimal policy based on this.
Distilling Internet-Scale Vision-Language Models into Embodied Agents
Theodore Sumers (Princeton University), Ishita Dasgupta (DeepMind)
Knowledge DistillationRobotic IntelligenceReinforcement Learning from Human FeedbackTransformerReinforcement LearningPrompt EngineeringVision Language ModelMultimodality
🎯 What it does: Using a pre-trained vision-language model to generate natural language descriptions of trajectories, which are then used as re-labeling targets for Hindsight Experience Replay (HER) to train embodied agents to achieve the mapping from language to actions.
Distortion and Uncertainty Aware Loss for Panoramic Depth Completion
Zhiqiang Yan (Nanjing University of Science and Technology), Jian Yang (Nanjing University of Science and Technology)
RestorationDepth EstimationImage
🎯 What it does: This paper proposes a loss function for panoramic depth completion—DUL (Distortion and Uncertainty aware Loss), which enhances depth prediction accuracy by combining distortion awareness and uncertainty estimation.
Distributed Contextual Linear Bandits with Minimax Optimal Communication Cost
Sanae Amani (University of California), Lin Yang
Reinforcement Learning
🎯 What it does: This study investigates the distributed contextual linear bandit problem, proposes an information-theoretic lower bound, and designs the DisBE-LUCB and DecBE-LUCB algorithms based on batch elimination, achieving near-optimal scheduling and communication costs.
Distributed Linear Bandits under Communication Constraints
Sudeep Salgia (Cornell University), Qing Zhao (Cornell University)
Tabular
🎯 What it does: A distributed linear bandit algorithm called Progressive Learning and Sharing (PLS) is proposed, which achieves collaborative learning and information sharing under a limited capacity channel, ensuring that the overall cumulative loss approximates the optimal order of centralized learning.
Distribution Free Domain Generalization
Peifeng Tong (Peking University), Song Xi Chen (Peking University)
ClassificationDomain AdaptationImage
🎯 What it does: This paper proposes a distribution-free domain generalization (DFDG) framework that utilizes standardized MMD statistics and covariance filtering to construct a unified measure of inter-domain and inter-class differences, thereby extracting domain-invariant features under the premise of no target domain labels, ultimately achieving accurate predictions for the target domain.
Distribution Free Prediction Sets for Node Classification
Jase Clarkson (University of Oxford)
ClassificationGraph Neural NetworkGraph
🎯 What it does: This paper proposes a neighborhood-weighted synthetic prediction set method (NAPS) for node classification tasks in graph neural networks, addressing the issue of insufficient coverage caused by the non-exchangeability of traditional synthetic predictions in graph structures.
Distribution-dependent McDiarmid-type Inequalities for Functions of Unbounded Interaction
Shaojie Li (Renmin University of China), Yong Liu (Renmin University of China)
🎯 What it does: A distribution-dependent McDiarmid-type discrete inequality is proposed for functions with unbounded interactions among independent variables, and its specific forms are provided under sub-Gaussian, sub-exponential, and heavy-tailed distributions, followed by applications to sample means, U statistics, and V statistics.
Distributional Offline Policy Evaluation with Predictive Error Guarantees
Runzhe Wu (Cornell University), Wen Sun (Cornell University)
Reinforcement LearningDiffusion modelTabular
🎯 What it does: This paper proposes a new distributed offline policy evaluation method—Fitted Likelihood Estimation (FLE), which transforms distributed evaluation into a series of maximum likelihood estimation (MLE) problems and uses a trainable generative model to approximate the return distribution.
Diverse and Faithful Knowledge-Grounded Dialogue Generation via Sequential Posterior Inference
Yan Xu (Hong Kong University of Science and Technology), Ying Nian Wu (UCLA)
GenerationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: A knowledge-driven dialogue generation framework SPI based on a dual-layer latent variable is proposed, which completes the joint optimization of knowledge selection and response generation using posterior inference instead of variational inference.
Diversity-enhancing Generative Network for Few-shot Hypothesis Adaptation
Ruijiang Dong (Hong Kong Baptist University), Bo Han (Hong Kong Baptist University)
GenerationDomain AdaptationGenerative Adversarial NetworkImage
🎯 What it does: A Diversity Enhanced Generative Network (DEG-Net) is proposed for the Few-shot Hypothesis Adaptation (FHA) problem, generating diverse unlabeled data to improve the performance of target domain classifiers.
Divide and Conquer Dynamic Programming: An Almost Linear Time Change Point Detection Methodology in High Dimensions
Wanshan Li (Carnegie Mellon University), Alessandro Rinaldo (Carnegie Mellon University)
Anomaly DetectionOptimizationTime SeriesBiomedical DataFinance Related
🎯 What it does: An algorithm named Divide and Conquer Dynamic Programming (DCDP) is proposed for efficiently detecting and locating multiple change points in high-dimensional time series, covering various scenarios such as mean, linear regression, and Gaussian graphical models.
Dividing and Conquering a BlackBox to a Mixture of Interpretable Models: Route, Interpret, Repeat
Shantanu Ghosh (Boston University), kayhan Batmanghelich
Explainability and InterpretabilityKnowledge DistillationMixture of ExpertsImageBiomedical DataMagnetic Resonance Imaging
🎯 What it does: This paper proposes an iterative knowledge distillation framework that gradually decomposes a powerful black-box model into a set of interpretable expert models (MoIE) and a residual network, using routers to select samples for the corresponding expert or residual;
DIVISION: Memory Efficient Training via Dual Activation Precision
Guanchu Wang (Rice University), Xia Hu (Rice University)
CompressionComputational EfficiencyConvolutional Neural NetworkTransformerImage
🎯 What it does: A training framework called DIVISION is proposed, which is based on double-precision activation compression. It maintains high precision for low-frequency components while applying low-precision quantization to high-frequency components, significantly reducing activation storage memory during training.
Do Embodied Agents Dream of Pixelated Sheep: Embodied Decision Making using Language Guided World Modelling
Kolby Nottingham (University of California Irvine), Roy Fox (University of California Irvine)
Robotic IntelligenceTransformerLarge Language ModelReinforcement LearningPrompt EngineeringWorld ModelVideo
🎯 What it does: By using a few prompts, the language model generates an Abstract World Model (AWM) to guide embedded reinforcement learning agents in Minecraft for task decomposition and exploration, thereby achieving modular sub-goal strategy learning for complex synthesis tasks.
Do Machine Learning Models Learn Statistical Rules Inferred from Data?
Aaditya Naik (University of Pennsylvania), Eric Wong (University of Pennsylvania)
Domain AdaptationAnomaly DetectionAutonomous DrivingOptimizationBiomedical DataFinance Related
🎯 What it does: This paper proposes a method for automatically generating statistical quantile rules and achieving unsupervised model adaptation through violation penalties during testing.
Do Not Train It: A Linear Neural Architecture Search of Graph Neural Networks
Peng XU, Bei Yu (Chinese University of Hong Kong)
OptimizationComputational EfficiencyNeural Architecture SearchGraph Neural NetworkGraphBenchmark
🎯 What it does: A method for neural architecture search (NAC) for graph neural networks that does not require training weights is proposed. By transforming the NAS problem into sparse coding, an approximately optimal result can be obtained using a randomly initialized linear GNN output layer.
Do Perceptually Aligned Gradients Imply Robustness?
Roy Ganz (Technion), Michael Elad (Technion)
OptimizationAdversarial AttackConvolutional Neural NetworkTransformerDiffusion modelImage
🎯 What it does: A new training objective is designed to directly encourage the model to obtain Perceptual Aligned Gradients (PAG) during non-adversarial training, and it is verified that PAG can improve the model's adversarial robustness.
Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the Machiavelli Benchmark
Alexander Pan (University of California), Dan Hendrycks (Center For AI Safety)
TransformerLarge Language ModelReinforcement LearningPrompt EngineeringTextBenchmarkChain-of-Thought
🎯 What it does: A large text adventure game benchmark, MACHIAVELLI, has been constructed to evaluate the moral behavior and utilitarian tendencies of artificial intelligence in pursuit of rewards.
Do You Remember? Overcoming Catastrophic Forgetting for Fake Audio Detection
XiaoHui Zhang, Chu Yuan Zhang
ClassificationDomain AdaptationConvolutional Neural NetworkSupervised Fine-TuningAudio
🎯 What it does: This paper proposes a continuous learning algorithm called Regularized Adaptive Weight Modification (RAWM) to avoid catastrophic forgetting when transferring an audio synthesis detection model to a new dataset.
DoCoFL: Downlink Compression for Cross-Device Federated Learning
Ron Dorfman (Technion), Kfir Yehuda Levy
CompressionFederated LearningConvolutional Neural NetworkRecurrent Neural NetworkImageText
🎯 What it does: Proposes the DoCoFL framework, which achieves downlink compression in cross-device federated learning by using anchor points and correction decomposition to shorten the download window and significantly reduce bandwidth usage.
Does a Neural Network Really Encode Symbolic Concepts?
Mingjie Li (Shanghai Jiao Tong University), Quanshi Zhang (Shanghai Jiao Tong University)
ClassificationImagePoint CloudTabular
🎯 What it does: Investigate whether neural networks truly encode symbolic concepts, explaining network reasoning based on the decomposition of interactive concepts.
Does Continual Learning Equally Forget All Parameters?
Haiyan Zhao (University of Technology Sydney), Chengqi Zhang (University of Technology Sydney)
OptimizationKnowledge DistillationSequential
🎯 What it does: The study investigates which parameters are more prone to forgetting in continual learning and proposes a method to fine-tune these sensitive parameters only on buffer data, significantly improving the performance of existing continual learning algorithms.
Does Sparsity Help in Learning Misspecified Linear Bandits?
Jialin Dong (University of California), Lin Yang
OptimizationTabular
🎯 What it does: This paper studies the role of sparsity in learning the incorrectly specified linear bandwidth problem and proposes a new algorithm that can achieve O(ε)-optimal actions with fewer queries, breaking the previous lower bound on sample complexity.
DoG is SGD's Best Friend: A Parameter-Free Dynamic Step Size Schedule
Maor Ivgi (Tel Aviv University), Yair Carmon (Tel Aviv University)
OptimizationTransformerSupervised Fine-TuningImageText
🎯 What it does: A dynamic SGD step size formula without learning rate parameters (Distance over Gradients, DOG) is proposed, achieving parameter-independent adaptive updates;
Domain Adaptation for Time Series Under Feature and Label Shifts
Huan He (Harvard University), Marinka Zitnik (Harvard University)
Domain AdaptationConvolutional Neural NetworkMultimodalityTime Series
🎯 What it does: The RAINCOAT method is proposed for unsupervised domain adaptation of time series, addressing the challenges of feature shift and label shift.
DoMo-AC: Doubly Multi-step Off-policy Actor-Critic Algorithm
Yunhao Tang (Google DeepMind), Michal Valko (Google DeepMind)
Reinforcement Learning
🎯 What it does: This paper proposes the dual multi-step offline Actor-Critic algorithm DoMo-AC, which implements a combination of multi-step policy evaluation and improvement based on its theoretical foundation DoMo-VI, ultimately achieving accelerated convergence and more stable policy gradient estimation.
Double-Weighting for Covariate Shift Adaptation
Jose Ignacio Segovia (Basque Center for Applied Mathematics), Anqi Liu (Johns Hopkins University)
ClassificationDomain AdaptationTabular
🎯 What it does: A dual-weighted minimization risk classifier (MRC) framework is proposed for covariance shift adaptation, which simultaneously weights both training and testing samples, addressing the performance degradation issue of traditional single-weighting methods when the supports are non-overlapping or have a large ratio.
Doubly Adversarial Federated Bandits
Jialin Yi (London School of Economics and Political Science), Milan Vojnovic (London School of Economics and Political Science)
Recommendation SystemOptimizationFederated LearningTabularSequential
🎯 What it does: A Doubly Adversarial Federated Bandits framework is proposed, and the FEDEXP3 algorithm is designed to achieve sublinear regret convergence in a distributed manner without exchanging raw information.
Doubly Optimal No-Regret Learning in Monotone Games
Yang Cai (Yale University), Weiqiang Zheng (Yale University)
OptimizationReinforcement LearningTabular
🎯 What it does: Designed and analyzed an accelerated optimistic gradient algorithm (AOG), achieving dual optimal performance for online learning in multi-player monotone games.
DP-Fast MH: Private, Fast, and Accurate Metropolis-Hastings for Large-Scale Bayesian Inference
Wanrong Zhang (Harvard University), Ruqi Zhang
Safty and PrivacyComputational EfficiencyTabular
🎯 What it does: A Metropolis-Hastings algorithm called DP-Fast MH is proposed, which can achieve differential privacy in large-scale Bayesian inference while using only small batches of data in most iterations.
DRCFS: Doubly Robust Causal Feature Selection
Francesco Quinzan (KTH Royal Institute of Technology), Stefan Bauer (Helmholtz Munich)
Tabular
🎯 What it does: A new double robust causal feature selection method (DRCFS) is proposed, aimed at identifying features that are highly correlated with a specific target variable, especially in nonlinear and high-dimensional settings.
DRew: Dynamically Rewired Message Passing with Delay
Benjamin Gutteridge (University of Oxford), Francesco Di Giovanni (University of Cambridge)
Graph Neural NetworkGraph
🎯 What it does: This paper proposes a dynamic reconnection message passing framework DRew (and its delayed version νDRew), which gradually expands the connectivity of the graph at each layer, allowing nodes to interact layer by layer according to distance, thereby alleviating the over-compression problem of traditional MPNNs.
Dropout Reduces Underfitting
Zhuang Liu (Meta AI), Trevor Darrell (University of California Berkeley)
ClassificationObject DetectionSegmentationConvolutional Neural NetworkTransformerImageStochastic Differential Equation
🎯 What it does: The paper studies how using dropout in the early stages of training can alleviate underfitting and proposes two new regularization strategies: early dropout and late dropout.
Drug Discovery under Covariate Shift with Domain-Informed Prior Distributions over Functions
Leo Klarner (University of Oxford), Yee Whye Teh (University of Oxford)
Drug DiscoveryTabular
🎯 What it does: This paper proposes a probabilistic model named Q-SAVI, which constrains neural networks in function space using domain knowledge priors to address the issues of label scarcity and covariate shift in drug discovery.
DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation
Yuhang Lai (University of Hong Kong), Tao Yu (University of Hong Kong)
GenerationAI Code AssistantLarge Language ModelTextBenchmark
🎯 What it does: Constructed DS-1000, a benchmark for data science code generation containing 1000 examples from StackOverflow.
DSGD-CECA: Decentralized SGD with Communication-Optimal Exact Consensus Algorithm
Lisang Ding (University of California), Wotao Yin (Alibaba US)
OptimizationFederated LearningTabular
🎯 What it does: A new decentralized stochastic gradient descent algorithm, DSGD-CECA, is proposed, which allows an arbitrary number of agents to collaborate in training, overcoming the limitations of previous algorithms regarding the number of agents.
Dual Focal Loss for Calibration
Linwei Tao (University of Sydney), Chang Xu (University of Sydney)
ClassificationConvolutional Neural NetworkSupervised Fine-TuningImageText
🎯 What it does: This paper proposes Dual Focal Loss, which simultaneously considers the logit of the true label and the second largest logit during the training of classification models to balance overconfidence and underconfidence, thereby improving the model's probability calibration performance.
Dual Propagation: Accelerating Contrastive Hebbian Learning with Dyadic Neurons
Rasmus Høier (Chalmers University of Technology), Christopher Zach (Chalmers University of Technology)
ClassificationOptimizationComputational EfficiencyConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: A Dual Propagation algorithm for Dyadic Neurons is proposed, utilizing single-stage closed-form layerwise inference to achieve contrastive Hebbian learning.
DualHSIC: HSIC-Bottleneck and Alignment for Continual Learning
Zifeng Wang (Northeastern University), Jennifer Dy (Northeastern University)
ClassificationRepresentation LearningContrastive LearningImage
🎯 What it does: This paper proposes DualHSIC, a general loss that can be seamlessly integrated into existing replay-based continual learning methods to alleviate task interference and enhance task-invariant knowledge sharing.
DUET: 2D Structured and Approximately Equivariant Representations
Xavier Suau (Apple), Luca Zappella (Apple)
ClassificationRepresentation LearningContrastive LearningImage
🎯 What it does: Proposes the DUET method, which utilizes transformation-variant and content-variant information in 2D structured representation learning MSSL.
dugMatting: Decomposed-Uncertainty-Guided Matting
Jiawei Wu (Fujian Agriculture and Forestry University), Joey Tianyi Zhou (Agency for Science Technology and Research)
RestorationSegmentationGenerative Adversarial NetworkImage
🎯 What it does: A framework for image matting based on uncertainty decomposition (dugMatting) is proposed, which reduces model uncertainty and observation noise through interactive suggestions and refinement modules.
Dynamic Constrained Submodular Optimization with Polylogarithmic Update Time
Kiarash Banihashem (University of Maryland), Morteza Monemizadeh (TU Eindhoven)
Optimization
🎯 What it does: A new dynamic algorithm is proposed for the maximization problem of monotone submodular functions under cardinality constraint k in a fully dynamic model, along with theoretical guarantees on its approximation ratio and query complexity.
Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscape
Yan Sun (University of Sydney), Dacheng Tao (University of Sydney)
OptimizationFederated LearningImage
🎯 What it does: The FedSMOO algorithm is proposed, which achieves global consistency and flat minimization in federated learning through dynamic regularization and global Sharpness Aware Minimization.
Dynamical Linear Bandits
Marco Mussi (Politecnico di Milano), Marcello Restelli (Politecnico di Milano)
Recommendation SystemOptimizationTabularTime Series
🎯 What it does: Proposed a Dynamic Linear Bandit (DLB) model and designed the DynLin-UCB algorithm for learning and decision-making.
Dynamics-inspired Neuromorphic Visual Representation Learning
Zhengqi Pei (Chinese Academy of Sciences), Shuhui Wang (Chinese Academy of Sciences)
Representation LearningConvolutional Neural NetworkTransformerImage
🎯 What it does: This paper proposes a dynamics-based neural network architecture (DyN) that transforms traditional weight networks into a form represented solely by the dynamic states of neurons and path integrals, and learns through entropy reduction.
E$(n)$ Equivariant Message Passing Simplicial Networks
Floor Eijkelboom (University of Amsterdam), Erik J Bekkers
Representation LearningDrug DiscoveryGraph Neural NetworkPoint CloudGraph
🎯 What it does: This paper proposes a message-passing simplicial network (EMPSNs) that is E(n) equivariant, combining geometric information and topological structure for learning geometric graphs and point clouds.
ED-Batch: Efficient Automatic Batching of Dynamic Neural Networks via Learned Finite State Machines
Siyuan Chen (Peking University), Todd Mowry (Carnegie Mellon University)
Computational EfficiencyReinforcement LearningText
🎯 What it does: The ED-Batch framework is proposed, which uses finite state machines (FSM) and reinforcement learning to automatically generate batch processing strategies for dynamic neural networks, and reduces data movement through PQ-tree memory allocation, significantly improving the execution efficiency of dynamic DNNs.
EF21-P and Friends: Improved Theoretical Communication Complexity for Distributed Optimization with Bidirectional Compression
Kaja Gruntkowska (University of Warwick), Peter Richtárik (King Abdullah University of Science and Technology)
OptimizationTabular
🎯 What it does: A method supporting bidirectional compression, EF21-P + DIANA/EF21-P + DCGD, is proposed in distributed optimization to reduce the communication volume between servers and worker nodes.
Effective and Efficient Structural Inference with Reservoir Computing
Aoran Wang (University of Luxembourg), Jun Pang (University of Luxembourg)
Auto EncoderGraphTime SeriesSequentialBiomedical DataMagnetic Resonance ImagingPhysics Related
🎯 What it does: This paper proposes a structural inference method that significantly reduces both the amount of data and the time steps by embedding Reservoir Computing (RC) networks into the variational autoencoder (VAE) inference framework and utilizing bi-level optimization (BOME) for joint training of the two branches.
Effective Minkowski Dimension of Deep Nonparametric Regression: Function Approximation and Statistical Theories
Zixuan Zhang (Georgia Institute of Technology), Tuo Zhao (Georgia Institute of Technology)
Tabular
🎯 What it does: This paper proposes a new concept of effective Minkowski dimension for describing function approximation and statistical theory in deep nonparametric regression, relaxing the assumption that input data must strictly lie on a low-dimensional manifold, allowing data to cluster around a certain low-dimensional subset.
Effective Neural Topic Modeling with Embedding Clustering Regularization
Xiaobao Wu (Nanyang Technological University), Anh Tuan Luu (Nanyang Technological University)
Auto EncoderText
🎯 What it does: This paper proposes the Embedding Clustering Regularization Topic Model (ECRTM), which addresses the topic collapse issue by incorporating embedding clustering regularization into neural topic models.
Effective Structured Prompting by Meta-Learning and Representative Verbalizer
Weisen Jiang (Southern University of Science and Technology), James Kwok
Meta LearningPrompt EngineeringText
🎯 What it does: A meta-learning based structured prompt tuning method called MetaPrompter is proposed, along with an unsupervised representative soft verbalizer named RepVerb.
Effectively Using Public Data in Privacy Preserving Machine Learning
Milad Nasr (Google Deepmind), Amir Houmansadr (University of Massachusetts Amherst)
OptimizationSafty and PrivacyDiffusion modelImageText
🎯 What it does: This paper studies how to more effectively utilize public data to enhance model performance in differential privacy machine learning.
Efficient Algorithms for Exact Graph Matching on Correlated Stochastic Block Models with Constant Correlation
Joonhyuk Yang (Korea Advanced Institute of Science and Technology), Hye Won Chung (Korea Advanced Institute of Science and Technology)
Graph Neural NetworkGraph
🎯 What it does: A low-order polynomial time algorithm is proposed to achieve exact graph matching under the constant correlation related stochastic block model (SBM).
Efficient and Degree-Guided Graph Generation via Discrete Diffusion Modeling
Xiaohui Chen (Tufts University), Liping Liu (Tufts University)
GenerationRecurrent Neural NetworkGraph Neural NetworkDiffusion modelGraph
🎯 What it does: A graph generation model called EDGE based on discrete diffusion processes is proposed, which can efficiently generate sparse graphs with thousands of nodes.
Efficient and Equivariant Graph Networks for Predicting Quantum Hamiltonian
Haiyang Yu (Texas A&M University), Shuiwang Ji (Texas A&M University)
Computational EfficiencyDrug DiscoveryGraph Neural NetworkGraphPhysics Related
🎯 What it does: Proposed and implemented the SE(3)-equivariant graph network QHNet for efficient and accurate prediction of molecular Hamiltonian matrices.
Efficient Approximations of Complete Interatomic Potentials for Crystal Property Prediction
Yuchao Lin (Texas A&M University), Shuiwang Ji (Texas A&M University)
Graph Neural NetworkTabularPhysics Related
🎯 What it does: A crystal material property prediction method based on the complete summation of interatomic potential energies is proposed—PotNet.
Efficient Bound of Lipschitz Constant for Convolutional Layers by Gram Iteration
Blaise Delattre (Foxstream), Alexandre Allauzen (ESPCI PSL)
OptimizationComputational EfficiencyConvolutional Neural NetworkImage
🎯 What it does: This paper proposes a new Gram iteration method for quickly and deterministically providing an upper bound on the spectral norm of convolutional layers, thereby controlling the Lipschitz constant of the network.
Efficient displacement convex optimization with particle gradient descent
Hadi Daneshmand (Massachusetts Institute of Technology), Chi Jin (Princeton University)
Optimization
🎯 What it does: This paper studies the convergence properties and complexity of Particle Gradient Descent (PGD) with a finite number of particles on displacement convex functions, providing theoretical upper bounds for achievable approximation errors and optimization errors. It also validates experimental results on instances such as energy distance and uniformly distributed neural networks on a two-dimensional circle.
Efficient Exploration via Epistemic-Risk-Seeking Policy Optimization
Brendan O'Donoghue (Google DeepMind)
OptimizationReinforcement LearningTabular
🎯 What it does: A risk-sensitive policy gradient algorithm based on knowledge uncertainty, called ERSAC, is proposed to achieve efficient exploration by optimizing a zero-sum two-player game.
Efficient Graph Field Integrators Meet Point Clouds
Krzysztof Marcin Choromanski (Google Research), Adrian Weller (University of Cambridge)
Computational EfficiencyGraph Neural NetworkPoint CloudMesh
🎯 What it does: Two efficient graph field integral algorithms (SeparatorFactorization and RFDiffusion) are proposed to accelerate field integration on point clouds (meshes).
Efficient Latency-Aware CNN Depth Compression via Two-Stage Dynamic Programming
Jinuk Kim (Seoul National University), Hyun Oh Song (Seoul National University)
CompressionOptimizationComputational EfficiencyKnowledge DistillationConvolutional Neural NetworkImage
🎯 What it does: By optimizing the activation layer replacement with an identity function and merging consecutive convolutional layers through dynamic programming, the depth of CNNs is compressed;
Efficient Learning of Mesh-Based Physical Simulation with Bi-Stride Multi-Scale Graph Neural Network
Yadi Cao (University of California Los Angeles), Chenfanfu Jiang (University of California Los Angeles)
Graph Neural NetworkMeshGraphPhysics Related
🎯 What it does: For large-scale grid physical simulations, a trainable and efficient multi-scale graph neural network framework has been constructed, allowing the model to maintain low latency during inference and enabling rapid updates at different time steps.
Efficient List-Decodable Regression using Batches
Abhimanyu Das (Google Research), Rajat Sen (Google Research)
🎯 What it does: This paper proposes a polynomial-time algorithm for list-decodable linear regression in batches of data, capable of outputting a small set of regression parameters in scenarios where only α ∈ (0,1] batches are real data, while the remaining batches may be attacked or corrupted, with at least one parameter close to the true weights.
Efficient Online Reinforcement Learning with Offline Data
Philip J. Ball (University of Oxford), Sergey Levine (University of California Berkeley)
Reinforcement LearningTabular
🎯 What it does: A simple framework RLPD is proposed for directly utilizing offline data in online reinforcement learning, balancing sampling, value estimation, and sample efficiency.
Efficient Parametric Approximations of Neural Network Function Space Distance
Nikita Dhawan (University of Toronto), Roger Baker Grosse
Computational EfficiencyConvolutional Neural NetworkImageTabular
🎯 What it does: This paper proposes a network function space distance (Function Space Distance, FSD) approximation method based on activation function linearization called LAFTR, and implements it as the BGLN (Bernoulli Gated Linear Network) model to efficiently estimate the output difference between two networks on the training distribution without storing the complete training set.
Efficient Personalized Federated Learning via Sparse Model-Adaptation
Daoyuan Chen (Alibaba Group), Yaliang Li (Alibaba Group)
Federated LearningExplainability and InterpretabilityComputational EfficiencyImage
🎯 What it does: This paper proposes a framework for adaptively generating sparse personalized models in federated learning, called pFedGate, which utilizes a lightweight trainable gating layer to achieve model sparsification at the client level;
Efficient preconditioned stochastic gradient descent for estimation in latent variable models
Charlotte Baey (University of Lille), Sarah Lemler (University of Paris-Saclay)
OptimizationTabular
🎯 What it does: A preconditioned stochastic gradient descent algorithm is proposed for maximum likelihood estimation of latent variable models.
Efficient Quantum Algorithms for Quantum Optimal Control
Xiantao Li (Pennsylvania State University), Chunhao Wang (Pennsylvania State University)
OptimizationPhysics Related
🎯 What it does: An efficient quantum algorithm that combines time-varying Hamiltonian simulation with quantum gradient estimation is proposed for solving quantum optimal control problems, along with a complete error analysis and query complexity.
Efficient Rate Optimal Regret for Adversarial Contextual MDPs Using Online Function Approximation
Orin Levy (Tel Aviv University), Yishay Mansour (Google Research)
Reinforcement Learning
🎯 What it does: The OMG-CMDP! algorithm is proposed for regret minimization in adversarial contextual Markov decision processes (CMDP).
Efficient RL via Disentangled Environment and Agent Representations
Kevin Gmelin (Carnegie Mellon University), Deepak Pathak (Carnegie Mellon University)
Robotic IntelligenceReinforcement LearningAuto EncoderImage
🎯 What it does: The SEAR method is proposed, which uses robot masks to learn structured representations of the environment and robot disentanglement in visual reinforcement learning, incorporating it as an auxiliary loss into the RL objective.
Efficient Self-supervised Learning with Contextualized Target Representations for Vision, Speech and Language
Alexei Baevski (Meta), Michael Auli (Meta)
Computational EfficiencyRepresentation LearningTransformerImageTextMultimodalityAudio
🎯 What it does: This paper presents data2vec 2.0, a self-supervised pre-training framework that is uniformly used across the three modalities of vision, speech, and language, significantly improving training efficiency.
Efficient Sequence Transduction by Jointly Predicting Tokens and Durations
Hainan Xu (NVIDIA), Boris Ginsburg (NVIDIA)
RecognitionComputational EfficiencyTransformerSequentialAudio
🎯 What it does: The Token-and-Duration Transducer (TDT) model is proposed, which jointly predicts tokens and their durations to achieve faster and more accurate sequence transcription.
Efficient Training of Language Models using Few-Shot Learning
Sashank J. Reddi (Google Research), Sanjiv Kumar (Google Research)
OptimizationComputational EfficiencyMeta LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: A phased training framework based on few-shot learning is proposed, which quickly trains language models with a small number of samples and stacks to expand the network size at each stage.
Efficient Transformed Gaussian Processes for Non-Stationary Dependent Multi-class Classification
Juan Maroñas (Universidad Autonoma de Madrid), Daniel Hernández-Lobato (Universidad Autonoma de Madrid)
ClassificationComputational EfficiencyFlow-based ModelTabular
🎯 What it does: An efficient transformation Gaussian process (ETGP) is proposed for multi-class classification, constructed by first sampling a single GP and then obtaining C non-stationary and correlated processes through C reversible transformations.