ICML 2024 Papers — Page 16
International Conference on Machine Learning · 2610 papers
Multiply Robust Estimation for Local Distribution Shifts with Multiple Domains
Steven Wilkins-Reeves (University of Washington), Aude Hofleitner (Meta)
Domain AdaptationTabular
🎯 What it does: A two-stage 'multiplicative robust' estimation method is proposed to enhance the model's generalization performance under each segment, addressing multi-domain local distribution shifts.
Multiply-Robust Causal Change Attribution
Victor Quintas-Martinez (Massachusetts Institute of Technology), David Heckerman (Amazon)
Tabular
🎯 What it does: This paper proposes a multiple robust causal change attribution estimation method that quantifies the contributions of different causal mechanisms to changes in outcome distribution, given a causal graph.
MusicFlow: Cascaded Flow Matching for Text Guided Music Generation
K R Prajwal (VGG University of Oxford), Wei-Ning Hsu (Meta)
GenerationData SynthesisTransformerFlow-based ModelTextOrdinary Differential EquationAudio
🎯 What it does: This paper presents MusicFlow, a cascaded text-to-music generation model based on flow matching, capable of generating music from natural language descriptions and supporting tasks such as continuation and filling in the blanks.
MusicRL: Aligning Music Generation to Human Preferences
Geoffrey Cideron (Google DeepMind), Andrea Agostinelli
GenerationReinforcement Learning from Human FeedbackTransformerReinforcement LearningTextAudio
🎯 What it does: Fine-tuning the pre-trained MusicLM model using reinforcement learning to make the generated music more aligned with text descriptions, audio quality, and user preferences.
MuxServe: Flexible Spatial-Temporal Multiplexing for Multiple LLM Serving
Jiangfei Duan (Chinese University of Hong Kong), Hao Zhang (University of California San Diego)
OptimizationComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: The MuxServe system is designed to efficiently host various scales of LLMs using a space-time multiplexing method, dynamically separating the prefill and decoding stages while sharing computational and memory resources.
MVMoE: Multi-Task Vehicle Routing Solver with Mixture-of-Experts
Jianan Zhou (Nanyang Technological University), Xu Chi
OptimizationTransformerReinforcement LearningMixture of Experts
🎯 What it does: A multi-task vehicle routing problem solver MVMoE has been developed, utilizing a mixture of experts (MoE) to achieve unified learning and solving for various VRP variants.
Naive Bayes Classifiers over Missing Data: Decision and Poisoning
Song Bian (University of Wisconsin Madison), Paraschos Koutris (University of Wisconsin Madison)
ClassificationAdversarial AttackTabular
🎯 What it does: This study investigates the verifiable robustness of the Naive Bayes classifier on 'dirty' datasets with missing values, providing algorithms for decision problems and data poisoning attacks.
Nash Incentive-compatible Online Mechanism Learning via Weakly Differentially Private Online Learning
Joon Suk Huh (University of Wisconsin Madison), Kirthevasan Kandasamy (University of Wisconsin Madison)
OptimizationReinforcement Learning
🎯 What it does: This paper studies a multi-round mechanism design problem aimed at designing an incentive-compatible online learning scheme to maximize specific application goals without prior knowledge of the agents' type distribution.
Nash Learning from Human Feedback
Remi Munos (Google DeepMind), Bilal Piot (Google DeepMind)
OptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: This paper proposes a new framework for fine-tuning large language models based on human preferences, called NLHF (Nash Learning from Human Feedback);
NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models
Zeqian Ju (University of Science and Technology of China), sheng zhao
GenerationData SynthesisDiffusion modelAudio
🎯 What it does: A zero-shot text-to-speech (TTS) system called NaturalSpeech 3 has been developed, utilizing a Factorized Neural Audio Codec (FACodec) and a factorized diffusion model to generate the duration, content, prosody, quality details, and timbre of speech, achieving zero-shot synthesis while maintaining speech quality, similarity, naturalness of prosody, and intelligibility.
Navigating Complexity: Toward Lossless Graph Condensation via Expanding Window Matching
Yuchen Zhang (National University of Singapore), Yang You (National University of Singapore)
CompressionKnowledge DistillationGraph Neural NetworkGraph
🎯 What it does: The GEOM method is proposed to achieve lossless graph condensation, first using curriculum learning to generate diverse supervisory signals, and then effectively transferring this information to small synthetic graphs through extended window matching, supplemented by a knowledge embedding extractor to further enhance performance.
Navigating Scaling Laws: Compute Optimality in Adaptive Model Training
Sotiris Anagnostidis (ETH Zurich), Thomas Hofmann (ETH Zurich)
OptimizationComputational EfficiencyTransformerLarge Language ModelImageText
🎯 What it does: This paper proposes dynamically adjusting model shapes (such as the patch size of ViT, context length of LLM, model width, batch size, etc.) to traverse different scaling laws during training, significantly reducing the required computational load while maintaining the same performance.
NDOT: Neuronal Dynamics-based Online Training for Spiking Neural Networks
Haiyan Jiang (Mohamed bin Zayed University of Artificial Intelligence), Bin Gu (Mohamed bin Zayed University of Artificial Intelligence)
ClassificationOptimizationSpiking Neural NetworkImage
🎯 What it does: This paper proposes an online training method based on neuron dynamics called NDOT, which enables real-time forward gradient updates for SNNs without unfolding the time dimension.
Near-Linear Time Approximation Algorithms for k-means with Outliers
Junyu Huang (Central South University), Jianxin Wang (Central South University)
OptimizationTabular
🎯 What it does: A near-linear time sampling algorithm, Fast-Sampling and Center-Reduction, is proposed to solve the k-means with outliers problem.
Near-Optimal Regret in Linear MDPs with Aggregate Bandit Feedback
Asaf Cassel (Tel Aviv University), Dmitry Sotnikov (Amazon Science)
OptimizationReinforcement Learning
🎯 What it does: In the absence of stepwise reward feedback, the paper studies the integrated reward feedback (ABF) problem in online linear Markov decision processes (MDP) and proposes two near-optimal algorithms.
Near-Optimal Reinforcement Learning with Self-Play under Adaptivity Constraints
Dan Qiao (University of California), Yu-Xiang Wang (University of California)
OptimizationReinforcement Learning
🎯 What it does: A self-play, low-adaptability (batch) algorithm is proposed for two-player zero-sum Markov games with high deployment costs and minimal update frequency.
Nearest Neighbour Score Estimators for Diffusion Generative Models
Matthew Niedoba (University of British Columbia), Frank Wood (University of British Columbia)
GenerationData SynthesisComputational EfficiencyDiffusion modelScore-based ModelImageOrdinary Differential Equation
🎯 What it does: A score estimator based on nearest neighbors is proposed, which uses multiple training samples for self-normalized importance sampling to reduce the variance of score estimation and decrease bias.
Neighboring Perturbations of Knowledge Editing on Large Language Models
Jun-Yu Ma (University of Science and Technology of China), Jia-Chen Gu (University of California)
TransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: This paper studies the perturbation of neighboring knowledge when adding new knowledge to large language models and proposes an additive metric for measuring such perturbations, along with the evaluation benchmark PEAK, and a pluggable editing framework APP, which aims to reduce the forgetting of existing answer lists and the introduction of noise while adding knowledge.
Nesting Particle Filters for Experimental Design in Dynamical Systems
Sahel Iqbal (Aalto University), Hany Abdulsamad (Aalto University)
OptimizationReinforcement LearningTime Series
🎯 What it does: A nested particle filtering method (Inside‑Out SMC 2) is proposed for Bayesian experimental design with non-exchangeable data, and gradient optimization is performed within the particle MCMC framework.
Network Tight Community Detection
Jiayi Deng (Renmin University of China), Huimin Cheng (Boston University)
Graph Neural NetworkGraph
🎯 What it does: A Tight Community Detection (TCD) method is proposed to isolate discrete points and identify tight communities in networks.
Networked Inequality: Preferential Attachment Bias in Graph Neural Network Link Prediction
Arjun Subramonian (University of California), Yizhou Sun (University of California)
Graph Neural NetworkGraph
🎯 What it does: This study investigates the 'rich get richer' bias of GCN in link prediction and proposes a group fairness metric and a regularization method during training to alleviate this bias.
Neural Collapse for Cross-entropy Class-Imbalanced Learning with Unconstrained ReLU Features Model
Hien Dang (University of Texas at Austin), Nhat Ho (National University of Singapore)
ClassificationOptimizationConvolutional Neural NetworkImage
🎯 What it does: This study investigates the geometric structure of the last layer features and classifiers of neural networks under cross-entropy loss in the context of class-imbalanced data, and provides a closed-form representation of the global optimal solution for the UFM+ model.
Neural Collapse in Multi-label Learning with Pick-all-label Loss
Pengyu Li (University of Michigan), Qing Qu (University of Michigan)
ClassificationOptimizationImage
🎯 What it does: This paper studies the features and classifier structure at the termination stage of deep networks in multi-label classification tasks, proposes and proves the Multi-label Neural Collapse (M-lab NC) theory, and verifies its universality in practical models;
Neural Collapse meets Differential Privacy: Curious behaviors of NoisyGD with Near-Perfect Representation Learning
Chendi Wang (University of Pennsylvania), Yu-Xiang Wang (University of California)
OptimizationSafty and PrivacyRepresentation LearningTransformerSupervised Fine-TuningImage
🎯 What it does: This paper studies how to achieve differential privacy (DP) fine-tuning using Noisy Gradient Descent (NoisyGD/NoisySGD) when the features in the last layer of the neural network are close to Neural Collapse (NC). It analyzes the relationship between misclassification error and dimensionality through theoretical and experimental methods, explores the impact of perturbations on robustness, and proposes mitigation methods using PCA and feature normalization.
Neural Diffusion Models
Grigory Bartosh (University of Amsterdam), Christian A. Naesseth (University of Amsterdam)
GenerationData SynthesisDiffusion modelImageStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: A new neural diffusion model (NDMs) is proposed, which can define and learn time-dependent nonlinear data transformations, thereby overcoming the limitations of traditional diffusion models.
Neural Image Compression with Text-guided Encoding for both Pixel-level and Perceptual Fidelity
Hagyeong Lee (POSTECH), Jaeho Lee (POSTECH)
CompressionContrastive LearningImageText
🎯 What it does: A text-guided coding-based image compression method (TACO) is proposed, which can achieve high pixel-level (PSNR) and perceptual-level (LPIPS) quality simultaneously at standard bit rates.
Neural Jump-Diffusion Temporal Point Processes
Shuai Zhang (Academy of Mathematics and Systems Science, Chinese Academy of Sciences), Zhi-Ming Ma (Academy of Mathematics and Systems Science, Chinese Academy of Sciences)
Time SeriesSequentialBiomedical DataStochastic Differential Equation
🎯 What it does: A new framework is proposed that views intensity processes as solutions to Neural Jump Diffusion Stochastic Differential Equations (NJDSDE), addressing the limitation of traditional Temporal Point Processes (TPP) that require a predefined intensity form.
Neural NeRF Compression
Tuan Pham (University of California Irvine), Stephan Mandt (University of California Irvine)
CompressionNeural Radiance FieldMesh
🎯 What it does: Compression of grid-based NeRF (mainly TensoRF-VM) is performed using neural compression techniques to encode feature planes into smaller storage files.
Neural Networks Learn Statistics of Increasing Complexity
Nora Belrose (EleutherAI), Xiaoli Fern (Oregon State University)
ClassificationLarge Language ModelImageText
🎯 What it does: Research and verify the use of low-order statistics (such as mean and covariance) followed by high-order statistics in the training process of neural networks to simplify the distributional bias (DSB).
Neural operators meet conjugate gradients: The FCG-NO method for efficient PDE solving
Alexander Rudikov (Skolkovo Institute of Science and Technology), Ivan Oseledets (Skolkovo Institute of Science and Technology)
OptimizationComputational EfficiencyTabularPhysics Related
🎯 What it does: Using neural operators as nonlinear preconditioners for solving elliptic partial differential equations with Flexible Conjugate Gradient (FCG).
Neural Operators with Localized Integral and Differential Kernels
Miguel Liu-Schiaffini (California Institute of Technology), Anima Anandkumar (California Institute of Technology)
Convolutional Neural NetworkBenchmarkPhysics Related
🎯 What it does: This paper proposes a local operator within the neural operator framework, achieving efficient learning of function space mappings through differential kernels and local integral kernels.
Neural SPH: Improved Neural Modeling of Lagrangian Fluid Dynamics
Artur Toshev, Johannes Brandstetter (Johannes Kepler University)
Graph Neural NetworkGraphPhysics Related
🎯 What it does: This paper proposes the Neural SPH framework, which enhances the physical consistency and long-term stability of the GNN simulator in Lagrangian fluid dynamics by incorporating external force handling and SPH relaxation steps during the training and inference phases.
Neural Tangent Kernels for Axis-Aligned Tree Ensembles
Ryuichi Kanoh (National Institute of Informatics), Mahito Sugiyama (National Institute of Informatics)
ClassificationExplainability and InterpretabilityTabular
🎯 What it does: This paper focuses on axis-aligned soft tree ensemble learning, deriving a closed-form expression using neural tangent kernel (NTK) theory, and analyzing the learning behaviors of two training methods: always maintaining axis alignment (AAA) throughout the training process and maintaining axis alignment only at initialization (AAI).
Neural Tangent Kernels Motivate Cross-Covariance Graphs in Neural Networks
Shervin Khalafi (University of Pennsylvania), Alejandro Ribeiro (University of Pennsylvania)
Graph Neural NetworkTime SeriesBiomedical DataMagnetic Resonance Imaging
🎯 What it does: The study investigates the impact of NTK on GNN and proposes constructing graph structures using input-output cross covariance to enhance learning effectiveness.
Neural-Kernel Conditional Mean Embeddings
Eiki Shimizu (Graduate University of Advanced Studies), Dino Sejdinovic (University of Adelaide)
OptimizationReinforcement LearningTabular
🎯 What it does: A new method that combines neural networks with Conditional Mean Embedding (CME) is proposed for efficient representation of conditional distributions.
NeuralIndicator: Implicit Surface Reconstruction from Neural Indicator Priors
Shi-Sheng Huang (Beijing Normal University), Hua Huang (Beijing Normal University)
RestorationGenerationPoint Cloud
🎯 What it does: This paper proposes an unsupervised neural implicit surface reconstruction method called NeuralIndicator, which can recover high-fidelity and complete 3D surfaces from incomplete or noisy point clouds.
Neuro-Symbolic Temporal Point Processes
Yang Yang (Chinese University of Hong Kong), Shuang Li (Chinese University of Hong Kong)
Time SeriesSequentialBiomedical Data
🎯 What it does: This paper proposes a neural-symbolic temporal point process model (NS-TPP) that learns to predict the intensity function of target events while automatically inducing a set of interpretable temporal logic rules in the process.
Neuro-Visualizer: A Novel Auto-Encoder-Based Loss Landscape Visualization Method With an Application in Knowledge-Guided Machine Learning
Mohannad Elhamod (Virginia Tech), Anuj Karpatne (Virginia Tech)
OptimizationExplainability and InterpretabilityAuto EncoderTabularSequentialPhysics Related
🎯 What it does: A nonlinear loss landscape visualization method based on autoencoders, called Neuro-Visualizer, is proposed, which can learn a two-dimensional low-dimensional manifold of high-dimensional model parameter space and visualize training trajectories.
Neurodegenerative Brain Network Classification via Adaptive Diffusion with Temporal Regularization
Hyuna Cho (Pohang University of Science and Technology), Won Hwa Kim (Pohang University of Science and Technology)
ClassificationGraph Neural NetworkDiffusion modelGraphBiomedical DataComputed TomographyAlzheimer's Disease
🎯 What it does: Graph classification of neurodegenerative brain networks, considering the temporal evolution of disease progression.
Neuroexplicit Diffusion Models for Inpainting of Optical Flow Fields
Tom Fischer (Saarland University), Eddy Ilg (Saarland University)
RestorationExplainability and InterpretabilityComputational EfficiencyDiffusion modelOptical FlowImageVideo
🎯 What it does: A hybrid model (neuroexplicit) is designed for the interpolation of optical flow fields under sparse masks. By combining explicit PDE diffusion with neural network predicted diffusion tensors, high-quality and interpretable interpolation results are achieved.
New Bounds on the Cohesion of Complete-link and Other Linkage Methods for Agglomerative Clustering
Sanjoy Dasgupta (University of California), Eduardo Sany Laber (Pontifical Catholic University of Rio de Janeiro)
Optimization
🎯 What it does: A new theoretical bound is proposed, proving that complete-linkage has better approximation performance on the maximum diameter and average diameter in clustering, and this analysis is extended to other hierarchical clustering methods such as average-linkage and minimax.
New Sample Complexity Bounds for Sample Average Approximation in Heavy-Tailed Stochastic Programming
Hongcheng Liu (University of Florida), Jindong Tong (University of Florida)
Optimization
🎯 What it does: In convex or strongly convex stochastic programming, this paper studies the non-asymptotic sample complexity of sample average approximation (SAA) and its regularized variants under heavy-tailed distributions, providing new upper bounds on sample complexity that no longer depend on the complexity measure of the feasible region.
NeWRF: A Deep Learning Framework for Wireless Radiation Field Reconstruction and Channel Prediction
Haofan Lu (University of California Los Angeles), Omid Abari (University of California Los Angeles)
Neural Radiance FieldPoint Cloud
🎯 What it does: Proposes the NeWRF framework, which learns indoor wireless channel fields using sparse measurement points and accurately predicts channels at unmeasured points.
NExT-Chat: An LMM for Chat, Detection and Segmentation
Ao Zhang (National University of Singapore), Tat-Seng Chua (National University of Singapore)
Object DetectionSegmentationTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageMultimodality
🎯 What it does: A position embedding-based pix2emb method is proposed, and a multimodal large language model NExT-Chat is constructed, capable of simultaneously performing chat, object detection, and segmentation.
NExT-GPT: Any-to-Any Multimodal LLM
Shengqiong Wu (National University of Singapore), Tat-Seng Chua (National University of Singapore)
GenerationData SynthesisTransformerLarge Language ModelDiffusion modelImageVideoTextMultimodalityAudio
🎯 What it does: Developed NExT-GPT, an end-to-end arbitrary input-arbitrary output multimodal large language model that supports perception and generation across four modalities: text, images, videos, and audio.
NExT: Teaching Large Language Models to Reason about Code Execution
Ansong Ni (Yale University), Pengcheng Yin (Google DeepMind)
AI Code AssistantTransformerLarge Language ModelTextChain-of-Thought
🎯 What it does: In the program repair task, the NEXT method is proposed, allowing LLMs to generate natural language reasoning steps to repair code through execution traces.
No Dimensional Sampling Coresets for Classification
Meysam Alishahi (University of Utah), Jeff M. Phillips (University of Leipzig)
ClassificationOptimization
🎯 What it does: A first method for constructing core samples (coresets) for classification problems that does not depend on dimensionality is proposed, which can approximate the loss function of the original data with a small sample under a given error.
No Double Descent in Principal Component Regression: A High-Dimensional Analysis
Daniel Gedon (Uppsala University), Thomas B. Schön (Uppsala University)
TabularAgriculture Related
🎯 What it does: The theoretical analysis of the generalization risk of principal component regression (PCR) in high-dimensional data under the spiked covariance model is conducted, proving that there is no double descent phenomenon.
No Free Prune: Information-Theoretic Barriers to Pruning at Initialization
Tanishq Kumar (Harvard University), Mark Sellke (Harvard University)
Image
🎯 What it does: This paper presents an explanation from the perspective of information theory on why pruning at initialization is difficult, and proves the mutual information of the pruning mask is related to the importance of the data.
No Wrong Turns: The Simple Geometry Of Neural Networks Optimization Paths
Charles Guille-Escuret (Mila), Ioannis Mitliagkas (Mila)
OptimizationConvolutional Neural NetworkTransformerImageText
🎯 What it does: Through empirical research on training paths under various networks, datasets, and optimizers, the geometric quantities of RSI (Restricted Sine Inequality) and EB (Error Bound) during the stochastic gradient descent (SGD) process were measured and analyzed. The ratio γ (the cosine similarity between the gradient and the target direction) was introduced to evaluate the 'error-free turning' characteristics of the optimization path.
No-Regret Reinforcement Learning in Smooth MDPs
Davide Maran (Politecnico di Milano), Marcello Restelli (Politecnico di Milano)
Reinforcement Learning
🎯 What it does: A new framework for obtaining no-regret guarantees in continuous state/action spaces is proposed, defining ν-smooth MDPs and presenting two algorithms: LEGENDRE-ELEANOR and LEGENDRE-LSVI.
Noise-Adaptive Confidence Sets for Linear Bandits and Application to Bayesian Optimization
Kwang-Sung Jun (University of Arizona), Jungtaek Kim (University of Pittsburgh)
OptimizationTabular
🎯 What it does: A new noise-adaptive confidence set is proposed for linear bandit problems and applied to Bayesian optimization.
Noise-Aware Algorithm for Heterogeneous Differentially Private Federated Learning
Saber Malekmohammadi (University of Waterloo), YANG CAO
Federated LearningSafty and PrivacyImage
🎯 What it does: In the context of heterogeneous differential privacy federated learning (DPFL), the Robust-HDP algorithm is proposed, achieving noise-aware aggregation on untrusted servers.
Non-Asymptotic Analysis for Single-Loop (Natural) Actor-Critic with Compatible Function Approximation
Yudan Wang (University at Buffalo), Shaofeng Zou (University at Buffalo)
OptimizationReinforcement Learning
🎯 What it does: This paper provides the tightest non-asymptotic convergence upper bounds for the single-loop Actor-Critic (AC) and Natural Actor-Critic (NAC) algorithms with compatible function approximation. It proves that under a two-time-scale setting with a single trajectory, AC converges to an ε-stationary point with a sample complexity of O(ε⁻²), while NAC converges to the global optimal policy in the ε+√ε_actor neighborhood with a sample complexity of O(ε⁻³).
Non-clairvoyant Scheduling with Partial Predictions
Ziyad Benomar (ENSAE), Vianney Perchet (Criteo AI Lab)
Optimization
🎯 What it does: The study investigates how to utilize a non-clairvoyant algorithm with only partial job size predictions (only B jobs) in single-machine preemptive scheduling, providing theoretical lower and upper bounds.
Non-confusing Generation of Customized Concepts in Diffusion Models
Wang Lin (Zhejiang University), Hanwang Zhang (Nanyang Technological University)
GenerationData SynthesisDiffusion modelContrastive LearningImageText
🎯 What it does: To address the issue of concept confusion in text-guided diffusion models during multi-concept custom generation, a two-stage contrastive fine-tuning method called CLIF is proposed. This method first applies contrastive learning to the concept word embeddings in the text encoder stage, and then jointly fine-tunes LoRA in the image decoder stage to achieve confusion-free synthesis of multiple concepts.
Non-convex Stochastic Composite Optimization with Polyak Momentum
Yuan Gao (CISPA), Sebastian U Stich
OptimizationTabularStochastic Differential Equation
🎯 What it does: This paper studies the convergence of the stochastic proximal gradient method with Polyak momentum in non-convex composite optimization problems, proving that the method can achieve optimal convergence rates for any batch size.
Non-parametric Online Change Point Detection on Riemannian Manifolds
Xiuheng Wang (Universite Cote dAzur), Cédric Richard (Universite Cote dAzur)
Anomaly DetectionTime SeriesAudio
🎯 What it does: A non-parametric online change point detection algorithm based on the generalized Karcher mean on Riemannian manifolds is proposed, utilizing Riemannian stochastic gradient descent for online mean estimation, and theoretical bounds for false positive rate and detection rate are provided.
Non-stationary Online Convex Optimization with Arbitrary Delays
Yuanyu Wan (Zhejiang University), Lijun Zhang (Nanjing University)
OptimizationReinforcement Learning
🎯 What it does: This paper studies non-stationary online convex optimization (OCO) with arbitrary delays, proposing a simple algorithm called DOGD and analyzing its dynamic regret bounds.
Non-Vacuous Generalization Bounds for Large Language Models
Sanae Lotfi (New York University), Andrew Gordon Wilson (New York University)
TransformerLarge Language ModelText
🎯 What it does: This paper presents the first non-vacuous generalization error bound for large-scale language models and proves that it can generalize on out-of-training data.
Nonlinear Filtering with Brenier Optimal Transport Maps
Mohammad Al-Jarrah (University of Washington), Amirhossein Taghvaei (University of Washington)
Anomaly DetectionOptimizationGenerative Adversarial NetworkMultimodalityTime Series
🎯 What it does: A nonlinear filtering method based on Brenier optimal transport (OT) mapping is proposed, along with a theoretical analysis of its consistency and stability. Subsequently, the OT mapping is implemented through neural networks and numerically validated on multi-dimensional and multi-modal problems.
Nonparametric Teaching of Implicit Neural Representations
Chen Zhang (University of Hong Kong), Ngai Wong (University of Hong Kong)
OptimizationComputational EfficiencyRepresentation LearningImageAudio
🎯 What it does: This paper proposes an Implicit Neural Teaching (INT) framework based on non-parametric teaching, which accelerates the training of implicit neural representations (INR) by dynamically selecting samples with the largest residuals.
Nonsmooth Implicit Differentiation: Deterministic and Stochastic Convergence Rates
Riccardo Grazzi (Istituto Italiano di Tecnologia), Saverio Salzo (Università La Sapienza)
OptimizationHyperparameter SearchTabular
🎯 What it does: This paper studies the fixed point differentiation problem for non-differentiable contraction mappings, providing deterministic convergence rates for ITD (Iterative Differentiation) and AID (Approximate Implicit Differentiation), and proposes the first provably convergent stochastic method, NSID; experiments are then conducted to validate these results in bilevel optimization (hyperparameter optimization and data poisoning).
Not all distributional shifts are equal: Fine-grained robust conformal inference
Jiahao Ai (Peking University), Zhimei Ren (University of Pennsylvania)
Domain AdaptationOptimizationTabular
🎯 What it does: This paper proposes a fine-grained robust adaptive prediction interval method (WRCP and D-WRCP) that can provide prediction intervals with coverage close to the preset level in the presence of distribution drift.
Not Just Pretty Pictures: Toward Interventional Data Augmentation Using Text-to-Image Generators
Jianhao Yuan (University of Oxford), Philip Torr (University of Oxford)
Data SynthesisDomain AdaptationPrompt EngineeringDiffusion modelImageText
🎯 What it does: The study uses text-to-image (T2I) generators (such as Stable Diffusion) for intervention data augmentation (IDA) and evaluates their effectiveness in single-domain generalization (SDG) and reducing reliance on spurious features (RRSF) tasks.
Novel Spectral Algorithms for the Partial Credit Model
Duc Nguyen (University of Pennsylvania), Anderson Ye Zhang (University of Pennsylvania)
Recommendation SystemTabularFinance Related
🎯 What it does: A spectral algorithm for Partial Credit Models (PCM) is proposed to estimate project parameters;
O$n$ Learning Deep O($n$)-Equivariant Hyperspheres
Pavlo Melnyk (Linköping University), Cuong Le (Linköping University)
ClassificationRecognitionPoint Cloud
🎯 What it does: A deep learning model that achieves O(n) symmetry in any dimension is proposed—Deep Equivariant Hyperspheres. It constructs spherical decision boundaries through regular n-simplices and realizes equivariant feature extraction for point cloud data.
OAK: Enriching Document Representations using Auxiliary Knowledge for Extreme Classification
Shikhar Mohan (Microsoft), Manik Varma (Microsoft Research)
ClassificationRepresentation LearningContrastive LearningText
🎯 What it does: The OAK framework is proposed to enhance extreme classification performance by embedding auxiliary knowledge into document representations.
Observable Propagation: Uncovering Feature Vectors in Transformers
Jacob Dunefsky (Yale University), Arman Cohan (Yale University)
TransformerLarge Language ModelText
🎯 What it does: A method for discovering and analyzing linear feature vectors from Transformer language models with almost no data is proposed—Observable Propagation (OBPROP).
ODIM: Outlier Detection via Likelihood of Under-Fitted Generative Models
Dongha Kim (Sungshin Women's University), Yongdai Kim (Seoul National University)
Anomaly DetectionAuto EncoderImageTextTabular
🎯 What it does: An unsupervised anomaly detection method ODIM based on low-fit generative model likelihood is proposed.
ODIN: Disentangled Reward Mitigates Hacking in RLHF
Lichang Chen (University of Maryland), Bryan Catanzaro (Nvidia)
Reinforcement Learning from Human FeedbackReinforcement LearningText
🎯 What it does: A reward model separation technique called ODIN is proposed and experimented with to eliminate length reward bias in RLHF, thereby mitigating reward hacking.
Off-policy Evaluation Beyond Overlap: Sharp Partial Identification Under Smoothness
Samir Khan (Stanford University), Johan Ugander (Stanford University)
Tabular
🎯 What it does: Under the condition of no overlap, a precise partial identification framework for offline policy evaluation (OPE) is proposed using the smoothness (Lipschitz) assumption, along with a closed-form linear programming solution method; asymptotically optimal estimators and convergence rate analysis are also provided.
Offline Actor-Critic Reinforcement Learning Scales to Large Models
Jost Tobias Springenberg (Google DeepMind), Martin Riedmiller (Google DeepMind)
Robotic IntelligenceTransformerReinforcement LearningAgentic AIMultimodalitySequential
🎯 What it does: This study investigates how to extend offline actor-critic reinforcement learning to large-scale models, achieving better performance than behavior cloning on 132 multi-task continuous control tasks.
Offline Imitation from Observation via Primal Wasserstein State Occupancy Matching
Kai Yan (University of Illinois Urbana-Champaign), Yu-Xiong Wang (University of Illinois Urbana-Champaign)
OptimizationRobotic IntelligenceReinforcement Learning from Human FeedbackReinforcement LearningContrastive LearningTabular
🎯 What it does: A new offline observation-based imitation learning framework called PW-DICE is proposed, which utilizes the original Wasserstein distance to match the state occupancy distributions of experts and learners, and achieves unconstrained convex optimization through the addition of lazy regularization.
Offline Inverse RL: New Solution Concepts and Provably Efficient Algorithms
Filippo Lazzati (Politecnico di Milano), Alberto Maria Metelli (Politecnico di Milano)
Autonomous DrivingReinforcement LearningSequential
🎯 What it does: A feasible reward set definition for offline inverse reinforcement learning is proposed, along with a corresponding PAC learning framework and two efficient algorithms, IRLO and PIRLO.
Offline Multi-Objective Optimization
Ke Xue (Nanjing University), Chao Qian (Nanjing University)
OptimizationNeural Architecture SearchReinforcement LearningTabularBenchmark
🎯 What it does: The first offline multi-objective optimization (offline MOO) benchmark set is proposed, covering various tasks such as synthetic functions, NAS, reinforcement learning, combinatorial optimization, scientific design, and real engineering, along with corresponding datasets and example implementations.
Offline Training of Language Model Agents with Functions as Learnable Weights
Shaokun Zhang (Pennsylvania State University), Qingyun Wu (Pennsylvania State University)
TransformerLarge Language ModelAgentic AIPrompt EngineeringText
🎯 What it does: A method is proposed to train LLM agents (AgentOptimizer) by learning updatable functions without modifying the LLM weights, achieving improved agent performance.
Offline Transition Modeling via Contrastive Energy Learning
Ruifeng Chen (Nanjing University), Yang Yu (Nanjing University)
Reinforcement LearningContrastive LearningTabular
🎯 What it does: An Energy-based Transition Model (ETM) is proposed for modeling transition dynamics in offline reinforcement learning, achieving accurate predictions of complex and discrete transitions through contrastive learning to train the energy function.
Offline-Boosted Actor-Critic: Adaptively Blending Optimal Historical Behaviors in Deep Off-Policy RL
Yu Luo (Tsinghua University), Xianyuan Zhan (Tsinghua University)
Reinforcement LearningSequential
🎯 What it does: This paper proposes an Offline-Boosted Actor-Critic (OBAC) framework that can parallelly train and utilize offline optimal policies during online reinforcement learning processes to enhance the performance of online policies, employing an adaptive constraint mechanism to dynamically introduce offline policies based on value comparisons.
OLLIE: Imitation Learning from Offline Pretraining to Online Finetuning
Sheng Yue (Tsinghua University), Yaoxue Zhang (Tsinghua University)
Robotic IntelligenceReinforcement Learning from Human FeedbackReinforcement LearningGenerative Adversarial NetworkImage
🎯 What it does: A framework for imitation learning from offline pre-training to online fine-tuning, called OLLIE, is proposed. It learns an approximate expert policy from offline data and synchronously obtains an aligned discriminator, supporting rapid online fine-tuning for high-dimensional tasks.
OMPO: A Unified Framework for RL under Policy and Dynamics Shifts
Yu Luo (Tsinghua University), Xianyuan Zhan (Tsinghua University)
Robotic IntelligenceReinforcement LearningGenerative Adversarial NetworkSequential
🎯 What it does: A unified RL framework called OMPO is proposed, which uses transition occupancy matching to address the distribution shift between policy and dynamics, enhancing the effectiveness of online policy learning.
On a Combinatorial Problem Arising in Machine Teaching
Joakim Sunde (University of Bergen), Jan Arne Telle (Charles University)
Tabular
🎯 What it does: A machine teaching model is studied, where the teacher mapping is constructed based on a size function of concepts and examples, with the main issue being to determine the minimum number of examples required for any concept, known as the teaching dimension.
On a Neural Implementation of Brenier's Polar Factorization
Nina Vesseron (CREST-ENSAE), marco cuturi
OptimizationTabularStochastic Differential Equation
🎯 What it does: This study implements a neural network method for Brenier polar decomposition and learns its inverse mapping for optimizing landscape sampling.
On Computational Limits of Modern Hopfield Models: A Fine-Grained Complexity Analysis
Jerry Yao-Chieh Hu (Northwestern University), Han Liu (Northwestern University)
RetrievalComputational Efficiency
🎯 What it does: Through fine-grained complexity analysis, the computational limits of the modern Hopfield model in memory retrieval are clarified, and a low-rank approximation implementation with nearly linear time is proposed;
On Convergence of Incremental Gradient for Non-convex Smooth Functions
Anastasia Koloskova (École Polytechnique Fédérale de Lausanne), Martin Jaggi (École Polytechnique Fédérale de Lausanne)
OptimizationTabular
🎯 What it does: This paper studies the convergence properties of incremental gradient algorithms on non-convex smooth functions, with a particular focus on the convergence of the stochastic gradient descent (SGD) algorithm for arbitrary data orders.
On dimensionality of feature vectors in MPNNs
César Bravo (Instituto de Ingeniería Matemática y Computacional Universidad Católica de Chile), Cristobal Rojas (Instituto de Ingeniería Matemática y Computacional Universidad Católica de Chile)
Graph Neural NetworkGraph
🎯 What it does: This paper proves that a one-dimensional feature vector using a non-polynomial analytic activation function (such as sigmoid) in MPNN can completely simulate the Weisfeiler–Leman (WL) graph isomorphism test, meaning that WL's recognition capability can be achieved with just a one-dimensional feature;
On Discrete Prompt Optimization for Diffusion Models
Ruochen Wang (University of California, Los Angeles), Boqing Gong (Google Research)
GenerationOptimizationAdversarial AttackLarge Language ModelPrompt EngineeringDiffusion modelImageText
🎯 What it does: A gradient-based discrete prompt optimization framework (DPO-Diff) is proposed, which can automatically generate optimized prompts for text-to-image diffusion models, enhancing image quality and enabling adversarial attacks.
On Gradient-like Explanation under a Black-box Setting: When Black-box Explanations Become as Good as White-box
Yi Cai (Freie Universität Berlin), Gerhard Wunder (Freie Universität Berlin)
Explainability and InterpretabilityImage
🎯 What it does: A black-box gradient estimation explanation method named GEEX is proposed, which approximates gradients using a query interface and generates gradient-based explanations through path integrals.
On Hypothesis Transfer Learning of Functional Linear Models
Haotian Lin (Pennsylvania State University), Matthew Reimherr (Pennsylvania State University)
TabularTime SeriesFinance Related
🎯 What it does: This paper proposes and theoretically analyzes two algorithms, TL-FLR and ATL-FLR, based on Optimal Transfer Learning (OTL) within the frameworks of Functional Linear Regression (FLR) and Functional Generalized Linear Models (FGLM), aimed at enhancing the predictive performance of the target task when information from the source task is limited.
On Interpolating Experts and Multi-Armed Bandits
Houshuang Chen (Shanghai Jiao Tong University), Chihao Zhang (Shanghai Jiao Tong University)
OptimizationReinforcement Learning
🎯 What it does: This paper studies a class of problems that lie between learning expert opinions and multi-armed bandits, defining the m-MAB and m-BAI models, and providing their optimal theoretical limits;
On Least Square Estimation in Softmax Gating Mixture of Experts
Huy Nguyen (University of Texas at Austin), Alessandro Rinaldo (University of Texas at Austin)
Mixture of Experts
🎯 What it does: The theoretical analysis of the least squares estimation (LSE) for softmax gated mixture of experts (MoE) models is conducted, proving that its regression function can achieve a parametric convergence rate, and providing estimation rates for different types of experts (strongly identifiable experts, ridge experts, polynomial experts).
On Mechanistic Knowledge Localization in Text-to-Image Generative Models
Samyadeep Basu (University of Maryland), Soheil Feizi (University of Maryland)
GenerationData SynthesisTransformerDiffusion modelImageText
🎯 What it does: Two methods, LOCOGEN and LOCOEDIT, are proposed for locating and editing the control layers of visual attributes in text-to-image generation models.
On Multi-Armed Bandit with Impatient Arms
Yuming Shao (Tsinghua University), Zhixuan Fang (Tsinghua University)
Recommendation SystemOptimizationReinforcement LearningTabular
🎯 What it does: This paper studies a multi-armed bandit (MAB) setting where an arm exits the game if the algorithm continuously ignores it. This setting is inspired by real-world scenarios such as online advertising and crowdsourcing, where an arm can only yield rewards after being pulled by the algorithm.
On Online Experimentation without Device Identifiers
Shiv Shankar (University of Massachusetts), Madalina Fiterau (University of Massachusetts)
GraphTabular
🎯 What it does: This paper proposes HIFIVE, a variational inference method for estimating the global average treatment effect (GATE) in scenarios where device identification is fragmented (users across multiple devices).
On PI Controllers for Updating Lagrange Multipliers in Constrained Optimization
Motahareh Sohrabi (Mila-Quebec AI Institute and Universite de Montreal), Jose Gallego-Posada (Mila-Quebec AI Institute and Universite de Montreal)
OptimizationTabular
🎯 What it does: A νPI algorithm based on a PI controller is proposed for updating Lagrange multipliers in Lagrangian constraint optimization, along with theoretical analysis and experimental validation.
On Positivity Condition for Causal Inference
Inwoo Hwang (Seoul National University), Sanghack Lee (Seoul National University)
🎯 What it does: The study investigates how to identify causal effects under non-strict positivity conditions, proposing general positivity conditions and designing corresponding algorithms.
On Prompt-Driven Safeguarding for Large Language Models
Chujie Zheng (Tsinghua University), Nanyun Peng (University of California Los Angeles)
OptimizationSafty and PrivacyTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: Study the working mechanism of safety prompts in large language models and propose a representation space-based safety prompt optimization method called DRO.
On Statistical Learning Theory for Distributional Inputs
Christian Fiedler (RWTH Aachen University), Sebastian Trimpe (RWTH Aachen University)
🎯 What it does: This paper derives two new oracle inequalities and generalization bounds based on algorithmic stability within the framework of distributed input kernel-based statistical learning, covering a wide range of loss functions, kernel functions, and Hilbertian embedding methods, particularly suitable for KME and sliced Wasserstein distance.
On Stronger Computational Separations Between Multimodal and Unimodal Machine Learning
Ari Karchmer (Boston University)
Computational EfficiencyMultimodality
🎯 What it does: This paper proposes an average-case computational separation between multimodal and unimodal learning, proving that under the low-noise LPN assumption, there exist multimodal learning tasks that can be solved in polynomial time on typical instances, while the corresponding unimodal tasks are unsolvable in polynomial time; it further demonstrates that any such super-polynomial computational separation can be used to construct cryptographic key agreement protocols, suggesting that strong computational advantages may not be common in real data.
On the Asymptotic Distribution of the Minimum Empirical Risk
Jacob Westerhout (University of Queensland), Hien Duy Nguyen (La Trobe University)
Tabular
🎯 What it does: This paper proposes a √n-level limit distribution theory under the conditions of non-independent, non-Euclidean parameters, and possibly discontinuous loss functions, and provides methods for Bootstrap consistency and model selection.