ICML 2023 Papers — Page 18
International Conference on Machine Learning · 1828 papers
Towards Omni-generalizable Neural Methods for Vehicle Routing Problems
Jianan Zhou (Nanyang Technological University), Jie Zhang (Nanyang Technological University)
OptimizationMeta LearningGraph Neural NetworkReinforcement LearningTabular
🎯 What it does: A comprehensive study on the generalization of neural methods for the vehicle routing problem in full size and distribution.
Towards Practical Preferential Bayesian Optimization with Skew Gaussian Processes
Shion Takeno (Nagoya Institute of Technology), Masayuki Karasuyama (Nagoya Institute of Technology)
OptimizationTabularBenchmark
🎯 What it does: This paper studies the problem of preferential Bayesian optimization, utilizing skew Gaussian processes to model preference relationships. It proposes a more efficient Gibbs sampling method and a low-variance Monte Carlo estimator, and subsequently introduces a preference optimization algorithm based on the 'hallucination believer' strategy, which can quickly and efficiently utilize the skew posterior with low sample complexity.
Towards Quantum Machine Learning for Constrained Combinatorial Optimization: a Quantum QAP Solver
Xinyu Ye (Shanghai Jiao Tong University), Junchi Yan (Shanghai Jiao Tong University)
OptimizationSupervised Fine-TuningGraph
🎯 What it does: A quantum neural network QAP-QNN based on variational quantum circuits is proposed and implemented, using supervised learning methods to solve the quadratic assignment problem (QAP) in constrained combinatorial optimization, and experimental validation is conducted through graph matching and the traveling salesman problem.
Towards Reliable Neural Specifications
Chuqin Geng (McGill University), Xujie Si (University of Toronto)
Convolutional Neural NetworkImage
🎯 What it does: A normalization method based on Neural Activation Patterns (NAP) is proposed, defining the correctness and robustness of neural networks using activation patterns rather than the input data itself;
Towards Robust and Safe Reinforcement Learning with Benign Off-policy Data
Zuxin Liu (Carnegie Mellon University), Ding Zhao (Carnegie Mellon University)
Robotic IntelligenceReinforcement Learning
🎯 What it does: This paper proposes the SAFER algorithm, which trains a safe and robust reinforcement learning policy using non-attack offline data through a two-stage EM approach.
Towards Robust Graph Incremental Learning on Evolving Graphs
Junwei Su (University of Hong Kong), Chuan Wu (University of Hong Kong)
Graph Neural NetworkGraph
🎯 What it does: This study investigates how structural drift under inductive node-wise GIL leads to catastrophic forgetting and proposes a Structural Drift Risk Mitigation (SSRM) method to reduce forgetting.
Towards Stable and Efficient Adversarial Training against $l_1$ Bounded Adversarial Attacks
Yulun Jiang (École Polytechnique Fédérale de Lausanne), Sabine Süsstrunk
OptimizationAdversarial AttackConvolutional Neural NetworkImage
🎯 What it does: An efficient and stable adversarial training method for l1 norm perturbations, Fast-EGl 1, is proposed to address the issue of catastrophic overfitting that traditional methods face under l1 perturbations.
Towards Sustainable Learning: Coresets for Data-efficient Deep Learning
Yu Yang (University of California Los Angeles), Baharan Mirzasoleiman (University of California Los Angeles)
OptimizationData-Centric LearningConvolutional Neural NetworkTransformerImageText
🎯 What it does: The CREST framework is proposed, which improves the sample efficiency and sustainability of deep learning by segmenting non-convex loss into piecewise quadratic approximations and constructing mini-batch coresets in each sub-region, dynamically updated.
Towards Theoretical Understanding of Inverse Reinforcement Learning
Alberto Maria Metelli (Politecnico di Milano), Marcello Restelli (Politecnico di Milano)
Reinforcement Learning
🎯 What it does: This paper studies the inverse reinforcement learning (IRL) problem for finite-horizon tunnels, proposing a PAC framework and Hausdorff cost metric. It conducts a theoretical analysis of the measurability and Lipschitz continuity of the feasible reward set and provides lower and upper bounds on the sample complexity for estimating the feasible reward set.
Towards Trustworthy Explanation: On Causal Rationalization
Wenbo Zhang (University of California Irvine), Hengrui Cai (University of California Irvine)
Explainability and InterpretabilityTextBiomedical DataElectronic Health Records
🎯 What it does: This paper proposes a self-explanatory text model based on causal inference—causal rationalization—to extract both necessary and sufficient credible explanations from text;
Towards Unbiased Training in Federated Open-world Semi-supervised Learning
Jie ZHANG, Wenchao Xu (Hong Kong Polytechnic University)
Federated LearningImage
🎯 What it does: Proposes the FedoSSL framework to address the biased training problem caused by unseen classes in federated semi-supervised learning;
Towards Understanding and Improving GFlowNet Training
Max W Shen, Tommaso Biancalani (Recursion Pharmaceuticals)
OptimizationDrug DiscoveryReinforcement LearningFlow-based ModelGraphBiomedical Data
🎯 What it does: Conducted an in-depth analysis of the training process of Generative Flow Networks (GFlowNet), proposed efficient evaluation methods, and based on this, introduced three techniques to improve training, ultimately achieving significant sample efficiency improvements on multiple biochemical design tasks.
Towards Understanding and Reducing Graph Structural Noise for GNNs
Mingze Dong (Yale University), Yuval Kluger (Yale University)
Graph Neural NetworkGraph
🎯 What it does: A new graph structure noise assessment metric ESNR is proposed, along with a graph rearrangement framework GPS based on self-supervised graph link prediction, aimed at quantifying and reducing the negative impact of graph structure noise on the performance of Graph Neural Networks (GNNs).
Towards Understanding Ensemble Distillation in Federated Learning
Sejun Park (Korea Advanced Institute of Science and Technology), Ganguk Hwang (Korea Advanced Institute of Science and Technology)
Federated LearningKnowledge DistillationTabular
🎯 What it does: A theoretical framework based on kernel ridge regression (KRR) is constructed, analyzing the knowledge distillation-based ensemble distillation method in federated learning, and proposing an iterative ensemble distillation algorithm with de-regularization.
Towards Understanding Generalization of Graph Neural Networks
Huayi Tang (Renmin University of China), Yong Liu (Renmin University of China)
Graph Neural NetworkGraph
🎯 What it does: Under the transductive SGD training, a high-probability theoretical upper bound on the generalization gap of graph neural networks is provided, along with an analysis of architecture-related constants for mainstream GNN models.
Towards Understanding Generalization of Macro-AUC in Multi-label Learning
Guoqiang Wu (Shandong University), Yilong Yin (Shandong University)
ClassificationText
🎯 What it does: This paper conducts a theoretical study on the Macro-AUC metric in multi-label learning, providing generalization error upper bounds for different approximate loss functions and validating its theoretical predictions through experiments.
TR0N: Translator Networks for 0-Shot Plug-and-Play Conditional Generation
Zhaoyan Liu (Layer 6 AI), Gabriel Loaiza-Ganem (Layer 6 AI)
GenerationData SynthesisAuto EncoderGenerative Adversarial NetworkImageText
🎯 What it does: TR0N transforms any pre-trained unconditional generative model (such as GANs, VAEs) into a zero-shot conditional generation framework that can generate samples based on any condition (category, text, image semantics) without the need for additional data or fine-tuning.
Tractable Control for Autoregressive Language Generation
Honghua Zhang (University of California), Guy Van den Broeck (University of California)
GenerationOptimizationKnowledge DistillationRecurrent Neural NetworkLarge Language ModelText
🎯 What it does: The GeLaTo framework is proposed, which combines solvable probabilistic models (such as HMM) with large-scale autoregressive language models to generate text that meets lexical constraints.
Trading-Off Payments and Accuracy in Online Classification with Paid Stochastic Experts
Dirk van der Hoeven (University of Amsterdam), Nicolò Cesa-Bianchi (Università degli Studi di Milano)
ClassificationOptimizationComputational EfficiencyMixture of ExpertsTabular
🎯 What it does: This study investigates the scenario of paid random experts in online classification, where the learner decides the amount to pay each expert in each round and classifies based on the experts' predictions.
Trainability, Expressivity and Interpretability in Gated Neural ODEs
Timothy Doyeon Kim (Princeton University), Kamesh Krishnamurthy (Princeton University)
Explainability and InterpretabilityComputational EfficiencyRecurrent Neural NetworkTime SeriesSequentialOrdinary Differential EquationAudio
🎯 What it does: This paper proposes and studies the Gated Neural Ordinary Differential Equation (gnODE), exploring its trainability, expressiveness, and interpretability, and conducting experimental validation on various synthetic and real datasets.
Training Deep Surrogate Models with Large Scale Online Learning
Lucas Thibaut Meyer (Univ. Grenoble Alpes), Bruno Raffin (Univ. Grenoble Alpes)
Time SeriesPhysics Related
🎯 What it does: An online learning framework has been developed that can generate PDE simulation data in real-time on supercomputers and directly use it to train deep surrogate models, eliminating disk I/O and storage bottlenecks.
Training Normalizing Flows from Dependent Data
Matthias Kirchler (Hasso Plattner Institute for Digital Engineering), Marius Kloft (University of Kaiserslautern-Landau)
Flow-based ModelTabularBiomedical DataAlzheimer's DiseaseFinance Related
🎯 What it does: Proposes a maximum likelihood objective for training normalizing flows in the presence of data dependencies, and provides an efficient mini-batch optimization method;
Training-Free Neural Active Learning with Initialization-Robustness Guarantees
Apivich Hemachandra (National University of Singapore), Bryan Kian Hsiang Low (National University of Singapore)
ClassificationOptimizationData-Centric LearningConvolutional Neural NetworkImageTabular
🎯 What it does: This paper proposes a training-independent neural network active learning method (EV-GP) that selects labeled samples by minimizing the output variance based on NTK, ensuring the model's robustness and generalization performance under random initialization.
Trajectory-Aware Eligibility Traces for Off-Policy Reinforcement Learning
Brett Daley (University of Alberta), Marlos C. Machado (University of Alberta)
Reinforcement LearningTabular
🎯 What it does: A unified multi-step backtracking operator M is proposed, integrating the traditional per-decision eligibility trace method with trajectory-aware methods, providing a proof of convergence for this operator, and validating the new RBIS trajectory-aware algorithm on discrete MDPs.
TRAK: Attributing Model Behavior at Scale
Sung Min Park (Massachusetts Institute of Technology), Aleksander Madry (Massachusetts Institute of Technology)
Explainability and InterpretabilityComputational EfficiencyConvolutional Neural NetworkTransformerImageText
🎯 What it does: A data attribution method named TRAK is proposed, which can efficiently trace the sources of model predictions in large-scale non-convex models.
Transcendental Idealism of Planner: Evaluating Perception from Planning Perspective for Autonomous Driving
Weixin Li, Xiaodong Yang (QCraft)
Autonomous DrivingOptimizationPoint Cloud
🎯 What it does: A perception assessment framework TIP based on a planning perspective is proposed, utilizing the expected utility maximization theory to map perception errors into Hilbert space and quantify their impact on autonomous driving planning decisions.
Transformed Distribution Matching for Missing Value Imputation
He Zhao (CSIRO Data61), Edwin V. Bonilla (CSIRO Data61)
OptimizationData-Centric LearningFlow-based ModelTabular
🎯 What it does: In datasets with missing values, an unsupervised missing value imputation method based on distribution matching, called TDM, is proposed;
Transformer-based Stagewise Decomposition for Large-Scale Multistage Stochastic Optimization
Chanyeong Kim (Korea Advanced Institute of Science and Technology), Woo Chang Kim (Korea Advanced Institute of Science and Technology)
OptimizationTransformerTabularFinance Related
🎯 What it does: A Transformer-based stage decomposition algorithm called TranSDDP is proposed for large-scale multi-stage stochastic programming.
Transformers as Algorithms: Generalization and Stability in In-context Learning
Yingcong Li (University of California Riverside), Samet Oymak (University of Michigan)
TransformerTabularTime Series
🎯 What it does: This paper views the role of Transformer in in-context learning as algorithm learning and provides a theoretical analysis of its generalization error and stability.
Transformers Learn In-Context by Gradient Descent
Johannes von Oswald (ETH Zurich), Max Vladymyrov (Google Research)
OptimizationMeta LearningTransformerTabular
🎯 What it does: This paper constructs linear self-attention weights to demonstrate that the Transformer achieves gradient descent-style contextual learning during autoregressive training, which further evolves into accelerated gradient descent (GD++) in multi-layer Transformers. Additionally, the introduction of MLP allows the Transformer to perform gradient descent on deep representations, thereby solving nonlinear regression problems.
Transformers Meet Directed Graphs
Simon Geisler (Technical University of Munich), Cosmin Paduraru (Google DeepMind)
Graph Neural NetworkTransformerGraph
🎯 What it does: This paper proposes the adaptation of Transformer on directed graphs and designs two types of direction-aware positional encodings.
Trapdoor Normalization with Irreversible Ownership Verification
Hanwen Liu (Peking University), Yadong MU
ClassificationOptimizationKnowledge DistillationImageGraph
🎯 What it does: Proposes the Trapdoor Normalization (TdN) scheme, embedding an irreversible watermark in the normalization layer of deep models to achieve ownership verification;
Traversing Between Modes in Function Space for Fast Ensembling
Eunggu Yun, Juho Lee (AITRICS)
ClassificationComputational EfficiencyKnowledge DistillationConvolutional Neural NetworkImage
🎯 What it does: The researchers proposed a 'Bridge Network' that directly predicts the output of deep neural networks in the low-loss subspace of Bézier curves using a small number of features, thereby avoiding the need for full forward propagation for each subspace model during the inference phase.
Trompt: Towards a Better Deep Neural Network for Tabular Data
Kuan-Yu Chen (SinoPac Holdings), Tien-Hao Chang
Prompt EngineeringTabularBenchmark
🎯 What it does: A new deep learning model for tabular data based on a Prompt learning framework, called Trompt, is proposed, which separates the intrinsic information of columns from the feature importance of sample specificity.
Truncating Trajectories in Monte Carlo Reinforcement Learning
Riccardo Poiani (Politecnico di Milano), Marcello Restelli (Politecnico di Milano)
OptimizationReinforcement Learning
🎯 What it does: A budget-based non-uniform trajectory truncation strategy (DCS) is proposed, which minimizes the confidence interval of expected return estimates by using trajectories of different lengths in Monte Carlo estimation, and this strategy is applied to existing offline policy optimization algorithms.
Trustworthy Policy Learning under the Counterfactual No-Harm Criterion
Haoxuan Li (Peking University), Peng Wu (Beijing Technology and Business University)
Recommendation SystemOptimizationReinforcement LearningTabular
🎯 What it does: A trustworthy policy learning method that meets the individual no-harm criterion is proposed, constructing a no-harm upper bound based on potential outcomes and the principal stratification theory, and providing proofs of consistency and asymptotic normality.
Tuning Computer Vision Models With Task Rewards
André Susano Pinto (Google DeepMind), Xiaohua Zhai (Google DeepMind)
Object DetectionSegmentationTransformerReinforcement LearningImage
🎯 What it does: First, a general visual model is pre-trained using maximum likelihood, and then fine-tuned with REINFORCE using a task-related reward function, aiming to align the model's predictions with the actual task risks.
Tuning Language Models as Training Data Generators for Augmentation-Enhanced Few-Shot Learning
Yu Meng (University of Illinois Urbana Champaign), Jiawei Han (University of Illinois Urbana Champaign)
ClassificationData SynthesisMeta LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes a method in the few-shot learning scenario where high-quality, label-discriminative synthetic training data is generated by fine-tuning a self-regressive pre-trained language model, and then these data are used to augment the original training set to improve the performance of the classification model.
Two Losses Are Better Than One: Faster Optimization Using a Cheaper Proxy
Blake Woodworth (Inria), Francis Bach (Inria)
OptimizationConvolutional Neural NetworkImageTabular
🎯 What it does: An approximate point iteration algorithm PROXYPROX is proposed, which combines an easily accessible proxy loss function with the original objective function to achieve accelerated optimization using single-step primal gradients and the Bregman upper bound of the proxy.
Two-Scale Gradient Descent Ascent Dynamics Finds Mixed Nash Equilibria of Continuous Games: A Mean-Field Perspective
Yulong Lu (University of Massachusetts)
OptimizationGenerative Adversarial Network
🎯 What it does: This paper analyzes the dynamics of two-scale Mean-Field Gradient Descent Ascent (GDA) and proves that in continuous zero-sum games with entropy regularization, this dynamics can exponentially converge to a unique mixed Nash equilibrium at any positive temperature; it also provides the convergence of the annealed version of the dynamics to the mixed Nash equilibrium of the original unregularized objective when the temperature decays logarithmically over time.
UMD: Unsupervised Model Detection for X2X Backdoor Attacks
Zhen Xiang (University of Illinois), Bo Li (University of Illinois)
Anomaly DetectionImage
🎯 What it does: An unsupervised model detection method UMD is proposed for detecting X2X backdoor attacks with arbitrary source and target class combinations, and inferring all backdoor class pairs;
Uncertain Evidence in Probabilistic Models and Stochastic Simulators
Andreas Munk (University of British Columbia), Frank Wood (Inverted AI)
Time SeriesPhysics Related
🎯 What it does: This paper explores different interpretations and methods for handling uncertain evidence in Bayesian inference, and theoretically derives and experimentally validates the applicability and consistency of three common methods—Jeffrey's rule, virtual evidence, and distributed evidence—under different scenarios; it also provides four categories of classification and handling criteria for uncertain evidence.
Uncertainty Estimation by Fisher Information-based Evidential Deep Learning
Danruo DENG, Pheng-Ann Heng (Chinese University of Hong Kong)
ClassificationAnomaly DetectionImage
🎯 What it does: The study proposes an Evidential Deep Learning (I-EDL) framework based on the Fisher Information Matrix, which improves evidence learning with dynamic weights and negative log-determinant regularization to enhance the reliability of uncertainty estimation.
Uncertainty Estimation for Molecules: Desiderata and Methods
Tom Wollschläger (Technical University of Munich), Stephan Günnemann (Technical University of Munich)
Graph Neural NetworkGraph
🎯 What it does: This paper proposes a confidence estimation method LNK for predicting molecular force fields and systematically evaluates the shortcomings of existing UE methods. It defines six requirements at the physical and application levels and ultimately presents a localized neural kernel model that can obtain energy and force uncertainty in a single forward pass.
Unconstrained Online Learning with Unbounded Losses
Andrew Jacobsen (University of Alberta), Ashok Cutkosky (Boston University)
Optimization
🎯 What it does: This paper proposes a new online learning algorithm suitable for unbounded domains and non-Lipschitz loss, ensuring dynamic regret minimization under these conditions.
Uncovering Adversarial Risks of Test-Time Adaptation
Tong Wu (Princeton University), Prateek Mittal (Princeton University)
Domain AdaptationOptimizationAdversarial AttackImage
🎯 What it does: This study investigates the robustness of Test-Time Adaptation (TTA) against distribution shifts, but finds that mutual influence within batches leads to security vulnerabilities, and proposes a Distribution Intrusion Attack (DIA) framework that can attack TTA.
Under-Counted Tensor Completion with Neural Incorporation of Attributes
Shahana Ibrahim (Oregon State University), Eugene Seo (Brown University)
Tabular
🎯 What it does: A method for low-rank Poisson tensor completion based on a low-rank Poisson tensor model and a nonlinear attribute neural network is proposed;
Understand and Modularize Generator Optimization in ELECTRA-style Pretraining
Chengyu Dong (University of California), Xiaodong Liu (Microsoft Research)
GenerationOptimizationTransformerTextBenchmark
🎯 What it does: In ELECTRA-style pre-training, the system analyzes the impact of generator capacity on discriminator performance and proposes to control generator training by completely decoupling the optimizers of the generator and discriminator, significantly reducing the model's sensitivity to generator size and improving downstream task performance.
Understanding and Defending Patched-based Adversarial Attacks for Vision Transformer
Liang Liu (University of Pittsburgh), Jun Yang (University of Pittsburgh)
Adversarial AttackTransformerImage
🎯 What it does: Study the mechanism of ViT against attention-based adversarial patch attacks and propose the ARMOR defense scheme.
Understanding and Generalizing Contrastive Learning from the Inverse Optimal Transport Perspective
Liangliang Shi (Shanghai Jiao Tong University), Junchi Yan (Shanghai Jiao Tong University)
OptimizationRepresentation LearningContrastive LearningImage
🎯 What it does: This paper proposes to view contrastive learning as a set matching problem and learns feature representations through the Inverse Optimal Transport (IOT) framework.
Understanding Backdoor Attacks through the Adaptability Hypothesis
Xun Xian (University of Minnesota), Jie Ding (University of Minnesota)
Adversarial AttackConvolutional Neural NetworkImage
🎯 What it does: This paper studies the essential mechanism of backdoor attacks through theoretical analysis and experimental validation, proposes and verifies the 'adaptive hypothesis', and provides design insights for attack and defense.
Understanding Gradient Regularization in Deep Learning: Efficient Finite-Difference Computation and Implicit Bias
Ryo Karakida (Artificial Intelligence Research Center AIST), Kazuki Osawa (ETH Zurich)
OptimizationComputational EfficiencyConvolutional Neural NetworkImage
🎯 What it does: This paper studies the implementation and theory of Gradient Regularization (GR) in deep learning, proposing the use of the finite difference method (forward/few-difference) to compute the gradient regularization term, significantly reducing computational costs and improving generalization performance.
Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing
Jikai Jin (Peking University), Jason D. Lee (Princeton University)
OptimizationTabular
🎯 What it does: A fine-grained analysis of gradient descent (GD) in matrix sensing problems with RIP constraints is conducted, proving that GD approaches the optimal rank solution sequentially under small initialization, forming an increasing learning process; convergence conclusions are provided in both under-parameterized and over-parameterized settings.
Understanding Int4 Quantization for Language Models: Latency Speedup, Composability, and Failure Cases
Xiaoxia Wu (Microsoft), Yuxiong He (Microsoft)
Computational EfficiencyKnowledge DistillationTransformerLarge Language ModelText
🎯 What it does: A feasibility study on INT4 weight and activation quantization for Transformer language models is conducted, and an end-to-end INT4 inference pipeline is constructed.
Understanding Oversquashing in GNNs through the Lens of Effective Resistance
Mitchell Black (Oregon State University), Yusu Wang (University of California San Diego)
OptimizationRepresentation LearningGraph Neural NetworkGraph
🎯 What it does: This paper analyzes and utilizes effective resistance to measure the over-compression problem of GNNs, and proposes a reconnection algorithm based on total effective resistance to alleviate over-compression.
Understanding Plasticity in Neural Networks
Clare Lyle (Google DeepMind), Will Dabney (Google DeepMind)
Reinforcement LearningImage
🎯 What it does: This study investigates the phenomenon of decreased plasticity in deep reinforcement learning networks when facing non-stationary targets, systematically evaluating its causes and proposing countermeasures.
Understanding Self-Distillation in the Presence of Label Noise
Rudrajit Das (University of Texas at Austin), sujay sanghavi
ClassificationKnowledge DistillationConvolutional Neural NetworkImage
🎯 What it does: This paper studies the impact of self-distillation (SD) on model performance in supervised learning scenarios with label noise through theoretical derivation and experimental validation, focusing on linear regression and logistic regression problems.
Understanding Self-Predictive Learning for Reinforcement Learning
Yunhao Tang (Google DeepMind), Michal Valko (Google DeepMind)
Representation LearningReinforcement LearningSequentialOrdinary Differential Equation
🎯 What it does: This paper studies and improves the self-predictive learning algorithm in reinforcement learning through theoretical analysis and experiments, proposing bidirectional self-predictive learning to obtain richer and more stable representations.
Understanding the Complexity Gains of Single-Task RL with a Curriculum
Qiyang Li (University of California Berkeley), Sergey Levine (University of California Berkeley)
Computational EfficiencyReinforcement Learning
🎯 What it does: By constructing a task curriculum, a single RL problem is transformed into multi-task learning, thereby improving solving efficiency without using exploration rewards.
Understanding the Distillation Process from Deep Generative Models to Tractable Probabilistic Circuits
Xuejie Liu (Peking University), Yitao Liang (Beijing Institute for General Artificial Intelligence)
GenerationKnowledge DistillationImage
🎯 What it does: By distilling the latent variable information from deep generative models (such as VQ-VAE) into tractable probabilistic circuits (PC), the PC can achieve higher likelihood performance on image modeling tasks;
Understanding the Impact of Adversarial Robustness on Accuracy Disparity
Yuzheng Hu (University of Illinois), Han Zhao (University of Illinois)
ClassificationOptimizationAdversarial AttackImage
🎯 What it does: This paper theoretically analyzes the impact of adversarial robustness on standard accuracy and accuracy differences under Gaussian mixture models and stable distributions, revealing two mechanisms: directional changes and norm shrinkage caused by robustness constraints.
Understanding the Role of Feedback in Online Learning with Switching Costs
Duo Cheng (Virginia Tech), Bo Ji (Virginia Tech)
OptimizationReinforcement Learning
🎯 What it does: This paper studies the impact of feedback type and quantity on maximizing loss in online learning with switching costs, providing optimal regret lower and upper bounds under two settings: additional observation budget and total observation budget.
Unearthing InSights into Mars: Unsupervised Source Separation with Limited Data
Ali Siahkoohi (Rice University), Taichi Kawamura (Institut de Physique du Globe de Paris)
Time SeriesPhysics Related
🎯 What it does: This paper proposes an unsupervised source separation method in data-scarce environments, utilizing wavelet scattering covariance to separate mixed signals and remove thermally induced micro-tilts (glitches) from the seismic data of the NASA InSight mission.
Unifying Molecular and Textual Representations via Multi-task Language Modelling
Dimitrios Christofidellis (IBM Research Europe), Matteo Manica (IBM Research Europe)
GenerationDrug DiscoveryTransformerLarge Language ModelPrompt EngineeringTextMultimodality
🎯 What it does: A multi-task multi-domain language model Text+Chem T5 has been developed, capable of simultaneously handling chemical and natural language tasks such as forward reaction prediction, retrosynthesis, molecular generation, and molecular description;
Unifying Nesterov's Accelerated Gradient Methods for Convex and Strongly Convex Objective Functions
Jungbin Kim (Seoul National University), Insoon Yang (Seoul National University)
OptimizationTabularOrdinary Differential Equation
🎯 What it does: A unified framework for Nesterov accelerated gradient methods is proposed, which can handle both convex and strongly convex functions, and corresponding continuous-time ODEs and discrete algorithms are provided.
Unit Scaling: Out-of-the-Box Low-Precision Training
Charlie Blake (Graphcore), Carlo Luschi (Graphcore)
TransformerLarge Language ModelText
🎯 What it does: Proposed a Unit Scaling technique that maintains weights, activations, and gradients at unit variance during initialization with a fixed scaling factor in both forward and backward propagation, simplifying low-precision (FP16/FP8) training;
Universal Morphology Control via Contextual Modulation
Zheng Xiong (University of Oxford), Shimon Whiteson (University of Oxford)
Robotic IntelligenceTransformerReinforcement LearningTabularBenchmark
🎯 What it does: A hierarchical controller architecture called ModuMorph is proposed, which utilizes a supernetwork to generate morphology-based parameters and modulates the Transformer with a fixed attention mechanism, allowing the same network to adapt to various robotic morphologies.
Universal Physics-Informed Neural Networks: Symbolic Differential Operator Discovery with Sparse Data
Lena Podina (University of Waterloo), Mohammad Kohandel (University of Waterloo)
Time SeriesPhysics RelatedOrdinary Differential Equation
🎯 What it does: Using the Universal PINN method to discover unknown terms in differential equations under sparse noisy data, integrating PINN loss with UDE structure;
Unleashing Mask: Explore the Intrinsic Out-of-Distribution Detection Capability
Jianing Zhu (Hong Kong Baptist University), Bo Han (Hong Kong Baptist University)
Anomaly DetectionConvolutional Neural NetworkImage
🎯 What it does: This paper proposes Unleashing Mask (UM) and its pruning variant UMAP for trained classification models, which identify and forget the model's memory of anomalous samples through masking, thereby improving OOD detection performance in the absence of auxiliary samples.
Unlocking Slot Attention by Changing Optimal Transport Costs
Yan Zhang (Samsung), Cees G. M. Snoek (University of Amsterdam)
OptimizationImageVideo
🎯 What it does: An improved method of Slot Attention based on Optimal Transport is proposed, addressing the issue of indistinguishable information among similar queries caused by the set-equivariance limitation of traditional Slot Attention.
Unscented Autoencoder
Faris Janjos (Robert Bosch GmbH), J. Marius Zoellner
GenerationData SynthesisAuto EncoderImage
🎯 What it does: A deterministic sampling variational autoencoder based on Unscented Transform (UT) is proposed, replacing KL regularization with Wasserstein metric and further adding decoder gradient smoothing regularization;
Unsupervised Out-of-Distribution Detection with Diffusion Inpainting
Zhenzhen Liu (Cornell University), Kilian Q Weinberger
GenerationAnomaly DetectionDiffusion modelScore-based ModelImage
🎯 What it does: Under the premise of unsupervised learning, a Lift-Map-Detect (LMD) framework is designed utilizing the denoising mapping capability of diffusion models. This involves first performing mask lifting (Lift) on the input image, then using the diffusion model for filling (Map), and finally calculating the perceptual distance between the original and reconstructed images (Detect) to determine whether the sample is OOD.
Unsupervised Skill Discovery for Learning Shared Structures across Changing Environments
Sang-Hyun Lee (Seoul National University), Seung-Woo Seo (Seoul National University)
Robotic IntelligenceReinforcement LearningSequential
🎯 What it does: This paper proposes an unsupervised skill discovery algorithm that can learn and maintain a shared structure in a constantly changing environment, and removes redundant skills through a skill evaluation process.
Unveiling the Latent Space Geometry of Push-Forward Generative Models
Thibaut Issenhuth (Criteo AI Lab), David Picard (LIGM Ecole des Ponts Univ Gustave Eiffel CNRS)
GenerationData SynthesisTransformerAuto EncoderGenerative Adversarial NetworkImage
🎯 What it does: The paper studies the geometric structure of latent space in generative models (such as GANs and VAEs) and proposes that the latent space should be a 'simplicial cluster' to minimize the occurrence of generated samples falling outside the support of the target distribution, and verifies the relationship between this structure and model performance.
Unveiling The Mask of Position-Information Pattern Through the Mist of Image Features
Chieh Hubert Lin (University of California), Ming-Hsuan Yang (University of California)
GenerationData SynthesisConvolutional Neural NetworkImage
🎯 What it does: This study investigates the implicit absolute positional information in padding within convolutional neural networks and proposes a new evaluation method called PPP, systematically analyzing its formation, evolution, and impact on model performance.
UPop: Unified and Progressive Pruning for Compressing Vision-Language Transformers
Dachuan Shi (Tsinghua University), Jiaqi Wang (Shanghai AI Laboratory)
RetrievalCompressionTransformerVision Language ModelImageTextMultimodality
🎯 What it does: A unified and progressive pruning framework called UPop is proposed to compress visual-language Transformer models.
UPSCALE: Unconstrained Channel Pruning
Alvin Wan (Apple), Qi Shan (Apple)
ClassificationOptimizationComputational EfficiencyConvolutional Neural NetworkImage
🎯 What it does: A general export algorithm called UPSCALE is proposed, which can perform channel pruning without the constraints of traditional methods, and eliminates memory copies caused by inconsistent pruning in multi-branch networks through channel reordering, thus balancing high accuracy and low latency.
User-defined Event Sampling and Uncertainty Quantification in Diffusion Models for Physical Dynamical Systems
Marc Anton Finzi, Leonardo Zepeda-Nunez
Diffusion modelTime SeriesPhysics RelatedStochastic Differential Equation
🎯 What it does: Construct a diffusion model to probabilistically model chaotic dynamical trajectories and generate extreme events through posterior conditional sampling, providing uncertainty quantification;
User-level Private Stochastic Convex Optimization with Optimal Rates
Raef Bassily (Ohio State University), Ziteng Sun (Google Research)
OptimizationSafty and Privacy
🎯 What it does: Under user-level differential privacy constraints, the optimal error bounds for stochastic convex optimization are provided.
Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies
Gati Aher (Olin College of Engineering), Adam Tauman Kalai (Microsoft Research)
GenerationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: A zero-shot method named Turing Experiment (TE) is proposed and implemented, utilizing large language models (such as GPT) to simulate multiple 'human' participants, replicating four classic human experiments (Ultimatum Game, garden-path sentences, Milgram obedience experiment, Wisdom of Crowds) and evaluating the model's simulation accuracy.
Using Perturbation to Improve Goodness-of-Fit Tests based on Kernelized Stein Discrepancy
Xing Liu (Imperial College London), Axel Gandy (Imperial College London)
🎯 What it does: By applying target-invariant Markov perturbations to the candidate and target distributions, we improve the goodness-of-fit test based on kernelized Stein discrepancy.
VA-learning as a more efficient alternative to Q-learning
Yunhao Tang (Google DeepMind), Michal Valko (Google DeepMind)
Computational EfficiencyReinforcement LearningTabular
🎯 What it does: The VA-learning algorithm is proposed, which directly learns the advantage function and value function (without relying on the Q-function), achieving higher sample efficiency.
Variance Control for Distributional Reinforcement Learning
Qi Kuang (Shanghai University of Finance and Economics), Fan Zhou (Shanghai University of Finance and Economics)
Reinforcement LearningSequential
🎯 What it does: This paper proposes a new distributed reinforcement learning estimator—Quantiled Expansion Mean (QEM), which significantly reduces the variance of distributed value function estimation through modeling the heteroscedasticity of quantile errors and weighted least squares regression, thereby enhancing learning stability and convergence speed.
Variational Autoencoding Neural Operators
Jacob H Seidman, Paris Perdikaris (University of Pennsylvania)
GenerationData SynthesisSuper ResolutionAuto EncoderGenerative Adversarial NetworkTabular
🎯 What it does: An unsupervised learning framework named Variational Autoencoding Neural Operator (VANO) is proposed, which utilizes operator learning structures for dimensionality reduction and generation of continuous function data.
Variational Curriculum Reinforcement Learning for Unsupervised Discovery of Skills
Seongun Kim (Korea Advanced Institute of Science and Technology), Jaesik Choi (Korea Advanced Institute of Science and Technology)
Robotic IntelligenceReinforcement LearningSequential
🎯 What it does: The Variational Curriculum Reinforcement Learning (VCRL) framework is proposed, and based on this, the Value Uncertainty-based VUVC method is introduced for unsupervised learning of diverse skills and accelerating state space coverage.
Variational Mixture of HyperGenerators for Learning Distributions over Functions
Batuhan Koyuncu (Saarland University), Isabel Valera (Universidad Carlos III de Madrid)
GenerationData SynthesisSuper ResolutionFlow-based ModelAuto EncoderImagePoint Cloud
🎯 What it does: A mixed hypernetwork model VaMoH based on variational autoencoders is proposed to learn distributions in function space, supporting data generation and inference for continuous coordinates.
Variational Open-Domain Question Answering
Valentin Liévin (Technical University of Denmark), Ole Winther (University of Copenhagen)
RetrievalOptimizationTransformerTextBiomedical DataRetrieval-Augmented Generation
🎯 What it does: A retrieval-augmented question answering framework based on variational inference (VOD) is proposed and implemented, capable of end-to-end training of the retriever and reader models, and estimating task likelihood through self-normalized importance sampling.
Variational Sparse Inverse Cholesky Approximation for Latent Gaussian Processes via Double Kullback-Leibler Minimization
Jian Cao (Texas A&M University), Matthias Katzfuss (Texas A&M University)
Tabular
🎯 What it does: A variational Gaussian process method based on a dual Kullback-Leibler optimal sparse inverse Cholesky approximation is proposed.
Vector Quantized Wasserstein Auto-Encoder
Long Tung Vuong, Dinh Phung (Monash University)
GenerationData SynthesisCompressionAuto EncoderImage
🎯 What it does: Proposes VQ-WAE from a generative perspective, aligning the codebook with the data distribution using Wasserstein distance, and provides corresponding theory and training algorithms.
Vector-Valued Control Variates
Zhuo Sun (University College London), Francois-Xavier Briol (University College London)
TabularTime SeriesSequentialPhysics Related
🎯 What it does: Vector-valued control variates (vv-CV) are proposed to simultaneously reduce the variance of multiple related integral estimates.
VectorMapNet: End-to-end Vectorized HD Map Learning
Yicheng Liu (Shanghai Qi Zhi Institute), Hang Zhao (Tsinghua University)
Object DetectionGenerationAutonomous DrivingTransformerPoint Cloud
🎯 What it does: An end-to-end VectorMapNet framework is proposed, which directly predicts HD maps in sparse polygon (polyline) format from vehicle sensor inputs, avoiding traditional rasterization and post-processing steps.
Vertical Federated Graph Neural Network for Recommender System
Peihua Mai (National University of Singapore), Yan Pang (National University of Singapore)
Recommendation SystemFederated LearningSafty and PrivacyGraph Neural NetworkGraph
🎯 What it does: Proposed VerFedGNN, a graph neural network recommendation framework in the vertical federated learning scenario;
VIMA: Robot Manipulation with Multimodal Prompts
Yunfan Jiang (Stanford University), Linxi Fan (NVIDIA)
Robotic IntelligenceTransformerReinforcement LearningPrompt EngineeringVision Language ModelTextMultimodalityBenchmark
🎯 What it does: This paper proposes a unified framework for multimodal prompts (interleaving text and images), allowing various robotic operation tasks to be unified as a sequence modeling problem, and based on this, constructs the VIMA-BENCH benchmark and the VIMA robotic agent.
Von Mises Mixture Distributions for Molecular Conformation Generation
Kirk Swanson (University of Chicago), Eric M Jonas
GenerationDrug DiscoveryGraph Neural NetworkGraph
🎯 What it does: This paper proposes a new graph neural network, VonMisesNet, for directly generating 3D molecular conformations that satisfy the Boltzmann distribution from molecular structures.
Warm-Start Actor-Critic: From Approximation Error to Sub-optimality Gap
Hang Wang (University of California), Junshan Zhang (University of California)
OptimizationReinforcement Learning
🎯 What it does: This paper conducts a finite-time error analysis of the Warm-Start Actor-Critic algorithm, quantifying the impact of approximation error on the suboptimality gap, and provides upper and lower bounds.
Wasserstein Barycenter Matching for Graph Size Generalization of Message Passing Neural Networks
Xu Chu (Tsinghua University), Hong Mei (Peking University)
Graph Neural NetworkGraph
🎯 What it does: A Wasserstein barycenter matching (WBM) layer is proposed, embedded into a message passing neural network (MPNN) to enhance the graph size generalization capability;
Weak Proxies are Sufficient and Preferable for Fairness with Missing Sensitive Attributes
Zhaowei Zhu (University of California, Santa Cruz), Yang Liu (ByteDance Research)
Tabular
🎯 What it does: A fairness assessment method based on weak proxies is proposed, which can accurately estimate group fairness in the absence of sensitive attributes and provide theoretical guarantees;
Weakly Supervised Regression with Interval Targets
Xin Cheng (Chongqing University), Lei Feng (Nanyang Technological University)
Convolutional Neural NetworkTabular
🎯 What it does: This paper studies the interval target problem (RIT) in weakly supervised regression, proposing a statistical generative model and a convergence-consistent method based on interval constraints, and verifying its superiority through theoretical analysis and experimental validation.
Weighted Flow Diffusion for Local Graph Clustering with Node Attributes: an Algorithm and Statistical Guarantees
Shenghao Yang (University of Waterloo), Kimon Fountoulakis (University of Waterloo)
Graph Neural NetworkFlow-based ModelGraph
🎯 What it does: Proposes a weighted flow diffusion local graph clustering algorithm that utilizes node attributes.
Weighted Sampling without Replacement for Deep Top-$k$ Classification
Dieqiao Feng (Cornell University), Bart Selman (Cornell University)
ClassificationReinforcement LearningImage
🎯 What it does: This paper proposes a top-k loss function based on Weighted Sampling Without Replacement (WSWR) for training deep networks to improve top-k classification accuracy.