ICML 2023 Papers — Page 12
International Conference on Machine Learning · 1828 papers
Nonlinear Advantage: Trained Networks Might Not Be As Complex as You Think
Christian H.X. Ali Mehmeti-Göpel (University of Mainz), Jan Disselhoff (University of Mainz)
ClassificationConvolutional Neural NetworkTransformerImageText
🎯 What it does: This paper explores the 'nonlinear advantage' of deep networks by partially linearizing the channel-level nonlinear units (PReLU linearization) after training, and proposes Average Path Length (APL) and its normalized version (NAPL) to measure the effective depth and width of the network.
Nonlinear Causal Discovery with Latent Confounders
David Kaltenpoth (CISPA Helmholtz Center for Information Security), Jilles Vreeken (CISPA Helmholtz Center for Information Security)
Auto EncoderGraph
🎯 What it does: A method is proposed to discover nonlinear causal structures from observational data in the presence of potential confounding factors.
Nonparametric Density Estimation under Distribution Drift
Alessio Mazzetto (Brown University), Eli Upfal (Brown University)
🎯 What it does: This study investigates the limit risk of nonparametric density estimation in the context of distribution drift and provides optimal estimation methods.
Nonparametric Extensions of Randomized Response for Private Confidence Sets
Ian Waudby-Smith (Carnegie Mellon University), Aaditya Ramdas (Carnegie Mellon University)
Safty and PrivacyTabularTime Series
🎯 What it does: This paper proposes a non-parametric random response mechanism (NPRR) and constructs non-parametric, non-asymptotic confidence intervals (CI) for the overall mean and time-uniform confidence sequences (CS) under local differential privacy (LDP) constraints, achieving private statistical inference for bounded random variables.
Nonparametric Generative Modeling with Conditional Sliced-Wasserstein Flows
Chao Du (Sea AI Lab), Min Lin (Sea AI Lab)
RestorationGenerationData SynthesisFlow-based ModelImage
🎯 What it does: A parameter-free generative model CSWF is proposed, which can directly perform conditional generation and image inpainting through joint Sliced-Wasserstein Flow (SWF).
Nonparametric Iterative Machine Teaching
Chen Zhang (Jilin University), James Kwok
OptimizationImage
🎯 What it does: A non-parametric iterative machine teaching (NIMT) framework is proposed, along with the design of stochastic and greedy functional teaching algorithms.
Normalizing Flows for Interventional Density Estimation
Valentyn Melnychuk (Ludwig Maximilian University of Munich), Stefan Feuerriegel (Ludwig Maximilian University of Munich)
Flow-based ModelTabular
🎯 What it does: This paper proposes a fully parametric deep learning method called Interventional Normalizing Flows (INFs) for estimating the intervention distribution density of potential outcomes from observational data.
Not All Semantics are Created Equal: Contrastive Self-supervised Learning with Automatic Temperature Individualization
Zi-Hao Qiu (Nanjing University), Tianbao Yang (Texas A&M University)
ClassificationRetrievalOptimizationRepresentation LearningContrastive LearningImageMultimodality
🎯 What it does: A robust global contrastive loss based on Distributionally Robust Optimization (DRO) is proposed, and unsupervised contrastive learning is implemented through the adaptive temperature individualized algorithm iSogCLR.
Not all Strongly Rayleigh Distributions Have Small Probabilistic Generating Circuits
Markus Bläser (Saarland University)
🎯 What it does: Proved that there exists a strong Rayleigh distribution that cannot be represented by polynomial-sized probabilistic circuits.
NP-SemiSeg: When Neural Processes meet Semi-Supervised Semantic Segmentation
Jianfeng Wang (University of Oxford), Thomas Lukasiewicz (Vienna University of Technology)
SegmentationImage
🎯 What it does: This paper proposes NP-SemiSeg, which introduces the Neural Process framework into semi-supervised semantic segmentation, achieving quantifiable pixel-level uncertainty.
NTK-approximating MLP Fusion for Efficient Language Model Fine-tuning
Tianxin Wei (University of Illinois Urbana-Champaign), Jingrui He (University of Illinois Urbana-Champaign)
CompressionComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: A method for MLP fusion based on NTK approximation is proposed, which compresses the feedforward network of pre-trained language models in a single step to achieve efficient fine-tuning.
Nugget: Neural Agglomerative Embeddings of Text
Guanghui Qin (Johns Hopkins University), Benjamin Van Durme (Johns Hopkins University)
RetrievalCompressionRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: A text encoding method called NUGGET has been developed, which can dynamically generate multi-vector representations. It utilizes hard attention to select key tokens to generate 'nugget' vectors, supporting self-encoding and machine translation pre-training, and is applied to text similarity retrieval and long text language modeling.
NUNO: A General Framework for Learning Parametric PDEs with Non-Uniform Data
Songming Liu (Tsinghua University), Jun Zhu (Tsinghua University)
Point CloudMeshPhysics RelatedOrdinary Differential Equation
🎯 What it does: A general framework named NUNO is proposed, which decomposes non-uniform physical data through K-D tree domain decomposition and employs different sizes of uniform grids in each subdomain, followed by using mesh-based neural operators based on FFT or convolution to complete PDE predictions.
OCD: Learning to Overfit with Conditional Diffusion Models
Shahar Lutati (Tel Aviv University), Lior Wolf (Tel Aviv University)
ClassificationGenerationConvolutional Neural NetworkDiffusion modelImageTabularAudio
🎯 What it does: A conditional diffusion model supernetwork (OCD) has been constructed to predict fine-tuning weights relative to a baseline network for each input sample, achieving overfitting on a single sample without the need for real-time gradient updates.
ODS: Test-Time Adaptation in the Presence of Open-World Data Shift
Zhi Zhou (Nanjing University), Yu-Feng Li (Nanjing University)
Domain AdaptationOptimizationImage
🎯 What it does: This paper proposes an open-world data transfer adaptation framework (ODS) that can simultaneously address covariate shift and label distribution shift during the testing phase.
Off-Policy Average Reward Actor-Critic with Deterministic Policy Search
Naman Saxena (Indian Institute of Science), Shalabh Bhatnagar (Indian Institute of Science)
Reinforcement LearningSequentialOrdinary Differential Equation
🎯 What it does: A deterministic policy gradient theorem under average reward is proposed, and based on this, an offline average reward deep deterministic policy gradient algorithm ARO-DDPG is implemented.
Off-Policy Evaluation for Large Action Spaces via Conjunct Effect Modeling
Yuta Saito (Cornell University), Thorsten Joachims (Cornell University)
Reinforcement LearningTabular
🎯 What it does: This study focuses on offline policy evaluation in large action spaces and proposes the OffCEM estimator based on clustering effects and residual effects.
Offline Learning in Markov Games with General Function Approximation
Yuheng Zhang (University of Illinois at Urbana-Champaign), Nan Jiang (University of Illinois at Urbana-Champaign)
Reinforcement Learning
🎯 What it does: This paper proposes a unified offline multi-agent reinforcement learning framework that can learn approximate Nash equilibria, correlated equilibria, and coarse correlated equilibria in Markov games with universal function approximation.
Offline Meta Reinforcement Learning with In-Distribution Online Adaptation
Jianhao Wang (Institute for Interdisciplinary Information Sciences), Chongjie Zhang (Institute for Interdisciplinary Information Sciences)
Robotic IntelligenceMeta LearningReinforcement LearningTabularBenchmark
🎯 What it does: This paper explores the transfer-reward distribution shift problem between offline data and online adaptation in offline meta-reinforcement learning (offline meta-RL), and proposes an 'In-Distribution Online Adaptation with Uncertainty Quantification (IDAQ)' framework based on uncertainty quantification, enabling rapid adaptation to new tasks in the online phase without additional context.
Offline Reinforcement Learning with Closed-Form Policy Improvement Operators
Jiachen Li (University of California Santa Barbara), William Yang Wang (University of California Santa Barbara)
Reinforcement LearningTabularBenchmark
🎯 What it does: This paper proposes a Closed-Form Policy Improvement (CFPI) operator for Behavior-Constrained Policy Optimization (BCPO) in offline reinforcement learning, providing closed-form solutions under single Gaussian and Gaussian mixture behavior policies, and further implements first-order and iterative versions of offline RL algorithms.
Omnipredictors for Constrained Optimization
Lunjia Hu (Stanford University), Chutong Yang (Stanford University)
Optimization
🎯 What it does: This paper proposes a new omnipredictor for constrained optimization problems and studies its complexity and impact. The omnipredictor allows learners to define these constraints based on known subpopulations without knowing the loss functions and constraints that will be applied subsequently.
OMS-DPM: Optimizing the Model Schedule for Diffusion Probabilistic Models
Enshu Liu (Tsinghua University), Yu Wang (Tsinghua University)
GenerationOptimizationRecurrent Neural NetworkDiffusion modelImage
🎯 What it does: Proposes to enhance generation speed and quality by assigning different models to the diffusion probability model to construct a model scheduling.
On Balancing Bias and Variance in Unsupervised Multi-Source-Free Domain Adaptation
Maohao Shen (Massachusetts Institute of Technology), Gregory Wornell (Massachusetts Institute of Technology)
Domain AdaptationImage
🎯 What it does: This paper analyzes the generalization error of Multi-Source Unsupervised Domain Adaptation (MSFDA) through information theory, revealing the essence of the bias-variance trade-off, and based on this, proposes a novel algorithm that combines domain aggregation, selective pseudo-labeling, and joint feature alignment.
On Bridging the Gap between Mean Field and Finite Width Deep Random Multilayer Perceptron with Batch Normalization
Amir Joudaki (ETH Zurich), Francis Bach (INRIA ENS PSL Paris)
Tabular
🎯 What it does: This paper demonstrates that batch normalization (BN) can suppress error amplification caused by depth by analyzing the mean field prediction of the Gram matrix of hidden representations during the initialization of deep multilayer perceptrons (MLP), and provides a non-asymptotic convergence upper bound for networks of finite width.
On Computing Optimal Tree Ensembles
Christian Komusiewicz (Friedrich Schiller University Jena), Manuel Sorge (Vienna University of Technology)
ClassificationOptimization
🎯 What it does: This paper studies how to construct a minimal set of decision trees (tree cluster) that can classify correctly under the premise of a given training dataset and number of trees.
On Coresets for Clustering in Small Dimensional Euclidean spaces
Lingxiao Huang (Nanjing University), Xuan Wu (Huawei)
OptimizationComputational Efficiency
🎯 What it does: This paper studies the core set problem of constructing k-MEDIAN in low-dimensional Euclidean spaces and proposes an efficient data compression method to alleviate the computational burden of processing large data.
On Data Manifolds Entailed by Structural Causal Models
Ricardo Dominguez-Olmedo (Max Planck Institute for Intelligent Systems), Bernhard Schölkopf (Max Planck Institute for Intelligent Systems)
OptimizationExplainability and InterpretabilityTabular
🎯 What it does: This paper studies the data manifold induced by Structural Causal Models (SCM) and constructs a metric based on Riemannian geometry, subsequently using this metric to generate both realistic and causally consistent counterfactual explanations; it also proposes a method for optimization on the manifold to achieve causal algorithmic recourse.
On Distribution Dependent Sub-Logarithmic Query Time of Learned Indexing
Sepanta Zeighami (University of Southern California), Cyrus Shahabi (University of Southern California)
Tabular
🎯 What it does: The paper constructs a learning-based index and theoretically proves that it can achieve expected sub-logarithmic or constant time lookups under mild distribution assumptions.
On Enhancing Expressive Power via Compositions of Single Fixed-Size ReLU Network
Shijun Zhang (Duke University), Hongkai Zhao (Duke University)
Recurrent Neural Network
🎯 What it does: This paper explores the approximation capability of a single fixed-size ReLU network through multiple compositions in approximating arbitrary continuous functions.
On Excess Mass Behavior in Gaussian Mixture Models with Orlicz-Wasserstein Distances
Aritra Guha (AT&T Chief Data Office), XuanLong Nguyen (University of Michigan)
🎯 What it does: This paper proposes and studies a new metric called Orlicz-Wasserstein distance, which is used to measure the posterior convergence rate of the parameters (mixture distributions) of the Dirichlet Process Gaussian Mixture Model (DPGMM) and provides theoretical convergence bounds.
On Heterogeneous Treatment Effects in Heterogeneous Causal Graphs
Richard A Watson, Rui Song (North Carolina State University)
Score-based ModelGraphTabular
🎯 What it does: A method for modeling and estimating heterogeneous treatment effects (HCE) in heterogeneous causal graphs (HCG) is proposed.
On Investigating the Conservative Property of Score-Based Generative Models
Chen-Hao Chao (National Tsing Hua University), Chun-Yi Lee (National Tsing Hua University)
GenerationData SynthesisScore-based ModelImage
🎯 What it does: This study investigates the conservativeness issue of Score-Based models and proposes a quasi-conservative Score-Based model (QCSBM) that maintains conservativeness by minimizing rotational density through regularization without architectural constraints, and validates its effectiveness.
On Kinetic Optimal Probability Paths for Generative Models
Neta Shaul (Weizmann Institute of Science), Yaron Lipman (Meta AI)
GenerationOptimizationFlow-based ModelImageOrdinary Differential Equation
🎯 What it does: This paper studies the differentiable Gaussian probability path space, proposing a single one-dimensional 'data separation function' to simplify the expression of kinetic energy, and seeks the kinetic optimal path within this space, providing an analytical ODE solution and a theoretical convergence of Cond-OT to optimal in high dimensions; it also presents numerical estimation and training methods for continuous normalizing flows.
On Many-Actions Policy Gradient
Michal Nauman (University of Warsaw), Marek Cygan (University of Warsaw)
Reinforcement LearningSequential
🎯 What it does: This paper studies how the variance of SPG decreases under multi-action sampling and provides conditions under which using multiple actions is more effective than extending trajectories.
On Over-Squashing in Message Passing Neural Networks: The Impact of Width, Depth, and Topology
Francesco Di Giovanni (University of Cambridge), Michael M. Bronstein
Graph Neural NetworkGraph
🎯 What it does: This paper theoretically studies the phenomenon of over-compression in Message Passing Neural Networks (MPNN) and systematically explores the impact of network width, depth, and graph topology on this issue.
On Penalty-based Bilevel Gradient Descent Method
Han Shen (Rensselaer Polytechnic Institute), Tianyi Chen (Rensselaer Polytechnic Institute)
OptimizationTabular
🎯 What it does: This paper proposes a penalty-based bilevel gradient descent method aimed at solving bilevel optimization problems, especially when the lower-level objective function is not strongly convex or is constrained.
On Pitfalls of Test-Time Adaptation
Hao Zhao (École Polytechnique Fédérale de Lausanne), Tao Lin (Westlake University)
Domain AdaptationImageBenchmark
🎯 What it does: This study investigates the practical performance of Test-Time Adaptation (TTA) under different distribution shifts, proposing a unified benchmark framework called TTAB, and systematically evaluates ten mainstream TTA methods.
On Pre-Training for Visuo-Motor Control: Revisiting a Learning-from-Scratch Baseline
Nicklas Hansen (University of California San Diego), Xiaolong Wang
Robotic IntelligenceConvolutional Neural NetworkReinforcement LearningImageVideo
🎯 What it does: This paper re-evaluates the effectiveness of Learning-from-Scratch (LfS) in visual perception-motor control tasks and makes a fair comparison with frozen pre-trained visual models (PVR, MVP, R3M).
On Preemption and Learning in Stochastic Scheduling
Nadav Merlis (ENSAE), Vianney Perchet (Criteo)
OptimizationTabular
🎯 What it does: This paper studies the single-machine scheduling problem, exploring job grouping by type and learning and scheduling when processing times follow an exponential distribution with unknown parameters. It proposes two types of algorithms: non-preemptive (ETC-U, UCB-U) and preemptive (ETC-RR, UCB-RR), and provides competitive ratio and lower bound analysis.
On Provable Copyright Protection for Generative Models
Nikhil Vyas (Harvard School of Engineering and Applied Sciences), Boaz Barak (Harvard School of Engineering and Applied Sciences)
GenerationData SynthesisGenerative Adversarial NetworkImage
🎯 What it does: This paper proposes the definition of 'Near-Access-Free (NAF)' and presents an algorithm for modifying generative models to ensure that copyright data from the training set is not significantly replicated during inference.
On Regularization and Inference with Label Constraints
Kaifu Wang (University of Pennsylvania), Dan Roth (University of Pennsylvania)
ClassificationOptimization
🎯 What it does: This paper compares two methods for encoding label constraints into the machine learning process: one is regularization with constraints by adding a constraint penalty to the training objective, and the other is constrained inference, which enforces the satisfaction of constraints during the inference phase. The theoretical analysis of their impact on optimal risk and generalization error is provided.
On Sampling with Approximate Transport Maps
Louis Grenioux (École Polytechnique), Marylou Gabrié (École Polytechnique)
Flow-based ModelTabular
🎯 What it does: This paper systematically compares three types of sampling methods using approximate transport mappings (normalizing flow), including flow-MCMC, neural-IS, and neutra-MCMC, and conducts empirical and theoretical analyses of their performance under multimodal, unimodal, low-dimensional, high-dimensional, and varying flow learning quality conditions. It also provides a new mixing time upper bound for independent Metropolis-Hastings (IMH) under strongly log-concave targets.
On Second-Order Scoring Rules for Epistemic Uncertainty Quantification
Viktor Bengs (Ludwig Maximilian University of Munich), Willem Waegeman (Ghent University)
🎯 What it does: The study proves that second-order loss functions cannot incentivize learners to accurately represent their knowledge uncertainty like first-order proper scoring rules.
On Strengthening and Defending Graph Reconstruction Attack with Markov Chain Approximation
Zhanke Zhou (Hong Kong Baptist University), Bo Han (Hong Kong Baptist University)
OptimizationAdversarial AttackGraph Neural NetworkGraph
🎯 What it does: This paper systematically studies graph reconstruction attacks (GRA) on graph neural networks (GNN), abstracting the attack problem as an approximation of the original Markov chain, and proposes a chain-based attack method MC-GRA and a chain-based defense method MC-GPB.
On the Complexity of Bayesian Generalization
Yu-Zhe Shi (Peking University), Yixin Zhu (Peking University)
Convolutional Neural NetworkImage
🎯 What it does: This study systematically explores two modes of concept generalization, rule-based and similarity-based, on large-scale natural visual data, and proposes the 'Representativeness of Attributes' (RoA) metric to quantify the subjective complexity of concepts.
On the Connection Between MPNN and Graph Transformer
Chen Cai (University of California San Diego), Yusu Wang (University of California San Diego)
Graph Neural NetworkTransformerGraphBenchmark
🎯 What it does: This paper studies the expressive power of message passing neural networks with virtual nodes (MPNN+VN) under the graph Transformer (GT) framework, and proves that it can approximate the self-attention layer of GT at different depths and widths.
On the Convergence of Federated Averaging with Cyclic Client Participation
Yae Jee Cho (Carnegie Mellon University), Tong Zhang (Hong Kong University of Science and Technology)
OptimizationFederated LearningTabular
🎯 What it does: This paper provides a theoretical framework to analyze the convergence of the Federated Averaging (FedAvg) algorithm under periodic client participation, considering different client optimizers such as GD, SGD, and shuffled SGD.
On the Convergence of Gradient Flow on Multi-layer Linear Models
Hancheng Min (Johns Hopkins University), Enrique Mallada (Johns Hopkins University)
OptimizationTabular
🎯 What it does: This study investigates the convergence of gradient flows in multilayer linear models, proving that under loss functions that satisfy the gradient dominance property, exponential convergence can be achieved as long as the weight initialization meets certain imbalance conditions.
On the Convergence of SARSA with Linear Function Approximation
Shangtong Zhang (University of Virginia), Romain Laroche
Reinforcement Learning
🎯 What it does: This paper studies the SARSA algorithm under linear function approximation and provides a finite sample convergence rate to a bounded region.
On the convergence of the MLE as an estimator of the learning rate in the Exp3 algorithm
Julien Aubert (Universite Cote d'Azur), Patricia Reynaud-Bouret (Universite Cote d'Azur)
Sequential
🎯 What it does: This paper studies the convergence properties of maximum likelihood estimation (MLE) for the learning rate when using the Exp3 algorithm, providing theoretical proofs and numerical simulations.
On the Convergence Rate of Gaussianization with Random Rotations
Felix Draxler (Heidelberg University), Ullrich Koethe
Flow-based ModelImage
🎯 What it does: This paper analyzes the convergence rate of the Gaussianization model under random rotation, proving that the number of layers required in high dimensions is at least linearly related to the dimension.
On the Correctness of Automatic Differentiation for Neural Networks with Machine-Representable Parameters
Wonyeol Lee (Stanford University), Alex Aiken (Stanford University)
🎯 What it does: Research on the automatic differentiation correctness of neural networks under machine-representable parameters
On the Effectiveness of Offline RL for Dialogue Response Generation
Paloma Sodhi (ASAPP), Ryan McDonald (ASAPP)
TransformerReinforcement LearningText
🎯 What it does: This paper improves task-oriented dialogue generation models through offline reinforcement learning methods, exploring the effects of teacher forcing, decision transformers, and implicit Q-learning.
On the Estimation of Gaussian Mixture Copula Models
ASHUTOSH TEWARI
Gaussian SplattingTabular
🎯 What it does: This paper re-examines the high-dimensional Gaussian mixture copula model (GMCM), addressing issues such as parameter non-identifiability, intractable likelihood, and difficult gradient computation, and proposes a feasible estimation scheme, which is validated through experiments.
On the Expressive Power of Geometric Graph Neural Networks
Chaitanya K. Joshi (University of Cambridge), Pietro Lio
Graph Neural NetworkGraph
🎯 What it does: This paper studies the expressive power of geometric graph neural networks, proposing the geometric Weisfeiler–Leman (GWL) test and using it to analyze the design space of G-equivariant and G-invariant GNNs.
On the Forward Invariance of Neural ODEs
Wei Xiao (Massachusetts Institute of Technology), Daniela Rus (Massachusetts Institute of Technology)
Autonomous DrivingOptimizationSafty and PrivacyReinforcement LearningOrdinary Differential Equation
🎯 What it does: This study proposes a forward invariance propagation method that utilizes Control Barrier Functions (CBF) and Higher-Order CBF (HOCBF) to ensure that neural ODEs meet predefined output specifications, such as safety constraints or physical laws, during inference and training.
On the Functional Similarity of Robust and Non-Robust Neural Representations
András Balogh (University of Szeged), Márk Jelasity (ELKH Szeged Research Group on Artificial Intelligence)
Representation LearningAdversarial AttackConvolutional Neural NetworkImage
🎯 What it does: This study investigates the functional similarity of internal representations between adversarially robust networks and standard networks, and empirically validates this through model stitching at different levels.
On the Generalization of Multi-modal Contrastive Learning
Qi Zhang (Peking University), Yisen Wang (Peking University)
Representation LearningTransformerContrastive LearningImageTextMultimodality
🎯 What it does: This paper studies the theoretical generality of multimodal contrastive learning (MMCL) and provides an upper bound on the generalization error for visual downstream tasks; it explains why MMCL outperforms self-supervised contrastive learning (SSCL) in downstream tasks through a unified perspective and enhances the performance of SimCLR using multimodal information from CLIP.
On the Global Convergence of Fitted Q-Iteration with Two-layer Neural Network Parametrization
Mudit Gaur (Purdue University), Mridul Agarwal (Amazon)
OptimizationReinforcement Learning
🎯 What it does: This paper studies a fitting Q-iteration algorithm based on a two-layer ReLU neural network parameterization and finds the sample complexity guarantees for this algorithm.
On the Global Convergence of Risk-Averse Policy Gradient Methods with Expected Conditional Risk Measures
Xian Yu (Ohio State University), Lei Ying (Michigan University)
OptimizationReinforcement LearningSequential
🎯 What it does: This paper studies the risk-sensitive policy gradient method using the Expected Conditional Risk Measure (ECRM) and provides an analysis of global convergence and iterative complexity.
On the Identifiability and Estimation of Causal Location-Scale Noise Models
Alexander Immer (ETH Zurich), Alexander Marx (ETH Zurich)
Tabular
🎯 What it does: Proposed and analyzed the causal identifiability of the Location-Scale Noise Model (LSNM), and provided two consistent estimation methods (based on feature mapping and neural networks);
On the Impact of Algorithmic Recourse on Social Segregation
Ruijiang Gao (University of Texas at Austin), Himabindu Lakkaraju (Harvard University)
Auto EncoderGenerative Adversarial NetworkTabular
🎯 What it does: This paper studies the delayed impact of algorithmic recursion on social segregation, proving that existing recursive methods exacerbate the characteristic differences between different groups;
On the Impact of Knowledge Distillation for Model Interpretability
Hyeongrok Han (Seoul National University), Sungroh Yoon (Seoul National University)
ClassificationExplainability and InterpretabilityKnowledge DistillationImageText
🎯 What it does: Research shows that knowledge distillation (KD) not only improves model accuracy but also significantly enhances model interpretability.
On the Importance of Feature Decorrelation for Unsupervised Representation Learning in Reinforcement Learning
Hojoon Lee (KAIST), Jaegul Choo (KAIST)
Representation LearningTransformerReinforcement LearningSequential
🎯 What it does: Proposes the SimTPR framework, which utilizes causal prediction and feature decorrelation to learn unsupervised temporal prediction representations, avoiding representation collapse;
On the Initialization of Graph Neural Networks
Jiahang Li (Hong Kong Polytechnic University), David Wipf
Graph Neural NetworkGraph
🎯 What it does: This paper theoretically derives the variance expressions of Graph Neural Networks (GNN) in forward and backward propagation, pointing out that classical initialization methods (such as Xavier, Lecun, Kaiming) overlook the effects of graph structure, message passing, and activation functions. Based on this, a novel initialization scheme called Virgo is proposed, aiming to maintain the balance of forward and backward variance at each layer, thereby enhancing model training stability and performance.
On the Interplay Between Misspecification and Sub-optimality Gap in Linear Contextual Bandits
Weitong Zhang (University of California), Quanquan Gu (University of California)
OptimizationReinforcement Learning from Human FeedbackImage
🎯 What it does: This paper studies the learning problem of linear contextual multi-armed bandits under model misspecification and proposes the DS-OFUL algorithm based on uncertainty screening of data, along with an improved analysis of SupLinUCB.
On the Occupancy Measure of Non-Markovian Policies in Continuous MDPs
Romain Laroche (Microsoft Research), Remi Tachet des Combes
Reinforcement Learning
🎯 What it does: This paper studies the dominance measure of non-Markovian policies in continuous state spaces and proves that when the dominance is finite or σ-finite, an equivalent Markovian policy can be constructed.
On the Optimality of Misspecified Kernel Ridge Regression
Haobo Zhang (Tsinghua University), Qian Lin (Tsinghua University)
🎯 What it does: This paper studies the problem of incorrectly specified kernel ridge regression (KRR) and proves that KRR is optimal for all 0 < s < 1 in Sobolev RKHS.
On the Power of Foundation Models
Yang Yuan (Tsinghua University)
GenerationData SynthesisPrompt EngineeringMultimodality
🎯 What it does: This paper proposes using a categorical framework to characterize the capabilities of Foundation Models after self-supervised pre-training for downstream tasks, and presents three core theorems that explain the limits and possibilities of prompt tuning and fine-tuning; it also derives a new structural learning generalization theorem that explains the creative performance of multimodal models such as CLIP and DALL-E 2.
On the Power of Pre-training for Generalization in RL: Provable Benefits and Hardness
Haotian Ye (Peking University), Simon Shaolei Du
Reinforcement Learning
🎯 What it does: This study investigates the role of pre-training in the generalization of reinforcement learning, proposing theoretical upper and lower bounds on the performance improvement of pre-training during testing, along with feasible algorithms.
On the Privacy-Robustness-Utility Trilemma in Distributed Learning
Youssef Allouah (Ecole Polytechnique Federale de Lausanne), John Stephan
OptimizationFederated LearningSafty and Privacy
🎯 What it does: This paper studies the trade-off among privacy, robustness, and accuracy in distributed learning, providing both lower and upper bounds for this tripartite relationship. It proves that under the conditions of satisfying differential privacy and adversarial robustness, the learning error cannot be lower than the sum of the three types of errors.
On the Relationship Between Explanation and Prediction: A Causal View
Amir-Hossein Karimi (Max Planck Institute for Intelligent Systems), Been Kim (Google Research)
Explainability and InterpretabilityHyperparameter SearchConvolutional Neural NetworkImage
🎯 What it does: This paper systematically evaluates the relationship between model explainability (E) and prediction (Y) through a causal inference framework, particularly exploring the causal impact of hyperparameters (H) on both, and distinguishing between direct and indirect effects.
On the Robustness of Randomized Ensembles to Adversarial Perturbations
Hassan Dbouk (University of Illinois), Naresh Shanbhag
ClassificationComputational EfficiencyAdversarial AttackConvolutional Neural NetworkImage
🎯 What it does: This paper studies the adversarial robustness of randomized ensemble classifiers (REC), providing theoretical limits, necessary and sufficient conditions, and proposes an augmented algorithm named BARRE for training efficient and robust RECs.
On the Robustness of Text Vectorizers
Rémi Catellier (Université Côte d'Azur), Damien Garreau (Université Côte d'Azur)
OptimizationRepresentation LearningTextOrdinary Differential Equation
🎯 What it does: This paper provides a formal proof of the robustness of text vectorization methods (concatenated vectors, TF-IDF, Doc2Vec) under Hamming distance, and presents the corresponding Lipschitz or Hölder upper bounds.
On the Role of Attention in Prompt-tuning
Samet Oymak (University of Michigan), Christos Thrampoulidis (University of British Columbia)
TransformerPrompt EngineeringImage
🎯 What it does: This paper proposes and theoretically analyzes a single-layer softmax prompt-attention mechanism, studying its expressive power, gradient learning dynamics, and generalization performance.
On the Statistical Benefits of Temporal Difference Learning
David Cheikhi (Columbia University), Daniel Russo (Columbia University)
Reinforcement LearningSequential
🎯 What it does: This paper theoretically compares the statistical efficiency of temporal difference (TD) learning and direct Monte Carlo (MC) estimation in the batch setting of finite state Markov reward processes, quantifying the error advantage of TD in value function and advantage estimation.
On the Stepwise Nature of Self-Supervised Learning
James B Simon, Joshua Albrecht (Generally Intelligent)
Representation LearningConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: This study investigates the training dynamics of joint embedding methods in self-supervised learning, finding that the model gradually learns each dimension of the high-dimensional embedding, exhibiting a discrete stepwise learning pattern.
On the Training Instability of Shuffling SGD with Batch Normalization
David Xing Wu, Suvrit Sra (Massachusetts Institute of Technology)
OptimizationConvolutional Neural NetworkImage
🎯 What it does: This paper studies the training instability of two sampling variants without replacement—Single Shuffle (SS) and Random Reshuffle (RR)—when using Batch Normalization (BN) with Stochastic Gradient Descent (SGD). Theoretical and experimental results show that SS+BN is prone to loss divergence or slow convergence, while RR+BN is more stable. For regression tasks, it is proven that SS and RR converge to different globally optimal 'distorted' risks. For classification tasks, the conditions under which SS+BN may diverge while RR+BN does not are explained through separability decomposition, and this is validated on both synthetic and real data.
On the Within-Group Fairness of Screening Classifiers
Nastaran Okati (Max Planck Institute for Software Systems), Manuel Gomez Rodriguez
Tabular
🎯 What it does: This study examines the unfairness of calibrated screening classifiers within the same group and proposes achieving intra-group monotonicity through post-processing to eliminate this unfairness.
On Uni-Modal Feature Learning in Supervised Multi-Modal Learning
Chenzhuang Du (Tsinghua University), Hang Zhao (Tsinghua University)
ClassificationRecognitionKnowledge DistillationRepresentation LearningVideoMultimodality
🎯 What it does: This paper studies the problem of insufficient unimodal features (Modality Laziness) in multimodal learning and proposes two concise post-fusion training strategies based on Unimodal Teacher (UMT) and Unimodal Ensemble (UME).
On User-Level Private Convex Optimization
Badih Ghazi (Google Research), Chiyuan Zhang (Google Research)
OptimizationSafty and Privacy
🎯 What it does: This paper proposes a new user-level differential privacy mechanism that achieves near-optimal distortion rates for convex optimization (including stochastic convex optimization and empirical risk minimization) without the need for smoothness assumptions.
One Transformer Fits All Distributions in Multi-Modal Diffusion at Scale
Fan Bao (Tsinghua University), Jun Zhu (Tsinghua University)
GenerationData SynthesisTransformerLarge Language ModelDiffusion modelImageTextMultimodality
🎯 What it does: A unified multimodal diffusion framework called UniDiffuser has been developed, capable of fitting image, text, and their joint and conditional distributions within a single Transformer model, supporting various generation tasks.
One-Shot Compression of Large Edge-Exchangeable Graphs using Bits-Back Coding
Daniel Severo (University of Toronto), Alireza Makhzani (Vector Institute for Artificial Intelligence)
CompressionGraph Neural NetworkGraph
🎯 What it does: A one-shot lossless compression method for large edge-exchangeable graphs, called Random Edge Coding (REC), is proposed, achieving optimal compression in conjunction with a non-parametric Pólya's Urn model.
One-Shot Federated Conformal Prediction
Pierre Humbert (Universite Paris-Saclay), Sylvain Arlot (Universite Paris-Saclay)
Federated LearningSafty and PrivacyTabular
🎯 What it does: Proposes a conformal prediction method for constructing effective prediction sets in a one-round communication federated learning environment;
One-shot Imitation in a Non-Stationary Environment via Multi-Modal Skill
Sangwoo Shin (Sungkyunkwan University), Honguk Woo (Sungkyunkwan University)
Robotic IntelligenceMeta LearningVision Language ModelContrastive LearningMultimodality
🎯 What it does: The OnIS framework is proposed to achieve single demonstration learning for complex tasks and adapt to environmental changes.
One-sided Matrix Completion from Two Observations Per Row
Steven Cao (Stanford University), Gregory Valiant (Stanford University)
Tabular
🎯 What it does: This paper studies an algorithm for recovering the right singular vectors from a low-rank matrix with only two observations per row, called one-sided matrix completion.
One-Step Estimator for Permuted Sparse Recovery
Hang Zhang (Amazon), Ping Li (LinkedIn Ads)
RestorationOptimizationImage
🎯 What it does: This paper studies the problem of label-free sparse recovery under multiple measurements, providing theoretical lower bounds on the number of samples and signal-to-noise ratio, and proposes a first-order estimator to achieve permutation recovery and sparse signal recovery.
One-vs-the-Rest Loss to Focus on Important Samples in Adversarial Training
Sekitoshi Kanai (NTT), Yasutoshi Ida (NTT)
Adversarial AttackConvolutional Neural NetworkGenerative Adversarial NetworkImage
🎯 What it does: A new adversarial training loss function SOVR (Switching One‑vs‑the‑Rest Loss) is proposed, which enhances the logit margin of important samples by using One‑vs‑the‑Rest (OVR) loss instead of traditional weighted cross-entropy, thereby improving the model's robustness against Auto-Attack.
Online Learning in Stackelberg Games with an Omniscient Follower
Geng Zhao (University of California), Michael Jordan
Optimization
🎯 What it does: This study investigates online learning in a two-player cooperative Stackelberg game, exploring the sample complexity of the leader and its lower/upper bounds when the follower is omniscient and always best responding; it also provides specific analyses of various structured examples (linear, ReLU, polynomial, etc.).
Online Learning with Feedback Graphs: The True Shape of Regret
Tomáš Kocák (University of Potsdam), Alexandra Carpentier (University of Potsdam)
OptimizationReinforcement Learning from Human FeedbackGraph Neural NetworkReinforcement LearningGraph
🎯 What it does: In the non-random multi-armed bandit problem under graph feedback, a new problem complexity R* is proposed, along with a lower bound and a matching upper bound for the EXP3-EX algorithm, proving that R* is the limit return rate.
Online Local Differential Private Quantile Inference via Self-normalization
Yi Liu (University of Alberta), Linglong Kong (University of Alberta)
Safty and PrivacyTabular
🎯 What it does: A global trustless intermediary, online, and binary query-based local differential privacy (LDP) algorithm is proposed for estimating overall quantiles and constructing confidence intervals.
Online Mechanism Design for Information Acquisition
Federico Cacciamani (Politecnico di Milano), Nicola Gatti (Politecnico di Milano)
Optimization
🎯 What it does: This study investigates the design of a receiver mechanism in the context of information acquisition to incentivize multiple senders to report information truthfully while maximizing the utility of the receiver.
Online Nonstochastic Control with Adversarial and Static Constraints
Xin Liu (ShanghaiTech University), Lei Ying (University of Michigan)
OptimizationTime Series
🎯 What it does: The COCA (Constrained Online Nonstochastic Control) algorithm is proposed to address the adversarial constraints and static constraints present in online non-stochastic control problems, achieving sublinear scheduling errors and constraint violations under known linear systems.
Online Platt Scaling with Calibeating
Chirag Gupta (Carnegie Mellon University), Aaditya Ramdas (Carnegie Mellon University)
ClassificationDomain AdaptationAdversarial AttackTabular
🎯 What it does: An online post-calibration method called Online Platt Scaling (OPS) is proposed, which combines Platt scaling with online logistic regression and further introduces Calibeating technology to achieve adaptive calibration for distribution drift and adversarial sequences.
Online Prototype Alignment for Few-shot Policy Transfer
Qi Yi (University of Science and Technology of China), Yunji Chen (Institute of Software Chinese Academy of Sciences)
Domain AdaptationReinforcement LearningAuto EncoderImage
🎯 What it does: Proposes an Online Prototype Alignment (OPA) framework that achieves cross-domain policy transfer with a small number of samples.
Online Restless Bandits with Unobserved States
Bowen Jiang (Shanghai Jiao Tong University), Chenghu Zhou (Institute of Geographic Sciences and Natural Resources Research)
OptimizationReinforcement Learning
🎯 What it does: A learning framework for online stateless unknown parameter sleeping multi-armed bandits is proposed, along with the TSEETC algorithm based on Thompson Sampling and Episodic Explore-Then-Commit;
Open-VCLIP: Transforming CLIP to an Open-vocabulary Video Model via Interpolated Weight Optimization
Zejia Weng (Fudan University), Yu-Gang Jiang (Fudan University)
ClassificationRecognitionTransformerContrastive LearningVideo
🎯 What it does: Transforming the CLIP model into an open-source vocabulary video classifier Open-VCLIP that can handle videos, utilizing lightweight temporal modeling and weight interpolation to enhance zero-shot action recognition.
Open-Vocabulary Universal Image Segmentation with MaskCLIP
Zheng Ding (University of California San Diego), Zhuowen Tu (University of California San Diego)
Object DetectionSegmentationTransformerVision Language ModelContrastive LearningImageMultimodality
🎯 What it does: The paper presents a model named MaskCLIP, designed for open vocabulary full image segmentation (semantic, instance, and panoptic segmentation), capable of handling categories described by any text during inference.
OpenFE: Automated Feature Generation with Expert-level Performance
Tianping Zhang (Tsinghua University), Jian Li (Tsinghua University)
TabularBenchmark
🎯 What it does: A framework named OpenFE for automatic feature generation is proposed, aiming to replace manual feature engineering with computational methods.
Opponent-Limited Online Search for Imperfect Information Games
Weiming Liu (Tencent AI Lab), Yang Wei
OptimizationComputational EfficiencyReinforcement LearningSequential
🎯 What it does: This paper proposes a new online search method called Opponent-Limited Subgame Solving (OLSS) to address the subgame solving problem in imperfect information games, particularly extending the Safe-1-Knowledge Limited Subgame Solving (Safe-1-KLSS).