These 421 ICML 2023 papers come with a code repository. Each shows an AI one-line summary below β get the verified repo link + the full 6-part summary (innovation, method, data, results, limitations) and search every ICML 2023 paper, free trial on arXivSub.
"Why did the Model Fail?": Attributing Model Performance Changes to Distribution Shifts
Haoran Zhang (Massachusetts Institute of Technology), Shalmali Joshi (Columbia University)
CodeDomain AdaptationExplainability and InterpretabilityBiomedical Data
π― What it does: This paper models the problem of attributing model performance degradation as a cooperative game, using causal graphs to decompose distribution shifts into actionable mechanisms, and allocates overall performance changes to each mechanism through Shapley values.
2D-Shapley: A Framework for Fragmented Data Valuation
Liu Zhihong, Ruoxi Jia (Virginia Tech)
CodeAnomaly DetectionData-Centric LearningTabular
π― What it does: A 2D-Shapley framework is proposed to quantify the contributions of fragmented data (neither sharing features nor samples), along with corresponding fairness axioms and analytical expressions.
π― What it does: This paper studies the use of self-supervised pre-training methods on lightweight Vision Transformers and evaluates their impact on downstream tasks.
A Complete Expressiveness Hierarchy for Subgraph GNNs via Subgraph Weisfeiler-Lehman Tests
Bohang Zhang (Peking University), Liwei Wang (Peking University)
CodeGraph Neural NetworkGraph
π― What it does: This paper systematically studies the expressiveness of subgraph Graph Neural Networks (subgraph GNNs), proposes and fully constructs the Subgraph WeisfeilerβLehman (SWL) hierarchy, reveals the expressive power of different aggregation and pooling designs, and proves that the strongest form is SSWL.
A Connection between One-Step RL and Critic Regularization in Reinforcement Learning
Benjamin Eysenbach (Carnegie Mellon University), Ruslan Salakhutdinov (Carnegie Mellon University)
CodeReinforcement Learning
π― What it does: This paper proves through theoretical derivation and experimental validation that when the regularization strength Ξ»=1, first-order RL and Critic regularization yield the same policy, and explores their performance in offline RL.
π― What it does: A deep learning-based conjugate direction method (DCDM) is proposed to iteratively solve large-scale sparse symmetric positive definite linear systems, focusing on the solution of the discrete Poisson equation.
π― What it does: A hybrid quantum-classical convolution layer based on Hadamard transform (HT-Perceptron) is proposed, which can replace traditional Conv2D to achieve frequency domain convolution.
CodeOptimizationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
π― What it does: This paper addresses the fine-tuning of pre-trained language models for downstream tasks from the perspective of the Neural Tangent Kernel (NTK). It derives the early training kernel (SignGD kernel) suitable for the Adam optimizer and demonstrates that during the fine-tuning process using natural language prompts, model updates often follow kernel behavior. Additionally, it explains the effectiveness of low-rank subspace fine-tuning methods (such as LoRA) from a kernel perspective.
A Neural PDE Solver with Temporal Stencil Modeling
Zhiqing Sun (Carnegie Mellon University), Shinjae Yoo (Brookhaven National Laboratory)
CodeSuper ResolutionOptimizationConvolutional Neural NetworkTime SeriesPhysics Related
π― What it does: A Temporal Stencil Modeling (TSM) method has been developed to achieve learnable flux approximation in time-varying partial differential equations in conservation form through a convolution volume method.
π― What it does: The paper uses information geometry methods to unify the modeling, distance measurement, trajectory re-indexing, and visualization of deep network prediction models trained on different tasks and learning algorithms (supervised, meta-learning, semi-supervised, self-supervised, contrastive learning, etc.), thereby revealing the structure and commonalities of the learnable task space.
A Reinforcement Learning Framework for Dynamic Mediation Analysis
Lin Ge (North Carolina State University), Rui Song (North Carolina State University)
CodeOptimizationReinforcement LearningTime SeriesBiomedical Data
π― What it does: A dynamic mediation analysis framework based on reinforcement learning is proposed, constructing a Mediated Markov Decision Process (MMDP), and decomposing the Average Treatment Effect (ATE) into Immediate Direct Effect (IDE), Immediate Mediation Effect (IME), Delayed Direct Effect (DDE), and Delayed Mediation Effect (DME).
A Robust Test for the Stationarity Assumption in Sequential Decision Making
Jitao Wang (University of Michigan), Zhenke Wu (University of Michigan)
CodeReinforcement LearningTabularTime SeriesSequentialBiomedical DataElectronic Health Records
π― What it does: A dual robust testing method in offline reinforcement learning is proposed to detect whether the stationarity assumption of the Markov Decision Process (MDP) holds.
π― What it does: This paper proposes a three-stage model based on two parameters: 'temperature' and 'load', to predict how hyperparameters during the training phase (such as the number of training epochs and batch size) affect the subsequent network pruning results. The model categorizes the gradient landscapes of the network before and after pruning into three regimes (Regime I, II-A, II-B).
A Toy Model of Universality: Reverse Engineering how Networks Learn Group Operations
Bilal Chughtai, Neel Nanda
CodeTransformerGraph
π― What it does: This study investigates the internal mechanisms of small neural networks when performing finite group composition tasks, discovering that the network implements a general algorithm based on representation theory (GCR), which is validated through reverse engineering and ablation studies.
π― What it does: A unified audio-visual learning framework, OneAVM, is proposed, achieving single-model inference for audio source localization, separation, and recognition tasks.
Accelerated Stochastic Optimization Methods under Quasar-convexity
Qiang Fu (Sun Yat-sen University), Ashia Camage Wilson
CodeOptimizationTabularTime Series
π― What it does: Two accelerated stochastic optimization methods for star-convex functions are proposedβQASGD and QASVRG, which can achieve fast convergence under L-smooth and (strong) star-convex conditions;
Accounting For Informative Sampling When Learning to Forecast Treatment Outcomes Over Time
Toon Vanderschueren (KU Leuven), Mihaela van der Schaar (University of Cambridge)
CodeDrug DiscoveryReinforcement LearningTime SeriesSequentialBiomedical Data
π― What it does: The study investigates how to learn to predict treatment outcomes over time in the presence of informative sampling in observational data.
Actor-Critic Alignment for Offline-to-Online Reinforcement Learning
Zishun Yu (University of Illinois Chicago), Xinhua Zhang (University of Illinois Chicago)
CodeReinforcement LearningTabular
π― What it does: A new offline-to-online reinforcement learning framework is proposed, which aligns the policy and value function obtained from offline learning through an 'actor-critic alignment' step, followed by online fine-tuning.
AdaNPC: Exploring Non-Parametric Classifier for Test-Time Adaptation
YiFan Zhang, Tieniu Tan (Nanjing University)
CodeDomain AdaptationImage
π― What it does: Proposes AdaNPC, a framework that utilizes a KNN non-parametric classifier for online adaptation during testing, storing and gradually updating source domain features and labels in memory.
Adaptive Annealed Importance Sampling with Constant Rate Progress
Shirin Goshtasbpour (ETH Zurich), Fernando Perez-Cruz (ETH Zurich)
CodeTabularOrdinary Differential Equation
π― What it does: Research and improve the intermediate distribution path and scheduling strategy of Annealed Importance Sampling (AIS), proposing a constant rate scheduling and implementing the Constant Rate AIS (CR-AIS) algorithm.
Bin Wu (Zhejiang University), Qiang Zhang (Zhejiang University)
CodeMeta LearningImage
π― What it does: An adaptive combinatorial continuous meta-learning algorithm ACML is proposed for learning shareable meta-knowledge in non-stationary task sequences.
Adaptive Estimation of Graphical Models under Total Positivity
Jiaxi Ying (Hong Kong University of Science and Technology), Daniel P. Palomar (Hong Kong University of Science and Technology)
CodeOptimizationGraph Neural NetworkGraphTime SeriesFinance Related
π― What it does: An adaptive multi-stage estimation method is proposed for estimating the precision matrix of M-matrices or diagonally dominant M-matrices under the MTP2 Gaussian graphical model.
π― What it does: An Adaptive Smoothing Gradient Learning (ASGL) method is proposed for directly training deep Spiking Neural Networks (SNNs), enabling the network to gradually converge to a true spiking network by mixing simulated activations and random spike noise during training.
π― What it does: Proposes the Adaptive Weighted Data Augmentation Consistency Regularization (AdaWAC) algorithm to address the issues of concept drift and the imbalance of sparse/dense label information in medical image segmentation.
Tony Tong Wang, Stuart Russell (University of California Berkeley)
CodeAdversarial AttackReinforcement Learning
π― What it does: Trained adversarial strategies against the top Go AI KataGo, achieving win rates of 99.9% and over 97% with minimal computational resources (only 0.13% of KataGo's training) without using search or using 10^7 search nodes, respectively.
π― What it does: This paper proposes a differentiable graph regularization method and a hierarchical graph convolutional network CoCN based on diagonal convolution, aimed at achieving Euclidean convolution transfer on graph data, capable of simultaneously learning node features and structural features.
π― What it does: A training framework for graph neural networks based on single-layer optimization (ALT-OPT) has been designed and implemented, replacing the traditional end-to-end two-layer optimization with alternating optimization of pseudo-labels and MLP parameters.
An Effective Meaningful Way to Evaluate Survival Models
Shi-ang Qi (University of Alberta), Russell Greiner (University of Alberta)
CodeBiomedical DataElectronic Health Records
π― What it does: A MAE-PO metric based on pseudo-observation is proposed to evaluate survival prediction models with right censoring, and the effectiveness of this metric is assessed through the generation of realistic semi-synthetic datasets.
An Instrumental Variable Approach to Confounded Off-Policy Evaluation
Yang Xu (North Carolina State University), Rui Song (North Carolina State University)
CodeReinforcement LearningVideo
π― What it does: This paper proposes a framework for offline policy evaluation (OPE) using instrumental variables (IV) in the presence of unmeasured confounding in MDP (and POMDP) environments.
π― What it does: A federated learning framework named FedAMD is proposed, which divides some participating clients into anchor point groups and miner groups through anchor point sampling, using large-batch gradients and multi-step small-batch updates to accelerate training and improve model performance.
Answering Complex Logical Queries on Knowledge Graphs via Query Computation Tree Optimization
Yushi Bai (Tsinghua University), Lei Hou (Tsinghua University)
CodeOptimizationExplainability and InterpretabilityGraph Neural NetworkGraph
π― What it does: The QTO (Query Computation Tree Optimization) method is proposed, which utilizes pre-trained knowledge graph embeddings (KGE) to score first-order subqueries and accurately solves the optimal entity allocation on the query computation tree through forward-backward recursive propagation, ensuring both the optimality of the answers and the interpretability of intermediate variables.
π― What it does: A new offline RL method called SAC-RND is developed, which utilizes random network distillation for uncertainty estimation and suppresses exploration in offline RL.
Applied Online Algorithms with Heterogeneous Predictors
Jessica Maghakian (Stony Brook University), Zhenhua Liu
CodeOptimizationReinforcement LearningTime Series
π― What it does: This paper studies the combination of online algorithms and diverse predictors, proposing an algorithm for the rental/purchase problem using parameter prediction and input prediction, and validating it on the task of minimizing bandwidth costs in distributed systems.
π― What it does: This study addresses the problem of non-random missing labels (MNAR) in semi-supervised learning, proposing to estimate the missing mechanism and debias any SSL algorithm through inverse propensity weighting.
π― What it does: This paper proposes the use of random tree decomposition in high-dimensional Bayesian optimization as a replacement for traditional data-based decomposition learning, constructing the Random Decomposition Upper Confidence Bound (RDUCB) algorithm, and proving its theoretical guarantee of optimal error and information gain balance in expectation.
π― What it does: This paper proposes a systematic method based on linear regression to select a representative subset from a large number of Atari games and approximate the median score of the complete ALE benchmark using this subset.
Automatic Intrinsic Reward Shaping for Exploration in Deep Reinforcement Learning
Mingqi Yuan (Hong Kong Polytechnic University), Wenjun Zeng (Eastern Institute for Advanced Study)
CodeReinforcement Learning
π― What it does: Enhancing exploration capabilities in reinforcement learning by automatically selecting and using different intrinsic reward functions.
Automatically marginalized MCMC in probabilistic programming
Jinlin Lai (University of Massachusetts Amherst), Daniel Sheldon (University of Massachusetts Amherst)
CodeOptimizationComputational EfficiencyTabular
π― What it does: This paper proposes an automated method for edge reversal and variable marginalization of directed graphical models generated by probabilistic programming languages, significantly reducing the variable dimension of HMC sampling and improving sampling efficiency.
π― What it does: This paper proposes the Auxiliary Modality Learning (AML) framework, systematically defines and classifies auxiliary modalities and network architectures, and introduces the Smart Auxiliary Modality Distillation (SAMD) method to automatically select the optimal auxiliary modality and enhance the performance of the main modality through curriculum distillation under the 'super model' condition.
B-Learner: Quasi-Oracle Bounds on Heterogeneous Causal Effects Under Hidden Confounding
Miruna Oprescu (Cornell University), Uri Shalit (Technion Israel Institute of Technology)
CodeTabularFinance Related
π― What it does: Proposes the B-Learner method for estimating the upper and lower bounds of conditional average treatment effects (CATE) in the presence of hidden confounding.
Bag of Tricks for Training Data Extraction from Language Models
Weichen Yu (Institute of Automation, Chinese Academy of Sciences), Shuicheng YAN
CodeHyperparameter SearchTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: In response to the issue of training data leakage in language models, this paper systematically evaluates and improves various techniques in the two main stages of training data extraction: generation and ranking, proposing a complete extraction process based on GPT-Neo.
Bayesian Progressive Deep Topic Model with Knowledge Informed Textual Data Coarsening Process
Zhibin Duan (Xidian University), Mingyuan Zhou (University of Texas at Austin)
CodeGraph Neural NetworkText
π― What it does: A knowledge-driven text data coarsening process and the corresponding advanced generative model ProGBN are proposed, which enhance the deep representation and quality of topics in topic models through a coarse-to-fine generation process.
π― What it does: This paper proposes an unsupervised skill discovery method based on behavior contrastive learning (BeCL), which maximizes the mutual information between different trajectory states under the same skill, achieving both diverse skill learning and improved state coverage.
Benign Overfitting in Two-layer ReLU Convolutional Neural Networks
Yiwen Kou (University of California), Quanquan Gu (University of California)
CodeConvolutional Neural Network
π― What it does: This study investigates the benign overfitting phenomenon of two-layer ReLU convolutional neural networks in the presence of label-flipping noise, providing convergence bounds for training error and risk limits for testing error.
π― What it does: By using more advanced diffusion models (EDM) to generate high-quality conditional images, combined with adversarial training (TRADES) to enhance the model's robustness against adversarial attacks.
π― What it does: This paper proposes the Forward-Looking GFlowNet (FL-GFN) method, which utilizes energy/reward information from intermediate states to achieve more efficient credit assignment and supports training using only incomplete trajectories.
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
Junnan Li (Salesforce Research), Steven Hoi (Salesforce Research)
CodeGenerationRetrievalTransformerLarge Language ModelVision Language ModelImageTextMultimodality
π― What it does: BLIP-2 pretrains a vision-language model by inserting a lightweight query Transformer (Q-Former) between a frozen image encoder and a large language model, completing zero-shot generation and retrieval tasks from images to text.
π― What it does: This paper studies how to construct neural networks on Symmetric Positive Definite (SPD) and Grassmann manifolds, proposing to extend gyrovector space theory to these two types of matrix manifolds, and based on this, defines basic operations and layers such as inner product, gyro angle, gyro isometries, and polynomial logistic regression (MLR) on SPD.
π― What it does: This paper proposes a comprehensive attention benchmark (CAB) for long sequence modeling and evaluates nine mainstream efficient attention architectures under four fine-grained attention modes (non-causal self-attention, causal self-attention, non-causal cross-attention, causal cross-attention).
π― What it does: This paper studies the spatial localization of the 'memorization' phenomenon in neural networks, proving that memorization is not concentrated in the last few layers of the model, but rather dispersed across a few neurons in different layers. It proposes an 'Example-Tied Dropout' method that directs memorization towards a pre-specified set of neurons.
π― What it does: Designed and trained a large-scale Graph Hyper-Network (GHN-3) that can quickly predict ImageNet pre-trained parameters and use these parameters to initialize models of various architectures, thereby reducing pre-training costs.
CataBEEM: Integrating Latent Interaction Categories in Node-wise Community Detection Models for Network Data
Yuhua Zhang (University of Michigan), Walter H. Dempsey (University of Michigan)
CodeGraph Neural NetworkGraph
π― What it does: A CataBEEM model under an edge-exchangeable framework is proposed to simultaneously learn latent categories at the interaction level and community structures at the node level, achieving scalable variational EM inference on large-scale interaction networks.
Causal Strategic Classification: A Tale of Two Shifts
Guy Horowitz (Technion Israel Institute of Technology), Nir Rosenfeld (Technion Israel Institute of Technology)
CodeClassificationOptimizationTabularFinance Related
π― What it does: This paper studies the strategic classification problem where users can change features and these changes affect the true labels, and proposes a learning framework and algorithm that incorporates causal relationships.
Chemically Transferable Generative Backmapping of Coarse-Grained Proteins
Soojung Yang (Massachusetts Institute of Technology), Rafael Gomez-Bombarelli
CodeGenerationProtein Structure PredictionAuto EncoderBiomedical Data
π― What it does: A generative post-mapping method has been developed that can quickly recover all atomic-level details from coarse-grained (only containing Ξ± carbon) structures;
π― What it does: Fine-tune the CLIP model with a focus on achieving better generalization capabilities under the OOD setting where domain shift and new classes (open class) coexist in downstream tasks.
James Chenhao Liang, Wenguan Wang (Zhejiang University)
CodeSegmentationTransformerImage
π― What it does: This paper proposes CLUSTSEG, a unified framework based on transformers that can simultaneously perform superpixel, semantic, instance, and panoptic segmentation tasks.
π― What it does: Developed a curriculum learning algorithm CLUTR based on unsupervised task representation learning for unsupervised environment design in reinforcement learning.
CO-BED: Information-Theoretic Contextual Optimization via Bayesian Experimental Design
Desi R. Ivanova (University of Oxford), Adam Foster (Microsoft Research)
CodeOptimizationReinforcement LearningTabular
π― What it does: A general context optimization framework CO-BED is proposed, which utilizes the expected information gain from Bayesian experimental design (CMV-EIG) to guide experimental design, thereby achieving better contextual decisions during the deployment phase.
Competing for Shareable Arms in Multi-Player Multi-Armed Bandits
Renzhe Xu (Tsinghua University), Peng Cui (Tsinghua University)
CodeOptimizationReinforcement Learning
π― What it does: A shared reward model for multi-player multi-armed bandits is designed, and the SMAA algorithm is proposed to implement the learning and competition of self-interested players.
π― What it does: In multi-agent reinforcement learning, this paper proposes a new framework called CAMA to simultaneously address the issues of attention dispersion and information insufficiency caused by dynamic team sizes and partial observability. The framework divides environmental entities into high-attention and low-attention parts through an entity partitioning module, enhancing attention on high-attention entities using inverse models, and using a global coach to generate compressed information for low-attention entities to achieve attention compensation, thereby enabling more robust and efficient collaborative decision-making.
π― What it does: Composer is constructed, a multi-conditional diffusion model that enables controllable and creative image synthesis by combining the decomposed representations of images.
Jiacheng Ye (University of Hong Kong), Lingpeng Kong (University of Hong Kong)
CodeGenerationRetrievalTransformerLarge Language ModelContrastive LearningText
π― What it does: This paper proposes a global example subset selection method called CEIL based on Conditional DPP, which utilizes the output probabilities of large language models for contrastive learning to automatically select the most helpful demonstration example set in In-Context Learning.
π― What it does: The paper explores the computational asymmetry of robust classification from both theoretical and experimental perspectives, proving that attacking ReLU classifiers is NP-hard, while finding robust parameters during the training phase is Ξ£Pβ-hard; it proposes a defense method based on attacks called Counter-Attack and provides its complexity; further experiments demonstrate that heuristic attacks can reliably approach the decision boundary distance, and the UG100 dataset is released.
Conditional Tree Matching for Inference-Time Adaptation of Tree Prediction Models
Harshit Varma (Indian Institute of Technology Bombay), Sunita Sarawagi (Indian Institute of Technology Bombay)
CodeTransformerText
π― What it does: CTREEOT is proposed, a differentiable and convergent conditional tree matching algorithm that adapts tree prediction models using a small number of cases during inference.
CodeGenerationScore-based ModelImageMultimodalityPhysics Related
π― What it does: This paper proposes a Conditional Strong Log-Convex (CSLC) generative model that decomposes high-dimensional data into several conditional log-convex sub-distributions through adaptive orthogonal projection, achieving theoretically guaranteed learning and sampling.
Virginia Aglietti (DeepMind), Silvia Chiappa (DeepMind)
CodeOptimizationGraphTabularElectronic Health Records
π― What it does: This paper proposes a constrained causal Bayesian optimization (cCBO) method under a known causal graph, aimed at finding the optimal intervention set that satisfies multiple constraints to optimize the expected value of the target variable.
π― What it does: This paper proposes the CoTASP method, which dynamically extracts sub-networks from a meta-policy network using sparse prompts to achieve continual task learning, addressing the plasticity-stability trade-off problem.
π― What it does: A Transformer architecture named Continuous Spatiotemporal Transformer (CST) is proposed, specifically designed for learning continuous spatiotemporal dynamical systems, ensuring the model outputs are continuous and smooth, achieving high-quality interpolation and prediction across various tasks.
π― What it does: A continuous parameterized mixed model (CPMM) utilizing neural ODEs is proposed, achieving density estimation and clustering of high-dimensional complex data through a hierarchical structure and curriculum learning.
π― What it does: A 3D representation learning framework called RECON is proposed, which integrates contrastive learning and generative reconstruction.
π― What it does: A non-standard wavelet decomposition (BCR-DE) framework based on the BeylkinβCoifmanβRokhlin (BCR) algorithm is proposed, simplifying long-time series neural controlled differential equations (NCDE) into a single integral transform and constraining the operator to Calderon-Zygmund (CZ) class, achieving efficient training and inference.
π― What it does: This paper studies the injectivity of ReLU layers on a finite radius sphere (and its non-negative part) within the framework of frame theory, and provides a computable upper bound bias estimation method (PBE) along with the corresponding reconstruction formula.
Covariate balancing using the integral probability metric for causal inference
Insung Kong (Seoul National University), Yongdai Kim (Seoul National University)
CodeAdversarial AttackTabular
π― What it does: Two weighting methods (parametric and non-parametric CBIPM) were designed using Integral Probability Metrics (IPM) to achieve covariate balance and estimate average treatment effects.
π― What it does: Analyzed and proved the critical point structure and convergence of deep linear networks when trained with the Bures-Wasserstein loss.
π― What it does: A robust training framework named CrossSplit is proposed, which randomly divides the training set with noisy labels into two parts, trains two networks on the two parts of the data respectively, and uses each other's predictions to soften the labels, combined with semi-supervised learning to reduce the model's memory of incorrect labels.
π― What it does: A DADAO algorithm is proposed for asynchronously, decoupled, and accelerated minimization of the sum of strongly convex smooth functions in distributed networks.
π― What it does: Discusses the feedback loop generated when model outputs are used as future training data and studies its impact on bias amplification.
π― What it does: This paper studies which training samples in contrastive self-supervised learning can most enhance model performance and proposes a subset selection method called SAS based on maximizing expected augmented similarity.
π― What it does: A composite reparameterization and convex data distillation algorithm based on implicit gradients (RCIG) is proposed, which can compress the original dataset into a minimal number of synthetic samples through a single gradient update and train high-quality models on various tasks.
Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time
Zichang Liu (Rice University), Beidi Chen (Carnegie Mellon University)
CodeComputational EfficiencyTransformerLarge Language ModelText
π― What it does: A dynamic sparse structure is proposed for the inference process of large-scale language models (LLMs), which retains only a portion of attention heads and MLP neurons for each input. The DEJAVU system is constructed to achieve real-time sparse prediction and hardware-friendly execution, significantly reducing inference latency.
Sattar Vakili (MediaTek Research), Ciara Pike-Burke (Imperial College London)
CodeOptimizationReinforcement LearningTabular
π― What it does: This paper studies the kernel bandwidth problem with stochastic delayed feedback and proposes an algorithm called BPE-Delay, aimed at optimizing the learning process of kernel functions.
π― What it does: A BenjaminiβHochberg (BH) multiple hypothesis testing framework (BHN) is proposed, which utilizes a small amount of clean data to identify and eliminate noisy labels in neural networks by treating noisy label detection as a multiple hypothesis testing problem and calibrating empirical p-values.
π― What it does: The Denoising MCMC (DMCMC) framework is proposed, which combines MCMC with reverse SDE/ODE solvers to accelerate sampling of score models.
π― What it does: Proposes and uses the Expected Perturbation Score (EPS) as a statistic for detecting adversarial samples, and constructs an EPS-based Maximum Mean Discrepancy (MMD) detection method called EPS-AD;
DevFormer: A Symmetric Transformer for Context-Aware Device Placement
Haeyeon Kim (Korea Advanced Institute of Science and Technology), Jinkyoo Park (Korea Advanced Institute of Science and Technology)
CodeOptimizationTransformerTabular
π― What it does: A Transformer model named DevFormer is proposed for offline context hardware design optimization, particularly for the decap placement problem.
CodeData SynthesisRecommendation SystemOptimizationTabularTime SeriesFinance Related
π― What it does: This paper proposes a framework that combines difference-in-differences and tree models to estimate the conditional average treatment effect (CATT) in the presence of unmeasured confounding in balanced panel data.
Differential Privacy, Linguistic Fairness, and Training Data Influence: Impossibility and Possibility Theorems for Multilingual Language Models
Phillip Rust (University of Copenhagen), Anders SΓΈgaard (University of Copenhagen)
CodeSafty and PrivacyExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningText
π― What it does: The study investigates the compatibility and conflicts between multilingual language models in differential privacy, language fairness, and the impact of training data sparsity.
Differentially Private Optimization on Large Model at Small Cost
Zhiqi Bu (Amazon Web Services), George Karypis (University of Minnesota)
CodeOptimizationSafty and PrivacyComputational EfficiencyTransformerLarge Language ModelImageText
π― What it does: The Book-Keeping (BK) algorithm is proposed, achieving nearly the same time and memory overhead as non-private training through a single round of backpropagation in differential privacy training.
π― What it does: This study investigates a one-shot drawing task, comparing the performance of humans and various generative models (VAE, GAN, diffusion models) in terms of drawing diversity and recognizability, and proposes originality metrics and generalization curves for more granular evaluation.
Yue Liu (National University of Defense Technology), Stan Z. Li (Westlake University)
CodeGraph Neural NetworkContrastive LearningGraph
π― What it does: A scalable deep graph clustering method Dink-Net is proposed, achieving a unified framework for self-supervised node discrimination and neural clustering modules.
Direct Parameterization of Lipschitz-Bounded Deep Networks
Ruigang Wang (University of Sydney), Ian Manchester
CodeOptimizationConvolutional Neural NetworkImage
π― What it does: A new direct parameterization method is proposed, allowing deep neural networks to automatically satisfy the β2 Lipschitz upper bound during training, and enabling learning through standard gradient optimization.
π― What it does: A continuous-time fractional model for diffusion on probability simplices is proposedβthe Dirichlet Diffusion Fractional Model (DDSM), which is applied to discrete data generation, especially for biological sequences (such as human promoters) and constrained data (Sudoku) generation and solving.
Discover and Cure: Concept-aware Mitigation of Spurious Correlation
Shirley Wu (Stanford University), James Zou (Stanford University)
CodeClassificationExplainability and InterpretabilityContrastive LearningImage
π― What it does: Proposes the DISC method, which uses interpretable concepts to identify and eliminate spurious correlations in image classification.
π― What it does: This paper proposes an end-to-end OC-GVFs method that automatically discovers object features from pixels using Slot Attention, and then maps these features to cumulants to learn a Generalized Value Function (GVF), thereby providing a rich and interpretable representation for control policies.
π― What it does: Constructed a decoupled variational autoencoder under the supervision of dynamical system parameters, achieving long-term and robust predictions of system states.