ICML 2023 Papers — Page 7
International Conference on Machine Learning · 1828 papers
Federated Online and Bandit Convex Optimization
Kumar Kshitij Patel (Toyota Technological Institute at Chicago), Nathan Srebro
OptimizationFederated Learning
🎯 What it does: This paper studies the minimization of average regret under adaptive opponents in distributed online and Bandit convex optimization, with a particular focus on the federated learning scenario with limited feedback.
FedHPO-Bench: A Benchmark Suite for Federated Hyperparameter Optimization
Zhen WANG, Yaliang Li (Alibaba Group)
OptimizationFederated LearningHyperparameter SearchConvolutional Neural NetworkGraph Neural NetworkTransformerTabularBenchmark
🎯 What it does: We propose and implement FEDHPO-BENCH, a benchmark suite specifically designed for hyperparameter optimization in federated learning, supporting three evaluation modes: raw, tabular, and surrogate; it also provides various combinations of federated learning tasks, models, and algorithms for unified comparison and extension.
FedVS: Straggler-Resilient and Privacy-Preserving Vertical Federated Learning for Split Models
Songze Li (Hong Kong University of Science and Technology), Jin Liu (Hong Kong University of Science and Technology)
Federated LearningSafty and PrivacyImageTabular
🎯 What it does: Proposes FedVS, a vertical federated learning framework for synchronous segmentation models, addressing the dual challenges of slow clients (stragglers) and privacy leakage.
FeDXL: Provable Federated Learning for Deep X-Risk Optimization
Zhishuai Guo (Texas A and M University), Tianbao Yang (Texas A and M University)
OptimizationFederated LearningConvolutional Neural NetworkImage
🎯 What it does: Two federated learning algorithms, FeDXL1 and FeDXL2, are proposed to solve deep X-risk optimization (including non-traditional objectives such as log loss, AUC, partial AUC, etc.) when data is distributed across multiple machines and cannot be centralized.
Feed Two Birds with One Scone: Exploiting Wild Data for Both Out-of-Distribution Generalization and Detection
Haoyue Bai (University of Wisconsin), Yixuan Li (University of Wisconsin)
Domain AdaptationAnomaly DetectionConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: This study proposes a unified framework SCONE, which utilizes unlabeled wild data (a mixture of ID, covariance shift OOD, and semantic shift OOD) to simultaneously enhance the model's generalization ability on covariance shift OOD and detection performance on semantic shift OOD.
Few-bit Backward: Quantized Gradients of Activation Functions for Memory Footprint Reduction
Georgii Sergeevich Novikov (Skolkovo Institute of Science and Technology), Ivan Oseledets (Skolkovo Institute of Science and Technology)
OptimizationComputational EfficiencyConvolutional Neural NetworkTransformerSupervised Fine-TuningImageTextMultimodality
🎯 What it does: During the training process of neural networks, the author proposes using low-bit quantization of gradients (i.e., approximating the derivative of the activation function as a piecewise constant) to replace the storage of the complete input tensor, thereby significantly reducing memory usage.
Few-Sample Feature Selection via Feature Manifold Learning
David Cohen (Technion Israel Institute of Technology), Ronen Talmon (Technion Israel Institute of Technology)
ClassificationTabular
🎯 What it does: A few-sample supervised feature selection method called ManiFeSt based on feature space manifold learning is proposed.
Fighting Fire with Fire: Contrastive Debiasing without Bias-free Data via Generative Bias-transformation
Yeonsung Jung (Korea Advanced Institute of Science and Technology), Eunho Yang (AITRICS)
ClassificationImage TranslationRepresentation LearningGenerative Adversarial NetworkContrastive LearningImage
🎯 What it does: This paper proposes a contrastive debiasing method called CDvG, which does not require bias labels or unbiased samples. It utilizes a generative bias transformation model to generate different biased views and learns bias-invariant representations through contrastive learning, thereby enhancing the model's generalization ability.
Finding Generalization Measures by Contrasting Signal and Noise
Jiaye Teng (Tsinghua University), Yang Yuan (Tsinghua University)
ClassificationOptimizationConvolutional Neural NetworkImageStochastic Differential Equation
🎯 What it does: A new generalization measurement index called REF Complexity (Relative Fitting degree between signal and noise) is proposed to quantify the fitting difference between the model-algorithm for real signals and random noise, and to evaluate generalization performance without using an additional validation set.
Finding the Missing-half: Graph Complementary Learning for Homophily-prone and Heterophily-prone Graphs
YIZHEN ZHENG, Shirui Pan (Griffith University)
ClassificationOptimizationGraph Neural NetworkGraph
🎯 What it does: This paper proposes Graph Complementary Learning (GOAL), which completes missing intra-class/inter-class edges and uses new graph convolution for node classification after completion.
Finite-Sample Analysis of Learning High-Dimensional Single ReLU Neuron
Jingfeng Wu (Johns Hopkins University), Sham M. Kakade (Harvard University)
OptimizationTabular
🎯 What it does: The paper studies the learning problem of a single ReLU neuron in a high-dimensional over-parameterized environment and provides a finite sample risk analysis of the GLM-tron algorithm.
Fisher Information Embedding for Node and Graph Learning
Dexiong Chen (ETH Zurich), Karsten Borgwardt (ETH Zurich)
Graph Neural NetworkGraph
🎯 What it does: This paper proposes the Fisher Information Embedding (FIE) framework, which maps the multiple neighborhood sets of nodes to a Gaussian mixture model using the Fisher information metric on statistical manifolds. It then obtains learnable embeddings of nodes through EM approximate maximum likelihood estimation, forming a new attention aggregation mechanism.
Flash: Concept Drift Adaptation in Federated Learning
Kunjal Panchal (University of Massachusetts), Hui Guan (University of Massachusetts)
OptimizationFederated LearningText
🎯 What it does: FLASH is proposed, an adaptive optimization algorithm that simultaneously addresses statistical heterogeneity and concept drift in federated learning, combining client early stopping training with server-side drift-aware learning rate adjustment.
FLEX: an Adaptive Exploration Algorithm for Nonlinear Systems
Matthieu Blanke (INRIA), Marc Lelarge (INRIA)
OptimizationRobotic IntelligenceReinforcement LearningTime Series
🎯 What it does: This paper proposes FLEX, an online adaptive exploration algorithm based on D-optimal design, aimed at learning nonlinear dynamic models with high sample efficiency and low computational cost.
FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU
Ying Sheng (Stanford University), Ce Zhang (ETH Zurich)
GenerationOptimizationComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: A high-throughput LLM inference framework called FlexGen has been designed and implemented, capable of performing inference for a 175B scale model on a single GPU through the collaboration and compression of memory, CPU, and disk.
Flexible Phase Dynamics for Bio-Plausible Contrastive Learning
Ezekiel Williams (University of Montreal), Guillaume Lajoie (University of Montreal)
OptimizationRepresentation LearningContrastive LearningImage
🎯 What it does: Proposed a time-local and non-periodic contrastive learning gradient estimation that is proven to be unbiased in both balanced and unbalanced models;
FlexRound: Learnable Rounding based on Element-wise Division for Post-Training Quantization
Jung Hyun Lee (NAVER Cloud), Dongsoo Lee (NAVER Cloud)
CompressionOptimizationTransformerLarge Language ModelImageText
🎯 What it does: In this study, the authors propose a learnable quantization rounding method called FlexRound based on element-wise division, aimed at post-training quantization (PTQ) to enhance the performance of quantized models.
Flipping Coins to Estimate Pseudocounts for Exploration in Reinforcement Learning
Sam Lobel (Brown University), George Konidaris (Brown University)
Reinforcement LearningImageSequential
🎯 What it does: By training a neural network to predict the square of the sample mean of the Rademacher distribution (coin toss), we estimate the pseudocount of states, thereby providing a count-based exploration reward for reinforcement learning agents.
For Pre-Trained Vision Models in Motor Control, Not All Policy Learning Methods are Created Equal
Yingdong Hu (Tsinghua University), Yang Gao (Tsinghua University)
Robotic IntelligenceConvolutional Neural NetworkTransformerReinforcement LearningContrastive LearningImageBenchmark
🎯 What it does: This paper systematically evaluates the transfer effects of 14 pre-trained visual models under three downstream control learning methods (reinforcement learning, behavior cloning, and visual reward functions) through large-scale benchmark experiments, covering a total of 21 robotic manipulation tasks in Meta-World, Robosuite, and Franka-Kitchen.
Forget Unlearning: Towards True Data-Deletion in Machine Learning
Rishav Chourasia (National University of Singapore), Neil Shah (National University of Singapore)
Safty and PrivacyComputational EfficiencyTabular
🎯 What it does: A new data deletion privacy guarantee is proposed, aimed at addressing the shortcomings of data deletion in existing machine learning, ensuring the privacy of deleted records.
Formalizing Preferences Over Runtime Distributions
Devon R. Graham (University of British Columbia), Tim Roughgarden (Columbia University)
OptimizationReinforcement LearningTabular
🎯 What it does: This paper axiomatizes the preference for the distribution of algorithm running times and provides a unified scoring function to compare algorithms under a given timeout distribution; it also proposes modeling unknown timeout distributions using the maximum entropy principle and estimating expected utility from trimmed samples using reversible utility functions.
Forward-Backward Gaussian Variational Inference via JKO in the Bures-Wasserstein Space
Michael Ziyang Diao (Massachusetts Institute of Technology), Adil Salim (Microsoft Research)
OptimizationTabularStochastic Differential Equation
🎯 What it does: The (Stochastic) Forward-Backward Gaussian Variational Inference (FB-GVI) algorithm is proposed to solve Gaussian variational inference problems in the Bures-Wasserstein space.
Fourmer: An Efficient Global Modeling Paradigm for Image Restoration
man zhou, Chongyi Li (Nankai University)
RestorationConvolutional Neural NetworkImage
🎯 What it does: An efficient global modeling image restoration framework named Fourmer is proposed, which utilizes Fourier space to decouple degradation and content and achieves restoration through deep networks.
FP-Diffusion: Improving Score-based Diffusion Models by Enforcing the Underlying Score Fokker-Planck Equation
Chieh-Hsin Lai (Sony Group Corporation), Stefano Ermon (Stanford University)
GenerationData SynthesisDiffusion modelScore-based ModelImage
🎯 What it does: This paper proposes to constrain the score matching model by introducing the 'score Fokker-Planck equation', ensuring that the learned scores satisfy self-consistent continuity features, thereby enhancing the likelihood and conservativeness of the generative model;
Fractional Denoising for 3D Molecular Pre-training
Shikun Feng (Institute for AI Industry Research), Weiying Ma (Institute for AI Industry Research)
Drug DiscoveryGraph Neural NetworkContrastive LearningGraph
🎯 What it does: This paper proposes a mixed noise that combines dihedral angle noise and coordinate noise, using a fractional denoising (Frad) method for self-supervised pre-training of molecular 3D structures;
FREDIS: A Fusion Framework of Refinement and Disambiguation for Unreliable Partial Label Learning
Congyu Qiao (Southeast University), Xin Geng (Southeast University)
ClassificationConvolutional Neural NetworkImage
🎯 What it does: A fusion framework named FREDIS is proposed, which combines refinement and disambiguation strategies to address the problem of Unreliable Partial Label Learning (UPLL);
Free-Form Variational Inference for Gaussian Process State-Space Models
Xuhui Fan (University of Newcastle), Scott A Sisson
Time SeriesBenchmark
🎯 What it does: A Gaussian Process State Space Model (GPSSM) inference method based on free-form variational inference (FFVD) is proposed, which can simultaneously capture the posterior correlation between state trajectories and inducing variables without making parametric assumptions.
From Adaptive Query Release to Machine Unlearning
Enayat Ullah (Johns Hopkins University), Raman Arora (Johns Hopkins University)
Optimization
🎯 What it does: This paper proposes a general machine unlearning framework based on adaptive query publishing and provides efficient unlearning algorithms for linear queries and prefix sum queries. By applying this framework to Stochastic Convex Optimization (SCO) and Generalized Linear Models (GLM), it achieves improved statistical risk upper bounds and dimension-independent rates. A weak unlearning strategy is also proposed for dynamic stream environments.
From Hypergraph Energy Functions to Hypergraph Neural Networks
Yuxin Wang (Fudan University), David Wipf (Amazon)
ClassificationOptimizationExplainability and InterpretabilityGraph Neural NetworkGraph
🎯 What it does: A class of hypergraph learning framework based on energy functions is designed, and trainable node embeddings are obtained through a two-layer optimization, ultimately constructing PhenomNN and its simplified version PhenomNNsimple.
From Noisy Fixed-Point Iterations to Private ADMM for Centralized and Federated Learning
Edwige Cyffers (University of Lille), Debabrota Basu (University of Lille)
OptimizationFederated LearningSafty and Privacy
🎯 What it does: This paper proposes a unified framework for differential privacy optimization algorithms as a noise fixed-point iteration framework, and derives private ADMM algorithms for centralized, federated learning, and decentralized scenarios, providing a theoretical analysis of privacy-utility trade-offs.
From Perception to Programs: Regularize, Overparameterize, and Amortize
Hao Tang (Cornell University), Kevin Ellis (Cornell University)
RecognitionOptimizationReinforcement LearningImage
🎯 What it does: This paper proposes a gradient-based neural symbolic program synthesis method called ROAP, which jointly learns a visual perception network and symbolic program structure to achieve complete self-supervised reasoning from perception to program.
From Relational Pooling to Subgraph GNNs: A Universal Framework for More Expressive Graph Neural Networks
Cai Zhou (Tsinghua University), Muhan Zhang (Peking University)
Graph Neural NetworkGraph
🎯 What it does: Proposes the k,l-WL framework, which runs k-dimensional Weisfeiler–Lehman after explicitly marking l nodes in the graph, unifying Relational Pooling and subgraph GNN, enhancing the expressive power of GNNs.
From Robustness to Privacy and Back
Hilal Asi (Apple), Lydia Zakynthinou (Northeastern University)
Safty and Privacy
🎯 What it does: A black-box method is proposed to convert any robust estimator into a pure differential privacy estimator, and it is proven to achieve optimal error in low-dimensional tasks.
From Temporal to Contemporaneous Iterative Causal Discovery in the Presence of Latent Confounders
Raanan Yehezkel Rohekar, Gal Novik (Intel Labs)
Time Series
🎯 What it does: A constrained causal discovery algorithm for discrete-time structural vector autoregressive processes (SVAR) with potential confounding variables, called TS-ICD, is proposed. It significantly reduces the number of conditional independence tests and enhances the interpretability of the graph by first learning long-term time-lag relationships and then short-term and synchronous relationships.
Fully Bayesian Autoencoders with Latent Sparse Gaussian Processes
Ba-Hien Tran (EURECOM), Maurizio Filippone (University of California)
GenerationData SynthesisAnomaly DetectionAuto EncoderTabularTime SeriesStochastic Differential Equation
🎯 What it does: This paper proposes a fully Bayesian autoencoder (BAE) that employs Bayesian treatment on both the latent space and decoder parameters, introducing a sparse Gaussian process (SPG) prior to model the correlation of latent variables.
Fully Dynamic Submodular Maximization over Matroids
Paul Duetting, Morteza Zadimoghaddam (Google Research)
Optimization
🎯 What it does: This paper presents the first algorithm for fully dynamic monotone submodular maximization under matroid constraints that supports insertions and deletions, providing a 4-approximation solution.
Fully-Adaptive Composition in Differential Privacy
Justin Whitehouse (Carnegie Mellon University), Steven Wu
Safty and Privacy
🎯 What it does: This paper studies fully-adaptive composition in differential privacy, proposing improved versions of the privacy filter and privacy odometer, and proving that the same privacy loss rate as advanced composition can be achieved even when allowing for adaptive selection of privacy parameters.
Function-Space Regularization in Neural Networks: A Probabilistic Perspective
Tim G. J. Rudner (New York University), Andrew Gordon Wilson (New York University)
ClassificationDomain AdaptationConvolutional Neural NetworkImage
🎯 What it does: A function space regularization method based on empirical Bayes (FS-EB) is proposed, achieving dual regularization in parameter space and function space through auxiliary inference to obtain an analyzable empirical prior.
Functional Neural Networks: Shift invariant models for functional data with applications to EEG classification
Florian Heinrichs (SNAP GmbH), Corinna Weber (SNAP GmbH)
ClassificationConvolutional Neural NetworkTime SeriesBiomedical DataElectrocardiogram
🎯 What it does: This paper proposes Functional Neural Networks (FNNs), which integrate concepts of functional data analysis (FDA) such as smoothing and basis function expansion into multilayer perceptrons and convolutional neural networks, achieving translation-invariant classification for functional inputs.
Fundamental Limits of Two-layer Autoencoders, and Achieving Them with Gradient Methods
Aleksandr Shevchenko (Institute of Science and Technology Austria), Marco Mondelli (Institute of Science and Technology Austria)
CompressionOptimizationAuto EncoderImage
🎯 What it does: This study investigates the fundamental limits of two-layer nonlinear autoencoders under proportional scaling (where the input dimension is linearly proportional to the representation dimension), proving that gradient methods can achieve global optimality and revealing the optimal structure (weight coupling, rotation invariance, or water-filling).
Fundamental Tradeoffs in Learning with Prior Information
Anirudha Majumdar (Princeton University)
Reinforcement LearningTabular
🎯 What it does: The concept of 'prioritized risk' is proposed, and its theoretical lower bound is provided, clarifying the fundamental limitations of learning performance when the prior does not completely match the true distribution.
FusionRetro: Molecule Representation Fusion via In-Context Learning for Retrosynthetic Planning
Songtao Liu (Pennsylvania State University), Dinghao Wu (Tencent AI Lab)
Drug DiscoveryGraph Neural NetworkTransformerGraphBenchmark
🎯 What it does: A backward synthesis planning framework called FusionRetro is proposed, which integrates chemical reaction pathway information using 'context learning', and a RetroBench benchmark is constructed for evaluation on the full USPTO dataset.
Future-conditioned Unsupervised Pretraining for Decision Transformer
Zhihui Xie (Shanghai Jiao Tong University), Shuai Li (Shanghai Jiao Tong University)
Robotic IntelligenceTransformerLarge Language ModelReinforcement LearningTabular
🎯 What it does: A Pretrained Decision Transformer (PDT) model is constructed, utilizing future trajectory information for Transformer pre-training on reward-free offline data, followed by fine-tuning on reward-based tasks through a return prediction network, achieving efficient unsupervised pre-training and rapid adaptation to downstream tasks.
GAT: Guided Adversarial Training with Pareto-optimal Auxiliary Tasks
Salah GHAMIZI, YVES LE TRAON
ClassificationSegmentationAdversarial AttackConvolutional Neural NetworkGenerative Adversarial NetworkImageComputed Tomography
🎯 What it does: A Guided Adversarial Training (GAT) method is proposed to enhance the model's adversarial robustness by incorporating self-supervised or domain knowledge auxiliary tasks on a small amount of training data, and introducing gradient curvature regularization and multi-objective weight scheduling in adversarial training.
Gaussian Process Priors for Systems of Linear Partial Differential Equations with Constant Coefficients
Marc Harkonen (Max Planck Institute for Mathematics in the Sciences), Bogdan Raita (Georgetown University)
Gaussian SplattingPhysics RelatedOrdinary Differential Equation
🎯 What it does: This paper proposes a Gaussian process prior (EPGP) constructed using the Ehrenpreis-Palamodov fundamental principle, which can generate samples that satisfy PDE constraints for any system of linear constant coefficient partial differential equations, and presents a sparse version (S-EPGP);
Gaussian processes at the Helm(holtz): A more fluid model for ocean currents
Renato Berlinghieri (Massachusetts Institute of Technology), Tamara Broderick (Massachusetts Institute of Technology)
Time SeriesPhysics Related
🎯 What it does: Using Gaussian Process (GP) for spatial inference of current velocity obtained from buoy observations, focusing on constructing a GP prior based on Helmholtz decomposition to simultaneously recover the current field and its physical quantities such as divergence and vorticity;
GC-Flow: A Graph-Based Flow Network for Effective Clustering
Tianchun Wang (Pennsylvania State University), Jie Chen (MIT IBM Watson AI Lab)
ClassificationGraph Neural NetworkFlow-based ModelGraph
🎯 What it does: A GC-Flow model is proposed that combines graph convolutional networks with reversible normalizing flows, directly modeling the conditional distribution of nodes using a generative approach while maintaining classification performance and improving clustering quality.
GEAR: A GPU-Centric Experience Replay System for Large Reinforcement Learning Models
Hanjing Wang (Shanghai Jiao Tong University), Luo Mai (University of Edinburgh)
Reinforcement LearningSequential
🎯 What it does: A GPU-centered distributed experience replay system, GEAR, has been developed for training large-scale reinforcement learning models.
GeCoNeRF: Few-shot Neural Radiance Fields via Geometric Consistency
Min-Seop Kwak (Korea University), Seungryong Kim (Korea University)
GenerationData SynthesisDepth EstimationConvolutional Neural NetworkNeural Radiance FieldImage
🎯 What it does: By introducing depth-guided image distortion and feature-level consistency loss under a limited number of viewpoints, GeCoNeRF is proposed to enhance the quality of neural radiance fields with sparse inputs.
General Covariance Data Augmentation for Neural PDE Solvers
Fanaskov Vladimir, Ivan Oseledets (Artificial Intelligence Research Institute)
Convolutional Neural NetworkRecurrent Neural NetworkTime SeriesPhysics Related
🎯 What it does: This paper proposes a data augmentation method based on the general covariance principle to generate training samples for neural networks solving PDEs, reducing reliance on expensive traditional PDE solvers.
General Sequential Episodic Memory Model
Arjun Karuvally (University of Massachusetts Amherst), Hava T Siegelmann
Sequential
🎯 What it does: Proposes the General Sequential Episodic Memory Model (GSEMM), which introduces delayed coupling and multiple time scales based on the Hopfield network to achieve a dynamic energy landscape that can store and retrieve memory sequences.
Generalization Analysis for Contrastive Representation Learning
Yunwen Lei (University of Hong Kong), Ding-Xuan Zhou (University of Sydney)
Representation LearningContrastive Learning
🎯 What it does: A new generalization error upper bound for the unsupervised pre-training task of contrastive learning is provided, significantly reducing the dependence on the number of negative samples k.
Generalization Bounds using Data-Dependent Fractal Dimensions
Benjamin Dupuis (Inria), Umut Simsekli (Inria)
ImageTabular
🎯 What it does: A neural network generalization error upper bound is proposed without the need for Lipschitz continuity assumptions, using data-related fractal dimensions and information theory methods.
Generalization on the Unseen, Logic Reasoning and Degree Curriculum
Emmanuel Abbe (École Polytechnique Fédérale de Lausanne), Kevin Rizk (Apple)
TransformerTabular
🎯 What it does: This paper introduces the concept of 'Generalization to Unseen Domains' (GOTU) and studies the reasoning ability of neural networks on unseen samples when part of the data distribution is completely ignored. Theoretical and experimental analyses are conducted on architectures such as random feature models, diagonal linear networks, and Transformers, revealing that they tend to learn minimum degree interpolators (MD interpolators) in unseen domains. A degree-based curriculum learning algorithm, Degree-Curriculum, is proposed to accelerate learning and improve length generalization.
Generalized Disparate Impact for Configurable Fairness Solutions in ML
Luca Giuliani (University of Bologna), Michele Lombardi (University of Bologna)
OptimizationExplainability and InterpretabilityTabular
🎯 What it does: A configurable family of GeDI metrics is proposed to measure the functional dependency between continuously protected attributes and model outputs, and fairness constraints are implemented through an optimization framework.
Generalized Implicit Follow-The-Regularized-Leader
Keyi Chen (Boston University), Francesco Orabona (Boston University)
OptimizationTabular
🎯 What it does: A general implicit FTRL framework is proposed, which can unify and improve the traditional FTRL's linearization and full loss updates, providing new directly optimizable update rules.
Generalized Polyak Step Size for First Order Optimization with Momentum
Xiaoyu Wang (Hong Kong University of Science and Technology), Tong Zhang (Hong Kong University of Science and Technology)
OptimizationImageTabular
🎯 What it does: An adaptive Polyak step size framework is proposed, specifically designed for first-order optimization algorithms with momentum (such as Heavy-Ball, MAG, Nesterov, etc.).
Generalized Reductions: Making any Hierarchical Clustering Fair and Balanced with Low Cost
Marina Knittel (University of Maryland), MohammadTaghi Hajiaghayi
Tabular
🎯 What it does: This paper proposes a set of tree operation methods that transform any hierarchical clustering structure into a fair, balanced, and low-cost hierarchical clustering.
Generalized Teacher Forcing for Learning Chaotic Dynamics
Florian Hess (Heidelberg University), Daniel Durstewitz (Heidelberg University)
Recurrent Neural NetworkTime SeriesBiomedical DataElectrocardiogram
🎯 What it does: This paper proposes a Generalized Teacher Forcing (GTF) technique for training recurrent neural networks to avoid gradient explosion when reconstructing chaotic dynamical systems, and achieves a low-dimensional interpretable model through a simplified shPLRNN structure.
Generalized-Smooth Nonconvex Optimization is As Efficient As Smooth Nonconvex Optimization
Ziyi Chen (University of Utah), Zhaosong Lu (University of Minnesota)
OptimizationTabularOrdinary Differential Equation
🎯 What it does: This paper proposes the class of α-symmetric generalized smooth functions L*_{sym}(α) and designs optimization algorithms on this class: deterministic β-normalized gradient descent (β-GD) and stochastic SPIDER, proving that they achieve optimal iteration/sample complexity in non-convex optimization.
Generalizing Neural Wave Functions
Nicholas Gao (Technical University of Munich), Stephan Günnemann
Graph Neural NetworkGraphPhysics Related
🎯 What it does: Proposes two networks, Globe and Moon, to solve the Schrödinger equation for various different molecules in a single pass.
Generated Graph Detection
Yihan Ma (CISPA Helmholtz Center for Information Security), Yang Zhang (CISPA Helmholtz Center for Information Security)
ClassificationAnomaly DetectionGraph Neural NetworkContrastive LearningGraph
🎯 What it does: A generative graph detection framework is proposed, defining four training/testing scenarios, and conducting binary classification experiments on generated graphs and real graphs using three models (end-to-end GCN, contrastive learning, metric learning).
Generating Language Corrections for Teaching Physical Control Tasks
Megha Srivastava (Stanford University), Dorsa Sadigh (Stanford University)
GenerationRobotic IntelligenceTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This study proposes a model (CORGI) that can automatically generate natural language corrective feedback for physics control tasks based on the comparison of student and expert trajectories.
Generating Novel, Designable, and Diverse Protein Structures by Equivariantly Diffusing Oriented Residue Clouds
Yeqing Lin (Columbia University), Mohammed AlQuraishi (Columbia University)
Protein Structure PredictionDiffusion modelPoint Cloud
🎯 What it does: This study presents Genie, a protein structure generator based on a diffusion probabilistic model, capable of generating designable, innovative, and diverse protein backbones.
Generating Private Synthetic Data with Genetic Algorithms
Terrance Liu (Carnegie Mellon University), Steven Wu
Data SynthesisOptimizationSafty and PrivacyTabular
🎯 What it does: This study proposes a new algorithm called PRIVATE-GSD for generating differentially private synthetic data, aimed at approximating the statistical properties of the underlying sensitive data.
Generative Adversarial Symmetry Discovery
Jianke Yang (University of California San Diego), Rose Yu (University of California San Diego)
GenerationData SynthesisGraph Neural NetworkGenerative Adversarial NetworkTabularTime SeriesPhysics Related
🎯 What it does: This paper presents LieGAN, a framework based on Generative Adversarial Networks that can automatically discover the symmetries of continuous Lie groups and their discrete subgroups from data and generate corresponding Lie algebra bases.
Generative Causal Representation Learning for Out-of-Distribution Motion Forecasting
Shayan Shirahmad Gale Bagi (University of Waterloo), Mark Crowley (University of Waterloo)
GenerationDomain AdaptationRepresentation LearningAuto EncoderGenerative Adversarial NetworkMultimodalityTime Series
🎯 What it does: A Generative Causal Representation Learning (GCRL) framework is proposed for human trajectory prediction in cross-domain and noisy environments.
Generative Decoding of Visual Stimuli
Eleni Miliotou (University of California Los Angeles), Paul Bogdan (University of Southern California)
RecognitionGenerationData SynthesisAuto EncoderImageMultimodalityMagnetic Resonance Imaging
🎯 What it does: This study investigates how to reconstruct real images from fMRI data and proposes a decoding architecture based on Hierarchical Variational Autoencoders (HVAE).
Generative Graph Dictionary Learning
Zhichen Zeng (University of Illinois at Urbana-Champaign), Hanghang Tong (University of Illinois at Urbana-Champaign)
ClassificationOptimizationRepresentation LearningGraph Neural NetworkGraph
🎯 What it does: A framework based on Generative Graph Dictionary Learning (FRAME) is proposed, utilizing Fused Gromov-Wasserstein distance and RBF kernel to achieve multi-layer (graph, subgraph, node) nonlinear embedding, which can be trained in either supervised or unsupervised mode.
Generative Pretraining for Black-Box Optimization
Satvik Mehul Mashkaria (University of California), Aditya Grover (University of California)
OptimizationTransformerReinforcement LearningTabular
🎯 What it does: A framework for offline black-box optimization called BONET is proposed, which uses a self-supervised generative model to learn the dynamics of the optimizer.
Geometric Autoencoders - What You See is What You Decode
Philipp Nazari (Heidelberg University), Fred A Hamprecht
Explainability and InterpretabilityRepresentation LearningAuto EncoderImagePoint Cloud
🎯 What it does: This paper proposes a Geometric Autoencoder, which constrains the geometric properties of the decoder to make low-dimensional visual embeddings more interpretable and less susceptible to misleading non-uniform stretching by the decoder.
Geometric Clifford Algebra Networks
David Ruhe (Microsoft Research), Johannes Brandstetter (Microsoft Research)
Convolutional Neural NetworkGraph Neural NetworkTime SeriesPhysics Related
🎯 What it does: This paper proposes Geometric Clifford Algebra Networks (GCANs), which construct adjustable geometric templates for modeling dynamic systems through group action layers based on geometric Clifford algebra.
Geometric Latent Diffusion Models for 3D Molecule Generation
Minkai Xu (Stanford University), Jure Leskovec (Stanford University)
GenerationData SynthesisDrug DiscoveryGraph Neural NetworkDiffusion modelAuto EncoderGraph
🎯 What it does: A geometric latent diffusion model (GEOLDM) has been developed to generate three-dimensional molecular structures from scratch.
GFlowNet-EM for Learning Compositional Latent Variable Models
Edward J Hu, Yoshua Bengio (Mila)
TransformerFlow-based ModelText
🎯 What it does: This paper proposes GFlowNet-EM, which uses GFlowNets as a posterior sampler for the E step of EM, supporting scalable learning for discrete combinatorial latent variable models (such as non-context-free grammars and discrete VAEs).
GFlowOut: Dropout with Generative Flow Networks
Dianbo Liu (Mila Quebec AI Institute), Yoshua Bengio (Mila Quebec AI Institute)
ClassificationOptimizationConvolutional Neural NetworkTransformerFlow-based ModelImage
🎯 What it does: This paper studies a Dropout model called GFlowOut, which utilizes Generative Flow Networks (GFlowNet) to learn the posterior distribution of discrete Dropout masks, thereby enhancing Bayesian approximation and uncertainty estimation.
GibbsDDRM: A Partially Collapsed Gibbs Sampler for Solving Blind Inverse Problems with Denoising Diffusion Restoration
Naoki Murata (Sony AI), Stefano Ermon (Stanford University)
RestorationDiffusion modelImageAudio
🎯 What it does: Using a pre-trained diffusion model as a data prior, GibbsDDRM is proposed to solve the blind linear inverse problem through a partially collapsed Gibbs sampler, which simultaneously estimates unknown data and unknown linear operator parameters.
Gibbsian Polar Slice Sampling
Philip Schär, Daniel Rudolf (University of Passau)
OptimizationComputational Efficiency
🎯 What it does: A new Gibbsian polar coordinate sampling method (GPSS) is proposed, which achieves efficient sampling by updating the direction and radius of the polar coordinates in a Gibbs manner.
Git-Theta: A Git Extension for Collaborative Development of Machine Learning Models
Nikhil Kandpal (University of North Carolina), Colin Raffel (University of North Carolina)
Large Language ModelSupervised Fine-TuningText
🎯 What it does: Introducing Git-Theta - an extension based on Git for versioning and collaborative management of machine learning model parameters;
Global Context Vision Transformers
Ali Hatamizadeh (NVIDIA), Pavlo Molchanov (NVIDIA)
Object DetectionSegmentationTransformerImage
🎯 What it does: Proposes the GC ViT architecture, which integrates local window self-attention with global query tokens, combined with a convolutional token generator and an improved downsampling module, achieving efficient context modeling and parameter utilization.
Global optimality for Euclidean CCCP under Riemannian convexity
Melanie Weber (Harvard University), Suvrit Sra (MIT)
OptimizationTabular
🎯 What it does: Proposes a geographic convex (g-convex) optimization problem that can be expressed in the difference of convex (DC) form under Riemannian geometry, and presents a CCCP algorithm that only requires Euclidean gradients;
Global optimality of Elman-type RNNs in the mean-field regime
Andrea Agazzi (University of Pisa), Sayan Mukherjee (Duke University)
OptimizationRecurrent Neural NetworkSequentialOrdinary Differential Equation
🎯 What it does: Analyzed Elman-type recurrent neural networks (RNNs) and their training under mean-field conditions, demonstrating that the gradient descent training dynamics of RNNs converge to the corresponding mean-field equations in the width limit, and proving that under certain weight initialization assumptions, the fixed point of the infinite width limit dynamics is globally optimal.
Global Optimization with Parametric Function Approximation
Chong Liu (University of California), Yu-Xiang Wang (University of California)
OptimizationHyperparameter SearchTabular
🎯 What it does: A global optimization algorithm GO-UCB based on parameter function approximation is proposed to solve high-dimensional noisy zero-order global optimization problems.
Global Selection of Contrastive Batches via Optimization on Sample Permutations
Vin Sachidananda (Stanford University), Chenguang Zhu (Microsoft Research)
OptimizationRepresentation LearningContrastive LearningImageText
🎯 What it does: Proposes Global Contrastive Batch Sampling (GCBS), which optimizes sample arrangement to approximate global contrastive loss, avoiding the high cost of hard negative mining.
GLOBE-CE: A Translation Based Approach for Global Counterfactual Explanations
Dan Ley (Harvard University), Daniele Magazzeni (J.P. Morgan AI Research)
Explainability and InterpretabilityComputational EfficiencyTabular
🎯 What it does: The GLOBE-CE framework is proposed to achieve Global Counterfactual Explanations (GCE) by summarizing model decision boundaries through variable amplitude translation vectors, providing interpretable global explanations.
GNN&GBDT-Guided Fast Optimizing Framework for Large-scale Integer Programming
Huigen Ye (Tsinghua University), Yu Jiang (Meituan)
OptimizationGraph Neural NetworkGraph
🎯 What it does: A fast optimization framework based on multi-task GNN and GBDT is proposed to solve large-scale integer programming problems using small-scale optimizers.
GNOT: A General Neural Operator Transformer for Operator Learning
Zhongkai Hao (Tsinghua University), Jun Zhu (Tsinghua University)
TransformerMixture of Experts
🎯 What it does: A general neural operator framework GNOT based on Transformer is proposed for learning the solution operator of PDEs, capable of handling irregular grids, multiple input functions, and multi-scale problems.
Go Beyond Imagination: Maximizing Episodic Reachability with World Models
Yao Fu (University of Michigan), Honglak Lee (University of Michigan)
Robotic IntelligenceReinforcement LearningWorld ModelSequential
🎯 What it does: Proposes GoBI, which combines world models and short-term memory to generate reachable states as intrinsic rewards.
GOAT: A Global Transformer on Large-scale Graphs
Kezhi Kong (University of Maryland), Tom Goldstein (University of Maryland)
ClassificationComputational EfficiencyGraph Neural NetworkTransformerGraph
🎯 What it does: A scalable global Transformer (GOAT) is proposed for large-scale graph node classification tasks, capable of handling both homogeneous and heterogeneous graphs simultaneously.
Gradient Descent Converges Linearly for Logistic Regression on Separable Data
Kyriakos Axiotis (Google Research), Maxim Sviridenko (Yahoo Research)
OptimizationTabular
🎯 What it does: This paper studies the linear convergence of gradient descent with variable learning rates in logistic regression on linearly separable data and extends it to sparse logistic regression.
Gradient Descent Finds the Global Optima of Two-Layer Physics-Informed Neural Networks
Yihang Gao (University of Hong Kong), Michael Ng
OptimizationTabularPhysics RelatedOrdinary Differential Equation
🎯 What it does: This paper studies the convergence properties of two-layer Physics-Informed Neural Networks (PINNs) under gradient descent and gradient flow, proving that under over-parameterization conditions, gradient descent can find the global optimal solution for linear second-order partial differential equations (PDEs).
Gradient Descent in Neural Networks as Sequential Learning in Reproducing Kernel Banach Space
Alistair Shilton (Deakin University), Svetha Venkatesh (Deakin University)
Optimization
🎯 What it does: This paper proposes a method to accurately represent the behavior of neural networks using power series, and proves that gradient descent is equivalent to regularized learning in the reproducing kernel Banach space (RKBS), providing an upper bound on Rademacher complexity.
Gradient Descent Monotonically Decreases the Sharpness of Gradient Flow Solutions in Scalar Networks and Beyond
Itai Kreisler (Tel Aviv University), Yair Carmon (Tel Aviv University)
OptimizationConvolutional Neural NetworkImageOrdinary Differential Equation
🎯 What it does: The study found that the sharpness of Gradient Flow Solution (GFS) decreases monotonically during the training of neural networks using Gradient Descent (GD), and it was proven that this property holds in scalar networks; further, this property was used to explain the Edge of Stability (EoS) phenomenon and the non-monotonic convergence of loss.
Gradient-based Wang--Landau Algorithm: A Novel Sampler for Output Distribution of Neural Networks over the Input Space
Weitang Liu (University of California), Jingbo Shang (Los Alamos National Laboratory)
Anomaly DetectionOptimizationConvolutional Neural NetworkImage
🎯 What it does: The researchers proposed a gradient-driven Wang-Landau sampler to sample the output distribution of neural networks across the entire input space, thereby obtaining a complete output histogram and corresponding representative input samples.
Gradient-Free Structured Pruning with Unlabeled Data
Azade Nova (Google DeepMind), Dale Schuurmans (University of Alberta)
CompressionOptimizationTransformerText
🎯 What it does: This paper proposes a gradient-independent structured pruning framework KCM, which compresses Transformer models using unlabeled data without retraining;
GRAFENNE: Learning on Graphs with Heterogeneous and Dynamic Feature Sets
Shubham Gupta (Indian Institute of Technology Delhi), Srikanta J. Bedathur
ClassificationGraph Neural NetworkGraph
🎯 What it does: Proposes the GRAFENNE framework to address the learning and continual learning issues of graph neural networks under heterogeneous node features and dynamic changes.
Graph Contrastive Backdoor Attacks
Hangfan Zhang (Pennsylvania State University), Dinghao Wu (Pennsylvania State University)
OptimizationAdversarial AttackGraph Neural NetworkContrastive LearningGraph
🎯 What it does: This paper studies backdoor attacks on Graph Contrastive Learning (GCL) and proposes the GCBA framework, which includes three attack scenarios: data, model, and natural backdoors.
Graph Generative Model for Benchmarking Graph Neural Networks
Minji Yoon (Carnegie Mellon University), Russ Salakhutdinov (Carnegie Mellon University)
Data SynthesisSafty and PrivacyGraph Neural NetworkTransformerGraphBenchmark
🎯 What it does: A novel graph generation model called Computation Graph Transformer (CGT) is proposed, which can generate synthetic graphs suitable for graph neural network (GNN) evaluation while meeting the three major requirements of scale, effectiveness, and privacy.
Graph Inductive Biases in Transformers without Message Passing
Liheng Ma (McGill University), Ser-Nam Lim (MetaAI)
Graph Neural NetworkTransformerGraph
🎯 What it does: This paper proposes a graph Transformer model called GRIT that does not use message passing, utilizing random walk relative position encoding, an updatable attention mechanism, and degree information injection to achieve adaptive learning of graph structures.
Graph Ladling: Shockingly Simple Parallel GNN Training without Intermediate Communication
AJAY KUMAR JAISWAL, Zhangyang Wang (University of Texas at Austin)
Graph Neural NetworkGraphBenchmark
🎯 What it does: A GNN training method based on model soup is proposed, which splits large graphs into subgraphs and independently trains multiple weak models, then merges them using a greedy interpolation method to improve performance without deepening or widening the model.
Graph Mixup with Soft Alignments
Hongyi Ling (Texas A&M University), Na Zou (Texas A&M University)
ClassificationData SynthesisGraph Neural NetworkGraph
🎯 What it does: Proposes S-Mixup, which achieves Mixup of graph data through soft alignment of nodes to generate high-quality augmented samples.