NeurIPS 2023 Papers — Page 13
Conference on Neural Information Processing Systems · 3218 papers
Fragment-based Pretraining and Finetuning on Molecular Graphs
Kha-Dinh Luong (University of California), Ambuj Singh (University of California)
Representation LearningDrug DiscoveryGraph Neural NetworkContrastive LearningGraph
🎯 What it does: By performing contrastive learning and predictive pre-training between molecular graphs and fragment graphs, the representation ability of GNN for molecular structures is enhanced.
Free-Bloom: Zero-Shot Text-to-Video Generator with LLM Director and LDM Animator
Hanzhuo Huang (ShanghaiTech University), Sibei Yang (ShanghaiTech University)
GenerationTransformerLarge Language ModelDiffusion modelVideoText
🎯 What it does: A zero-shot, no-training text-to-video generation process is proposed, utilizing a large language model (LLM) to generate temporal sequence prompts, and then employing a pre-trained latent diffusion model (LDM) to generate semantically coherent, temporally consistent, and high-quality videos using techniques such as joint noise sampling, step-aware attention transfer, and dual-path interpolation.
FreeMask: Synthetic Images with Dense Annotations Make Stronger Segmentation Models
Lihe Yang (University of Hong Kong), Hengshuang Zhao (University of Hong Kong)
SegmentationGenerationData SynthesisTransformerGenerative Adversarial NetworkImage
🎯 What it does: This paper utilizes generative models to synthesize a large number of high-quality synthetic images under semantic mask conditions, and significantly improves fully supervised semantic segmentation performance when combined with real data or pre-trained models.
Frequency Domain-Based Dataset Distillation
DongHyeok Shin, Il-chul Moon
Data SynthesisKnowledge DistillationImagePoint Cloud
🎯 What it does: Utilize frequency domain transformations (such as DCT) to parameterize data, select a small number of high-variance dimensions in the frequency domain, and optimize synthetic data, thereby achieving dataset distillation under a limited budget.
Frequency-domain MLPs are More Effective Learners in Time Series Forecasting
Kun Yi (Beijing Institute of Technology), Zhendong Niu (USTC)
Time Series
🎯 What it does: Proposes FreTS, which learns time series forecasting using MLP in the frequency domain.
Frequency-Enhanced Data Augmentation for Vision-and-Language Navigation
Keji He (Chinese Academy of Sciences), Xinchao Wang (National University of Singapore)
Data SynthesisRetrievalVision Language ModelImageText
🎯 What it does: This paper proposes a frequency domain-based data augmentation method (FDA) that enhances the visual-text matching and navigation performance of visual-language navigation (VLN) models by mixing high-frequency components in the Fourier domain.
From Cloze to Comprehension: Retrofitting Pre-trained Masked Language Models to Pre-trained Machine Reader
Weiwen Xu (Chinese University of Hong Kong), Lidong Bing (Alibaba Group)
ClassificationRetrievalTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes a model called PMR (Pre-trained Machine Reader), which transforms a pre-trained masked language model (MLM) into a machine reading comprehension (MRC) model. It significantly improves the performance of span extraction tasks through continuous pre-training using large-scale MRC-style training data automatically generated from Wikipedia hyperlinks.
From Discrete Tokens to High-Fidelity Audio Using Multi-Band Diffusion
Robin San Roman (Meta), Alexandre Défossez
GenerationData SynthesisCompressionDiffusion modelMultimodalityAudio
🎯 What it does: This paper studies a high-fidelity audio decoder based on multi-band diffusion, which can restore audio symbols generated by low-bitrate discrete encoders into high-quality audio waveforms.
From Pixels to UI Actions: Learning to Follow Instructions via Graphical User Interfaces
Peter Shaw (Google DeepMind), Kristina Toutanova (Google DeepMind)
Robotic IntelligenceTransformerReinforcement LearningAgentic AIImageBenchmark
🎯 What it does: Trained an agent that uses only pixel screenshot input and a general mouse and keyboard action space, capable of performing tasks in GUI instruction-following tasks like MiniWob++ and surpassing human performance.
From Tempered to Benign Overfitting in ReLU Neural Networks
Guy Kornowski (Weizmann Institute of Science), Ohad Shamir (Weizmann Institute of Science)
Recurrent Neural Network
🎯 What it does: This paper analyzes the generalization behavior of a 2-layer ReLU neural network when interpolating noisy training data, theoretically proving the occurrence of mild overfitting in one dimension and benign overfitting in higher dimensions, and experimentally validating the smooth transition between the two.
From Trainable Negative Depth to Edge Heterophily in Graphs
Yuchen Yan (University of Illinois at Urbana Champaign), Hanghang Tong (University of Illinois at Urbana Champaign)
ClassificationGraph Neural NetworkGraph
🎯 What it does: A trainable GCN depth parameter is proposed, extending the depth to the real number domain, with negative depth automatically capturing the heterogeneity of the graph.
From ViT Features to Training-free Video Object Segmentation via Streaming-data Mixture Models
Roy Uziel (Ben Gurion University of the Negev), Oren Freifeld (Ben Gurion University of the Negev)
Object DetectionSegmentationTransformerVideo
🎯 What it does: A novel unsupervised, non-training, low memory consumption semi-supervised video object segmentation method is proposed, utilizing pre-trained ViT features and a multi-scale vMF mixture model that can be updated on streaming data for object modeling, refined through pixel-level assignment and pixel-adaptive CRF.
Front-door Adjustment Beyond Markov Equivalence with Limited Graph Knowledge
Abhin Shah (Massachusetts Institute of Technology), Murat Kocaoglu (Purdue University)
TabularFinance Related
🎯 What it does: This paper proposes a method to estimate causal effects using known treatment variable subnode information for front-door adjustment without requiring a complete causal graph.
Full-Atom Protein Pocket Design via Iterative Refinement
ZAIXI ZHANG, Qi Liu (University of Science and Technology of China)
Drug DiscoveryProtein Structure PredictionGraph Neural NetworkTransformerGraph
🎯 What it does: A framework called FAIR has been designed for the joint design of sequences and three-dimensional structures of all-atom protein pockets (binding sites), achieved through a two-stage iterative full-shot refinement.
Fully Dynamic $k$-Clustering in $\tilde O(k)$ Update Time
Sayan Bhattacharya (University of Warwick), Nikos Parotsidis (Google Research)
OptimizationTabular
🎯 What it does: Designed and implemented an O(1) approximate fully dynamic k-median/k-means algorithm, with an amortized update time of ˜(O(k)) and a query time of ˜(O(k^2)).
Function Space Bayesian Pseudocoreset for Bayesian Neural Networks
Balhae Kim (KAIST), Juho Lee (KAIST)
Image
🎯 What it does: A Bayesian pseudo-core set based on function space was constructed and trained for scalable inference in Bayesian neural networks.
Functional Equivalence and Path Connectivity of Reducible Hyperbolic Tangent Networks
Matthew Farrugia-Roberts (University of Melbourne)
🎯 What it does: This study investigates the functional equivalence classes of reducible parameters in single hidden layer hyperbolic tangent networks and provides corresponding algorithmic formulations and proofs of path connectivity.
Functional Renyi Differential Privacy for Generative Modeling
Dihong Jiang (University of Waterloo), Yaoliang Yu (University of Waterloo)
GenerationData SynthesisSafty and PrivacyGenerative Adversarial NetworkGaussian SplattingImage
🎯 What it does: This paper proposes function-level Rényi differential privacy (f-RDP) theory and applies it to privacy-generating models (DP-kernel), achieving data generation by directly adding noise to the loss function in RKHS.
Functional-Group-Based Diffusion for Pocket-Specific Molecule Generation and Elaboration
Haitao Lin (Westlake University), Stan Z. Li (Westlake University)
GenerationDrug DiscoveryGraph Neural NetworkTransformerDiffusion modelGraph
🎯 What it does: A diffusion model based on functional groups, D3FG, is proposed for the specific molecular generation and modification of protein pockets.
Fused Gromov-Wasserstein Graph Mixup for Graph-level Classifications
Xinyu Ma (Peking University), Wenwu Zhu (Tsinghua University)
ClassificationOptimizationComputational EfficiencyGraph Neural NetworkGraph
🎯 What it does: A graph mixup data augmentation method based on the fused Gromov-Wasserstein (FGW) distance, called FGWMixup, is proposed. It aims to obtain the optimal node matching between two graphs through the optimal transport (OT) problem, thereby generating more representative mixed graphs for graph-level classification.
Future-Dependent Value-Based Off-Policy Evaluation in POMDPs
Masatoshi Uehara (Genentech), Wen Sun (Cornell University)
Reinforcement LearningSequential
🎯 What it does: This paper proposes a future-dependent value function to address the offline policy evaluation (OPE) problem in partially observable Markov decision processes (POMDPs), and provides the corresponding Bellman equation and minimax learning algorithm.
Gacs-Korner Common Information Variational Autoencoder
Michael Kleinman (University of California), Jonathan Kao (University of California)
GenerationData SynthesisRepresentation LearningAuto EncoderImageVideoMultimodality
🎯 What it does: A variational autoencoder based on Gács–Körner common information (GK‑VAE) is proposed, which can separate and quantify common and unique information from multi-view data.
GAIA: Delving into Gradient-based Attribution Abnormality for Out-of-distribution Detection
Jinggang Chen (Huazhong University of Science and Technology), Jing Xiao (Ping An Technology)
Anomaly DetectionExplainability and InterpretabilityTransformerImage
🎯 What it does: A non-parametric post-processing OOD detection framework GAIA is proposed, utilizing gradient attribution anomalies.
GALOPA: Graph Transport Learning with Optimal Plan Alignment
Yejiang Wang (Northeastern University), Ling Li (Northeastern University)
ClassificationRepresentation LearningGraph Neural NetworkGraph
🎯 What it does: Learn graph structure representations through self-supervised methods, aligning the optimal transport plan between graph space and representation space, allowing the GNN encoder to learn embeddings that maintain matching relationships between graphs.
Game Solving with Online Fine-Tuning
Ti-Rong Wu (Academia Sinica), I-Chen Wu (Academia Sinica)
Reinforcement Learning
🎯 What it does: Achieved efficient solving of the 7x7 Killall-Go game by online fine-tuning an AlphaZero-style Proof Cost Network (PCN) during the search process;
GAN You See Me? Enhanced Data Reconstruction Attacks against Split Inference
Ziang Li (Wuhan University), Xiaoyang Xu (Wuhan University)
RestorationGenerationData SynthesisAdversarial AttackGenerative Adversarial NetworkImage
🎯 What it does: Proposes two Split Inference data reconstruction attack methods based on StyleGAN: GLASS and GLASS++.
GAUCHE: A Library for Gaussian Processes in Chemistry
Ryan-Rhys Griffiths (Meta), Jian Tang
OptimizationDrug DiscoveryGraph Neural NetworkGraphTabular
🎯 What it does: Designed and implemented GAUCHE, a Gaussian process library specifically for chemistry, supporting various representations of molecules, reactions, and proteins, as well as their strings, fingerprints, and graph kernels, and integrated with GPyTorch/BoTorch;
Gaussian Differential Privacy on Riemannian Manifolds
Yangdi Jiang (University of Alberta), Bei Jiang (University of Alberta)
Safty and PrivacyTabular
🎯 What it does: A Gaussian Differential Privacy (GDP) mechanism is proposed and implemented on Riemannian manifolds, constructing a Riemannian Gaussian distribution and providing an algorithm to compute the privacy budget µ; experiments validate that this mechanism outperforms the traditional Riemannian Laplace mechanism on the unit sphere.
Gaussian Membership Inference Privacy
Tobias Leemann (University of Tübingen), Gjergji Kasneci (Technical University of Munich)
Safty and PrivacyImageTabular
🎯 What it does: A new privacy concept called f-Membership Inference Privacy (f-MIP) is proposed, and its implementation is achieved through hypothesis testing analysis of the SGD training process.
Gaussian Mixture Solvers for Diffusion Models
Hanzhong Allan Guo, Chongxuan Li (Renmin University of China)
GenerationData SynthesisDiffusion modelImageStochastic Differential Equation
🎯 What it does: A new Gaussian Mixture Solver (GMS) is proposed, which relaxes the assumption of Gaussian transition kernels in the reverse sampling of diffusion models and uses a Gaussian mixture model to approximate the true reverse transition distribution.
Gaussian Partial Information Decomposition: Bias Correction and Application to High-dimensional Data
Praveen Venkatesh (University of Washington), Stefan Mihalas (Allen Institute)
OptimizationComputational EfficiencyTabularTime Series
🎯 What it does: An efficient method for calculating and estimating partial information decomposition (Gaussian PID) for multivariate Gaussian distributions is proposed, along with bias correction for limited sample sizes.
Gaussian Process Probes (GPP) for Uncertainty-Aware Probing
Zi Wang (Google DeepMind), Been Kim (Google DeepMind)
ClassificationAnomaly DetectionConvolutional Neural NetworkGaussian SplattingImage
🎯 What it does: This paper proposes a detection framework based on Gaussian processes (Gaussian Process Probes, GPP), which measures the model's representation of concepts and its uncertainty by constructing the classifier distribution corresponding to the activation vectors of the model output;
General Munchausen Reinforcement Learning with Tsallis Kullback-Leibler Divergence
Lingwei Zhu (University of Alberta), Martha White (University of Alberta)
Reinforcement Learning
🎯 What it does: A generalized Munchausen Value Iteration (MVI(q)) algorithm using Tsallis KL divergence (q>1) is proposed and implemented to address the challenges of traditional KL regularization in function approximation.
Generalised f-Mean Aggregation for Graph Neural Networks
Ryan Kortvelesy (University of Cambridge), Amanda Prorok (University of Cambridge)
Graph Neural NetworkGraph
🎯 What it does: A learnable and interpretable aggregator called GenAgg is proposed, which can uniformly represent all commonly used aggregation functions and can be used as a drop-in replacement in GNNs;
Generalizable Lightweight Proxy for Robust NAS against Diverse Perturbations
Hyeonjeong Ha (Korea Advanced Institute of Science and Technology), Sung Ju Hwang (Korea Advanced Institute of Science and Technology)
OptimizationAdversarial AttackNeural Architecture SearchImage
🎯 What it does: A lightweight zero-cost proxy CRoZe is proposed to evaluate the robustness against various perturbations in the network initialization state and to quickly find general robust models in NAS search.
Generalizable One-shot 3D Neural Head Avatar
Xueting Li (NVIDIA), Jan Kautz (NVIDIA)
GenerationData SynthesisSuper ResolutionGenerative Adversarial NetworkImageVideoPoint Cloud
🎯 What it does: A one-step method for 3D avatar reconstruction and animation from a single facial photo is proposed.
Generalization bounds for neural ordinary differential equations and deep residual networks
Pierre Marion (Sorbonne Université)
TabularOrdinary Differential Equation
🎯 What it does: This paper derives the upper bound of generalization error for parameterized ordinary differential equations (including time-varying neural ODEs and deep residual networks) in supervised learning.
Generalization in the Face of Adaptivity: A Bayesian Perspective
Moshe Shenfeld (Hebrew University of Jerusalem), Katrina Ligett (Hebrew University of Jerusalem)
Gaussian Splatting
🎯 What it does: This paper studies the overfitting problem caused by adaptive queries and proposes a noise-adding based algorithm to address this issue. It proves that a simple noise-adding algorithm can provide variance-dependent guarantees, applicable to unbounded queries.
Generalized Bayesian Inference for Scientific Simulators via Amortized Cost Estimation
Richard Gao (University of Tübingen), Jakob H. Macke (Max Planck Institute for Intelligent Systems)
Neural Architecture SearchTabularBiomedical DataPhysics Related
🎯 What it does: A simulator parameter inference method based on generalized Bayesian inference, ACE, is proposed, which utilizes neural networks to approximate the cost function, thereby achieving efficient Bayesian inference without the need for real-time simulation.
Generalized Belief Transport
Junqi Wang (Rutgers University), Patrick Shafto (Rutgers University)
OptimizationTabular
🎯 What it does: A general belief transmission (GBT) framework is proposed, unifying classical learning models such as Bayesian inference, collaborative reasoning, and discriminative learning into a tunable parameter non-equilibrium optimal transport (UOT) problem. Within this framework, the continuity, differentiability, limit behavior, convergence, and adaptability to environmental drift of the model are studied.
Generalized equivalences between subsampling and ridge regularization
Pratik Patil (University of California), Jin-Hong Du (Carnegie Mellon University)
Image
🎯 What it does: This study investigates the equivalence of subsampling and ridge regression in the context of ensemble ridge estimators, proving that along a specific path between different ridge regularization parameters λ and subsample ratios ψ, the risk is equal to the estimator itself in the large sample limit.
Generalized Information-theoretic Multi-view Clustering
Weitian Huang (South China University of Technology), Hongmin Cai (South China University of Technology)
Auto EncoderTabular
🎯 What it does: A multi-view clustering framework based on information theory has been constructed, utilizing the information bottleneck idea to jointly learn unified representations and clustering under unsupervised conditions.
Generalized Logit Adjustment: Calibrating Fine-tuned Models by Removing Label Bias in Foundation Models
Beier Zhu (Nanyang Technological University), Hanwang Zhang (Nanyang Technological University)
ClassificationOptimizationTransformerSupervised Fine-TuningPrompt EngineeringImageMultimodality
🎯 What it does: This paper proposes a general logarithmic probability adjustment method (GLA) that removes the label bias of the base model by estimating the label prior of the base model and deducting it from the logits of the zero-shot model and the fine-tuned model, thereby improving performance on various downstream tasks.
Generalized Semi-Supervised Learning via Self-Supervised Feature Adaptation
Jiachen Liang (Institute of Computing Technology), Xilin CHEN
Domain AdaptationConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: To address the issue of inconsistent feature distribution in semi-supervised learning (FDM-SSL), a Self-Supervised Feature Adaptation (SSFA) framework is proposed, which first adapts the feature extractor using a self-supervised task, and then generates more reliable pseudo-labels, thereby improving the model's performance on labeled, unlabeled, and unseen distributions.
Generalized test utilities for long-tail performance in extreme multi-label classification
Erik Schultheis (Aalto University), Krzysztof Dembczynski (Yahoo! Research)
ClassificationOptimizationText
🎯 What it does: This paper addresses the long-tail label problem in extreme multi-label classification (XMLC) and proposes an inference method based on the Expected Test Utility (ETU) framework, which can directly optimize macro-average metrics (such as macro F1, macro recall, coverage, etc.) under a given budget k, thereby enhancing the recall ability for rare labels.
Generalized Weighted Path Consistency for Mastering Atari Games
Dengwei Zhao (Shanghai Jiao Tong University), Lei Xu (Shanghai Jiao Tong University)
Reinforcement LearningVideoBenchmark
🎯 What it does: This paper proposes a general method for achieving path consistency (PC) in continuous decision-making environments with immediate rewards (such as Atari games), called GW-PCZero;
Generalizing Importance Weighting to A Universal Solver for Distribution Shift Problems
Tongtong Fang (University of Tokyo), Masashi Sugiyama (RIKEN)
Domain AdaptationConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: A general importance weighting method (GIW) is proposed and implemented, capable of addressing all four types of distribution shift problems with support transitions, and utilizes the validation set to achieve support partitioning during training.
Generalizing Nonlinear ICA Beyond Structural Sparsity
Yujia Zheng (Carnegie Mellon University), Kun Zhang (Carnegie Mellon University)
Flow-based ModelImage
🎯 What it does: A new theory of non-linear ICA identifiability is proposed, extending to cases of under-completeness, partial sparsity, partial source dependence, and flexible grouping structures, along with corresponding identifiability proofs.
Generate What You Prefer: Reshaping Sequential Recommendation via Guided Diffusion
Zhengyi Yang (University of Science and Technology of China), Xiangnan He (University of Science and Technology of China)
GenerationRecommendation SystemTransformerDiffusion modelSequential
🎯 What it does: Re-defines the sequential recommendation task as a learning task to generate an 'ideal item' (oracle item), and directly generates this ideal item based on the user's historical interactions using a guided diffusion model, then provides a recommendation list through nearest neighbor retrieval.
Generating Behaviorally Diverse Policies with Latent Diffusion Models
Shashank Hegde (University of Southern California), Gaurav S. Sukhatme (University of Southern California)
GenerationCompressionRobotic IntelligenceReinforcement LearningDiffusion modelAuto EncoderSequential
🎯 What it does: This paper proposes a method to compress the policy archive generated by Quality Diversity Reinforcement Learning (QD-RL) into a single generative model using diffusion models, which can achieve compression while maintaining performance and coverage, and supports behavior- or language-based policy generation and sequence combination.
Generating Images with Multimodal Language Models
Jing Yu Koh (Carnegie Mellon University), Ruslan Salakhutdinov (Carnegie Mellon University)
GenerationRetrievalTransformerLarge Language ModelDiffusion modelImageTextMultimodalityRetrieval-Augmented Generation
🎯 What it does: In this paper, the authors propose the GILL (Generating Images with Large Language Models) framework, which combines a frozen text large language model (LLM) with a pre-trained image encoder/decoder through embedding mapping. This allows for the processing of any interleaved image and text inputs, and simultaneously generates text, retrieves images, and creates new images within the same model.
Generative Category-level Object Pose Estimation via Diffusion Models
Jiyao Zhang (Peking University), Hao Dong (Peking University)
Object DetectionPose EstimationDiffusion modelScore-based ModelPoint Cloud
🎯 What it does: This paper proposes a category-level object pose estimation framework based on diffusion models, transforming pose estimation into a conditional generation problem. It first uses a score-based diffusion model to generate multiple pose candidates, then employs an energy-based diffusion model to perform likelihood filtering on the candidates, and finally aggregates the results to obtain the final pose.
Generative Modeling through the Semi-dual Formulation of Unbalanced Optimal Transport
Jaemoo Choi (Seoul National University), Myungjoo Kang (Seoul National University)
GenerationAnomaly DetectionOptimizationGenerative Adversarial NetworkImage
🎯 What it does: A generation model based on the semi-dual form of unbalanced optimal transport (UOT) called UOTM is proposed to replace traditional OT generation models.
Generative Modelling of Stochastic Actions with Arbitrary Constraints in Reinforcement Learning
Changyu Chen (Singapore Management University), Pradeep Varakantham (Singapore Management University)
Reinforcement LearningFlow-based ModelSequential
🎯 What it does: A strategy network based on discrete regularization flow is designed, and IAR-A2C is proposed to achieve safe policy learning in constrained multi-dimensional discrete action spaces through invalid action rejection.
Generative Neural Fields by Mixtures of Neural Implicit Functions
Tackgeun You (POSTECH), Bohyung Han (Seoul National University)
GenerationData SynthesisMeta LearningDiffusion modelImagePoint Cloud
🎯 What it does: A generative neural field model based on mixed neural implicit functions (mNIF) is proposed, achieving efficient generation by learning a shared base network and latent mixture coefficients;
Generator Born from Classifier
Runpeng Yu (National University of Singapore), Xinchao Wang (National University of Singapore)
GenerationData SynthesisGenerative Adversarial NetworkImage
🎯 What it does: Without any training samples, a generator is restored and trained using the parameter information of a pre-trained classifier.
Generator Identification for Linear SDEs with Additive and Multiplicative Noise
Yuanyuan Wang (University of Melbourne), Mingming Gong (University of Melbourne)
Time SeriesStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: This paper studies whether the generator (drift and diffusion coefficients) of linear stochastic differential equations (SDEs) under additive noise and multiplicative noise can be uniquely determined from the distribution of the observed process, thereby enabling causal inference for the system.
GenS: Generalizable Neural Surface Reconstruction from Multi-View Images
Rui Peng (Peking University), Ronggang Wang (Peking University)
GenerationData SynthesisNeural Radiance FieldImagePoint Cloud
🎯 What it does: A generalizable neural surface reconstruction model GenS has been developed, capable of directly reconstructing 3D surfaces from multi-view images without the need for single-scene optimization.
GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-localization
Vicente Vivanco Cepeda (University of Central Florida), Mubarak Shah (University of Central Florida)
RetrievalTransformerContrastive LearningImage
🎯 What it does: Global image localization is achieved through the retrieval of images to GPS coordinates, constructing a retrieval system that can directly match any GPS coordinates;
Geodesic Multi-Modal Mixup for Robust Fine-Tuning
Changdae Oh (University of Seoul), Kyungwoo Song (Yonsei University)
ClassificationRetrievalSupervised Fine-TuningContrastive LearningImageTextMultimodality
🎯 What it does: Proposes Geodesic Multi-Modal Mixup (m²-Mix) to fine-tune CLIP, generating hard negative samples and enhancing the uniformity and alignment of cross-modal representations, thereby achieving more robust multi-modal transfer.
Geometric Algebra Transformer
Johann Brehmer (Qualcomm AI Research), Taco Cohen (Qualcomm AI Research)
TransformerPoint CloudPhysics Related
🎯 What it does: This paper presents the Geometric Algebra Transformer (GATr), a general geometric data network that utilizes projective geometric algebra (G_{3,0,1}) for data representation and implements E(3) equivariance through a Transformer.
Geometric Analysis of Matrix Sensing over Graphs
Haixiang Zhang (University of California), Javad Lavaei (University of California)
OptimizationGraph Neural NetworkGraph
🎯 What it does: This paper studies a new class of low-rank matrix sensing problems—Matrix Sensing over Graphs (MSoG)—and provides a rigorous analysis of its optimization landscape, including the strict saddle property and corresponding global convergence analysis.
Geometric Neural Diffusion Processes
Emile Mathieu (University Of Cambridge), Richard E Turner
Diffusion modelTime SeriesStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Construct a geometric diffusion model on infinite-dimensional function spaces, combining equivariant noise processes and equivariant fractional networks, for modeling non-Euclidean data such as tensor fields and spherical trajectories.
Geometric Transformer with Interatomic Positional Encoding
Yusong Wang (Xi'an Jiaotong University), Tie-Yan Liu (Microsoft Research)
TransformerGraph
🎯 What it does: Proposes Geoformer—a geometric molecular modeling framework based on Transformer, which utilizes Interatomic Position Encoding (IPE) to capture the three-dimensional structure of molecules and predict molecular properties.
Geometry-Aware Adaptation for Pretrained Models
Nicholas Roberts (University of Wisconsin-Madison), Frederic Sala (University of Wisconsin-Madison)
ClassificationRepresentation LearningContrastive LearningImageBiomedical Data
🎯 What it does: This paper presents LOKI, a lightweight adapter for pre-trained models that replaces the traditional argmax prediction rule with metric information from the label space, allowing for the prediction of unseen labels without the need for fine-tuning.
Geometry-Informed Neural Operator for Large-Scale 3D PDEs
Zongyi Li (NVIDIA), Anima Anandkumar (NVIDIA)
OptimizationComputational EfficiencyGraph Neural NetworkMesh
🎯 What it does: This paper proposes the Geometry-Informed Neural Operator (GINO), which can learn the solution operator of large three-dimensional partial differential equations under arbitrary geometries and mesh discretizations, particularly for three-dimensional CFD.
GeoPhy: Differentiable Phylogenetic Inference via Geometric Gradients of Tree Topologies
Takahiro Mimori (Waseda University), Michiaki Hamada (Waseda University)
Graph Neural NetworkTabular
🎯 What it does: A fully differentiable variational Bayesian system, GeoPhy, is proposed, which can jointly infer the evolutionary tree topology and branch lengths without pre-selecting tree topologies.
GeoTMI: Predicting Quantum Chemical Property with Easy-to-Obtain Geometry via Positional Denoising
Hyeonsu Kim (KAIST), Woo Youn Kim (KAIST)
Graph Neural NetworkAuto EncoderGraphPhysics Related
🎯 What it does: This paper proposes the GeoTMI training framework, which utilizes easily accessible geometric shapes (low precision or noisy geometry) for predicting quantum chemical properties, and achieves accuracy improvement through the maximization of mutual information among three components and a denoising process.
GEQ: Gaussian Kernel Inspired Equilibrium Models
Mingjie Li (Peking University), Zhouchen Lin (Peking University)
ClassificationOptimizationExplainability and InterpretabilityConvolutional Neural NetworkImage
🎯 What it does: This paper proposes a Gaussian kernel-based optimized induced equilibrium model (GEQ) to replace the traditional linear kernel, enhancing the feature extraction and interpretability of deep equilibrium models.
Getting ViT in Shape: Scaling Laws for Compute-Optimal Model Design
Ibrahim Alabdulmohsin (Google DeepMind), Lucas Beyer (Google DeepMind)
OptimizationComputational EfficiencyTransformerSupervised Fine-TuningImage
🎯 What it does: This paper proposes a shape optimization method based on multi-dimensional scaling laws to determine the width, depth, and MLP dimensions of visual Transformers under a given computational budget, thereby obtaining a computationally optimal model shape.
GEX: A flexible method for approximating influence via Geometric Ensemble
SungYub Kim, Eunho Yang (KAIST)
ClassificationComputational EfficiencyTransformerImage
🎯 What it does: A nonlinear influence function approximation method named GEX is proposed, utilizing geometric ensemble to eliminate the self-influence distribution bias caused by traditional bilinear approximation, thereby allowing for a more accurate assessment of the impact of training samples on model predictions during the post-training phase.
GIMLET: A Unified Graph-Text Model for Instruction-Based Molecule Zero-Shot Learning
Haiteng Zhao (Peking University), Qi Liu (Peking University)
Drug DiscoveryGraph Neural NetworkTransformerSupervised Fine-TuningTextGraphBiomedical Data
🎯 What it does: A unified graph-text Transformer model GIMLET is proposed, which utilizes natural language instructions to perform molecular property prediction tasks under zero-shot conditions.
Glance and Focus: Memory Prompting for Multi-Event Video Question Answering
Ziyi Bai (Key Laboratory of Intelligent Information Processing of Chinese Academy of Sciences Institute of Computing Technology), Xilin CHEN
RecognitionRetrievalTransformerVideoTextBenchmark
🎯 What it does: A two-stage video question-answering model called Glance-Focus is proposed, which first generates event memories from long videos through unsupervised or supervised methods, and then uses these memories as prompts to quickly locate key segments related to the questions and infer answers.
GLIME: General, Stable and Local LIME Explanation
Zeren Tan (Tsinghua University), Jian Li (Tsinghua University)
Explainability and InterpretabilityConvolutional Neural NetworkTransformerImageText
🎯 What it does: This paper studies an improved explanation framework called GLIME, aimed at enhancing the interpretability of black-box model predictions.
Global Convergence Analysis of Local SGD for Two-layer Neural Network without Overparameterization
Yajie Bao (Shanghai Jiao Tong University), Mingrui Liu (George Mason University)
OptimizationFederated LearningTabular
🎯 What it does: This paper proves that vanilla local SGD can achieve global convergence under non-convex loss in two-layer neural networks with Gaussian inputs and without over-parameterization, and achieves linear speedup in a multi-machine environment;
Global Identifiability of $\ell_1$-based Dictionary Learning via Matrix Volume Optimization
Jingzhou Hu (University of Florida), Kejun Huang (University of Florida)
CompressionOptimizationImage
🎯 What it does: This paper proposes a new dictionary learning objective - to minimize the determinant (i.e., volume) of the dictionary matrix under the constraint that the ℓ1 norm of each row of the coefficient matrix does not exceed 1, and proves that this objective can globally identify the complete dictionary and sparse coefficients when the 'sufficiently scattered' condition is met;
Global Optimality in Bivariate Gradient-based DAG Learning
Chang Deng (University of Chicago), Bryon Aragam (Carnegie Mellon University)
OptimizationTabular
🎯 What it does: This paper proposes a simple path-following optimization scheme and proves that it can globally converge to the global minimum of population loss in a bivariate setting.
Global Structure-Aware Diffusion Process for Low-light Image Enhancement
Jinhui HOU, Hui Yuan
RestorationDiffusion modelImageOrdinary Differential Equation
🎯 What it does: A low-light image enhancement framework based on diffusion models is proposed, utilizing global structure-aware rank regularization and uncertainty-guided pixel-level regularization to reduce the curvature of ODE trajectories and enhance details and contrast.
Global Update Tracking: A Decentralized Learning Algorithm for Heterogeneous Data
Sai Aparna Aketi (Purdue University), Kaushik Roy (Purdue University)
OptimizationFederated LearningConvolutional Neural NetworkImage
🎯 What it does: A decentralized learning algorithm GUT is proposed, which alleviates the impact of heterogeneous data on performance by tracking global model updates, and combines quasi-global momentum to obtain QG-GUTm.
Global-correlated 3D-decoupling Transformer for Clothed Avatar Reconstruction
Zechuan Zhang (Zhejiang University), Yi Yang (University of Technology Sydney)
RestorationGenerationTransformerImage
🎯 What it does: A global context 3D decoupling network based on Transformer (GTA) is proposed, capable of reconstructing high-quality 3D models of clothed humans from a single image.
Globally injective and bijective neural operators
Takashi Furuya (Shimane University), Maarten V. de Hoop (Rice University)
🎯 What it does: This paper conducts a rigorous theoretical analysis of the injectivity and bijectivity of neural operators in infinite-dimensional function spaces. It provides necessary and sufficient conditions for layer-wise injectivity under ReLU and bijective activation functions, and proves that injective neural operators are universal approximators. Additionally, it presents methods for maintaining injectivity in finite-rank implementations. The paper further discusses the bijectivity and invertibility of nonlinear integral operators (such as subnetworks, operator Transformers, and integral autoencoders) under sufficient conditions and constructs their inverse operators.
Globally solving the Gromov-Wasserstein problem for point clouds in low dimensional Euclidean spaces
Martin Ryner (Vironova AB), Johan Karlsson (KTH Royal Institute of Technology)
OptimizationPoint Cloud
🎯 What it does: A cutting plane algorithm utilizing low-rank structures is proposed, which can globally solve the Gromov-Wasserstein (GW) problem of point clouds in low-dimensional Euclidean space and provides a convergence proof.
GLOBER: Coherent Non-autoregressive Video Generation via GLOBal Guided Video DecodER
Mingzhen Sun (Institute of Automation, Chinese Academy of Sciences), Jing Liu (Institute of Automation, Chinese Academy of Sciences)
GenerationData SynthesisTransformerDiffusion modelVideo
🎯 What it does: A novel non-autoregressive video generation framework called GLOBER is proposed, which first generates global features as global guidance and then synthesizes video frames through a non-autoregressive decoder, achieving global coherence and local realism.
GloptiNets: Scalable Non-Convex Optimization with Certificates
Gaspard Beugnot (Inria), Alessandro Rudi (Inria)
Optimization
🎯 What it does: A global non-convex optimization algorithm GloptiNets based on the extended k-SoS model is proposed, along with an explicit optimality certificate.
GlucoSynth: Generating Differentially-Private Synthetic Glucose Traces
Josephine Lamp (University of Virginia), David Evans (University of Virginia)
Data SynthesisSafty and PrivacyRecurrent Neural NetworkAuto EncoderGenerative Adversarial NetworkTime SeriesSequentialElectronic Health Records
🎯 What it does: A privacy-preserving GAN framework named GlucoSynth is proposed for generating high-quality synthetic glucose time series data.
GlyphControl: Glyph Conditional Control for Visual Text Generation
Yukang Yang (Princeton University), Kai Chen (Microsoft Research Asia)
GenerationDiffusion modelImageText
🎯 What it does: Proposes the GlyphControl method, which implements controllable visual text generation by adding a glyph image-based ControlNet on Stable Diffusion.
GMSF: Global Matching Scene Flow
Yushan Zhang (Linköping University), Michael Felsberg (Linköping University)
Autonomous DrivingOptimizationGraph Neural NetworkTransformerOptical FlowPoint Cloud
🎯 What it does: A scene flow estimation method based on single-scale global matching, GMSF, is proposed.
GNeSF: Generalizable Neural Semantic Fields
Hanlin Chen (National University of Singapore), Gim Hee Lee (National University of Singapore)
SegmentationData SynthesisNeural Radiance FieldImage
🎯 What it does: A generalizable neural semantic field (GNeSF) is proposed, which integrates multi-view 2D semantics through a soft voting mechanism and utilizes disparity information to predict 3D semantics.
GNNEvaluator: Evaluating GNN Performance On Unseen Graphs Without Labels
Xin Zheng (Monash University), Shirui Pan (Griffith University)
Graph Neural NetworkGraph
🎯 What it does: A framework is proposed to evaluate the performance of trained Graph Neural Networks (GNN) on unlabelled test graphs;
Goal Driven Discovery of Distributional Differences via Language Descriptions
Ruiqi Zhong (University of California), Jacob Steinhardt (University of California)
TransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: This paper proposes a system for automatically discovering and generating differences between two large corpora. The goal-driven D5 task aligns user-specified exploration objectives with text and outputs natural language predicate descriptions of the differences.
Goal-conditioned Offline Planning from Curious Exploration
Marco Bagatella (ETH Zurich and Max Planck Institute for Intelligent Systems), Georg Martius (University of Tübingen and Max Planck Institute for Intelligent Systems)
OptimizationRobotic IntelligenceReinforcement LearningAgentic AISequential
🎯 What it does: In an offline setting without environmental interaction, learn and plan goal-directed behavior using trajectories generated by curiosity-driven exploration and a dynamics model.
Goal-Conditioned Predictive Coding for Offline Reinforcement Learning
Zilai Zeng (Brown University), Chen Sun (Brown University)
TransformerReinforcement LearningSequential
🎯 What it does: A two-stage framework is proposed: first, a sequence model is used to learn trajectory representations, and then this representation is used to train a goal-oriented policy.
Going Beyond Linear Mode Connectivity: The Layerwise Linear Feature Connectivity
Zhanpeng Zhou (Shanghai Jiao Tong University), Wei Hu (University of Michigan)
ClassificationConvolutional Neural NetworkImage
🎯 What it does: This paper proposes and validates the concept of Hierarchical Linear Feature Connectivity (LLFC), which states that along the linear interpolation path of two model parameters, the feature mappings of almost all layers maintain a linear relationship. It further explores the mechanism behind LLFC and its relationship with existing Permutation methods.
Going beyond persistent homology using persistent homology
Johanna Emilia Immonen (University of Helsinki), Vikas Garg (Aalto University)
RecognitionRepresentation LearningGraph Neural NetworkGraph
🎯 What it does: Through theoretical and experimental analysis, the expressive power of persistent homology (PH) under vertex color and edge color filtering in graph structure recognition is clarified, and a more powerful topological feature extraction method called RePHINE is proposed;
Gold-YOLO: Efficient Object Detector via Gather-and-Distribute Mechanism
Chengcheng Wang (Huawei), Kai Han (Huawei)
Object DetectionSegmentationConvolutional Neural NetworkTransformerSupervised Fine-TuningImage
🎯 What it does: This paper presents Gold-YOLO, an improved real-time object detector based on the YOLO series, with the core improvements being the Gather-and-Distribute (GD) mechanism and MAE pre-training;
GPEX, A Framework For Interpreting Artificial Neural Networks
Amir Akbarnejad (University of Alberta), Nilanjan Ray (University of Alberta)
Explainability and InterpretabilityKnowledge DistillationImage
🎯 What it does: This paper proposes the GPEX framework, which explains any feedforward artificial neural network (ANN) using Gaussian processes (GP), derives a new ELBO that globally matches the GP posterior with the ANN output, and implements knowledge distillation.
GPT-ST: Generative Pre-Training of Spatio-Temporal Graph Neural Networks
Zhonghang Li (South China University of Technology), Chao Huang (University of Hong Kong)
Graph Neural NetworkAuto EncoderGraphTime Series
🎯 What it does: This paper proposes GPT-ST, a pre-training framework based on masked autoencoders to enhance the predictive performance of spatiotemporal graph neural networks.
GPT4Tools: Teaching Large Language Model to Use Tools via Self-instruction
Rui Yang (Tsinghua University), Ying Shan (Tencent)
Image TranslationGenerationRetrievalTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringImageTextMultimodalityRetrieval-Augmented Generation
🎯 What it does: This paper proposes GPT4Tools, which utilizes GPT-3.5 as a teacher to generate tool instruction data, and fine-tunes open-source LLMs (such as Vicuna, LLaMA, OPT) with LoRA to actively invoke multimodal tools in visual tasks.
Gradient Descent with Linearly Correlated Noise: Theory and Applications to Differential Privacy
Anastasia Koloskova (École Polytechnique Fédérale de Lausanne), Hugh Brendan McMahan
OptimizationFederated LearningSafty and PrivacyConvolutional Neural NetworkRecurrent Neural NetworkGaussian SplattingImageTabularTime Series
🎯 What it does: This study investigates the convergence behavior of gradient descent under linearly correlated noise and proposes a more compact theoretical analysis and new factorization objectives for matrix factorization mechanisms in differential privacy (such as DP-FTRL and MF-DP-FTRL), followed by experimental validation on standard datasets.
Gradient Flossing: Improving Gradient Descent through Dynamic Control of Jacobians
Rainer Engelken (Zuckerman Mind Brain Behavior Institute Columbia University New York)
OptimizationRecurrent Neural NetworkSequential
🎯 What it does: This paper proposes a gradient flossing method that dynamically controls the Jacobian matrix of RNN gradients by regularizing the Lyapunov exponent, enhancing the stability of gradient propagation and training effectiveness.