ICML 2024 Papers — Page 10
International Conference on Machine Learning · 2610 papers
Fourier Controller Networks for Real-Time Decision-Making in Embodied Learning
Hengkai Tan (Tsinghua University), Jun Zhu (Tsinghua University)
Robotic IntelligenceReinforcement LearningSequential
🎯 What it does: A Fourier Controller Network (FCNet) is proposed, designing a model for low-frequency feature extraction and real-time decision-making from a frequency domain perspective.
FRAG: Frequency Adapting Group for Diffusion Video Editing
Sunjae Yoon (Korea Advanced Institute of Science and Technology), Chang D. Yoo (Korea Advanced Institute of Science and Technology)
GenerationDiffusion modelVideo
🎯 What it does: Proposes the FRAG mechanism, which dynamically adjusts the frequency-related receptive field in video diffusion editing to enhance video editing quality.
FrameQuant: Flexible Low-Bit Quantization for Transformers
Harshavardhan Adepu (University of Wisconsin-Madison), Vikas Singh (Google Research)
TransformerImageText
🎯 What it does: The FrameQuant method is proposed, achieving low-bit (≈2 bits) quantization on Transformer weights by fusing frames.
FRAPPÉ: A Group Fairness Framework for Post-Processing Everything
Alexandru Tifrea (ETH Zurich), Flavien Prost (Google DeepMind)
OptimizationSupervised Fine-TuningTabular
🎯 What it does: The FRAPP'E framework is proposed, which converts any regularized in-processing fairness method into a post-processing method, decoupling model training from fairness adjustment.
FreeBind: Free Lunch in Unified Multimodal Space via Knowledge Fusion
Zehan Wang (Zhejiang University), Zhou Zhao (Zhejiang University)
RetrievalRepresentation LearningContrastive LearningImageTextMultimodalityAudio
🎯 What it does: The FreeBind method is proposed, which integrates expert spatial knowledge into a pre-trained unified multimodal space through space bonds, achieving lightweight spatial enhancement.
From Biased Selective Labels to Pseudo-Labels: An Expectation-Maximization Framework for Learning from Biased Decisions
Trenton Chang (University of Michigan), Jenna Wiens (University of Michigan)
ClassificationBiomedical DataElectronic Health Records
🎯 What it does: Proposed and implemented the Disparate Censorship Expectation-Maximization (DCEM) method for learning fair and accurate binary classification models on data with disparate censorship.
From Coarse to Fine: Enable Comprehensive Graph Self-supervised Learning with Multi-granular Semantic Ensemble
Qianlong Wen (University of Notre Dame), Yanfang Ye (University of Notre Dame)
Knowledge DistillationRepresentation LearningDrug DiscoveryGraph Neural NetworkContrastive LearningGraphBiomedical Data
🎯 What it does: A plugin-based multi-granularity graph semantic integration framework MGSE is proposed, which utilizes knowledge distillation to learn and integrate representations from a single teacher model at different granularities;
From Fourier to Neural ODEs: Flow Matching for Modeling Complex Systems
Xin Li (National University of Defense Technology), Wei Lin (Fudan University)
Flow-based ModelTime SeriesPhysics RelatedOrdinary Differential Equation
🎯 What it does: Proposes a Fourier analysis-based framework for estimating gradients and using them as training objectives in a simulation-free neural ODE framework (FNODEs), achieving efficient dynamic modeling through positive feedback data augmentation.
From Generalization Analysis to Optimization Designs for State Space Models
Fusheng Liu (National University of Singapore), Qianxiao Li (National University of Singapore)
OptimizationRecurrent Neural NetworkTime SeriesSequential
🎯 What it does: The paper studies the generalization performance of state space models (SSM) and proposes a generalization upper bound based on temporal dependencies. It then uses this upper bound to design new initialization scaling rules and regularization methods to enhance the training stability and generalization ability of the model.
From Geometry to Causality- Ricci Curvature and the Reliability of Causal Inference on Networks
Amirhossein Farzam (Duke University), Guillermo Sapiro (Apple)
Graph Neural NetworkGraph
🎯 What it does: This paper explores the theoretical connection between graph curvature and causal inference in networks, verifying that Ricci curvature can assess the reliability of causal estimates, and proposes using Ricci flow to smooth networks to enhance GNN causal estimation.
From Inverse Optimization to Feasibility to ERM
Saurabh kumar Mishra, Sharan Vaswani (Sierra Project Team Inria)
OptimizationTabular
🎯 What it does: The paper proposes a method to transform the Context Inverse Optimization (CIO) problem into a convex feasibility problem and further map it to the Empirical Risk Minimization (ERM) framework, addressing the non-differentiable challenges of linear programming in inverse learning, and presents a scalable first-order optimization algorithm.
From Neurons to Neutrons: A Case Study in Interpretability
Ouail Kitouni (Massachusetts Institute of Technology), Mike Williams (Massachusetts Institute of Technology)
OptimizationExplainability and InterpretabilityKnowledge DistillationRepresentation LearningTabularPhysics Related
🎯 What it does: This study investigates the use of Mechanistic Interpretability methods to explain neural networks trained on nuclear physics data, uncovering low-dimensional representations within the network that match known semi-empirical mass formulas (SEMF) and shell models, and deriving an improved nuclear binding energy formula through symbolic regression.
From Self-Attention to Markov Models: Unveiling the Dynamics of Generative Transformers
Muhammed Emrullah Ildiz, Samet Oymak
GenerationData SynthesisTransformerPrompt EngineeringText
🎯 What it does: This paper studies the generative dynamics of single-layer self-attention models and maps them to a context-conditioned Markov chain (CCMC). Within this framework, it provides consistency and sample complexity theory, and analyzes the distribution collapse and text repetition issues caused by single-trajectory learning.
From Vision to Audio and Beyond: A Unified Model for Audio-Visual Representation and Generation
Kun Su (University of Washington), Eli Shlizerman (University of Washington)
GenerationRetrievalTransformerContrastive LearningVideoMultimodalityAudio
🎯 What it does: Trained a unified visual-audio model VAB, which learns audio and video representations through visual-conditioned masked audio token prediction and achieves parallel generation from video to audio.
From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systems
Jianliang He (Fudan University), Zhuoran Yang (Yale University)
Robotic IntelligenceTransformerLarge Language ModelReinforcement LearningContrastive LearningText
🎯 What it does: The paper analyzes the decision-making process of LLM agents through a hierarchical reinforcement learning framework, demonstrating that the LLM Planner achieves planning through Bayesian aggregation imitation learning and proposes exploration strategies to avoid linear regret.
From Yes-Men to Truth-Tellers: Addressing Sycophancy in Large Language Models with Pinpoint Tuning
Wei Chen (Zhejiang University), Jieping Ye (Alibaba Cloud)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: A positioning fine-tuning method (SPT) is proposed, which only fine-tunes a very small number of attention heads, aiming to precisely reduce the sycophantic behavior of large language models while maintaining their general capabilities.
Full-Atom Peptide Design based on Multi-modal Flow Matching
Jiahan Li (Helixon Research), Jianzhu Ma (Tsinghua University)
GenerationDrug DiscoveryProtein Structure PredictionFlow-based ModelBiomedical DataBenchmarkOrdinary Differential Equation
🎯 What it does: A multi-modal flow matching framework called PepFlow is proposed to generate full-atom peptide sequences, backbones (SE(3) framework), side-chain angles (high-dimensional rings), and amino acid types (probability simplices) based on target proteins.
Fully-Dynamic Approximate Decision Trees With Worst-Case Update Time Guarantees
Marco Bressan (University of Milan), Mauro Sozio (Institut Polytechnique de Paris)
ClassificationOptimization
🎯 What it does: The paper studies the problem of maintaining decision trees in a fully dynamic environment (where insertions and deletions can occur at any time) and proposes the algorithm FUDY-WC, which guarantees the quality of the tree (close to the offline optimal tree) after each update, with an update time of O(β^{-3} d log^4 n) in the worst case.
Fundamental Benefit of Alternating Updates in Minimax Optimization
Jaewook Lee (Korea Advanced Institute of Science and Technology), Chulhee Yun (Korea Advanced Institute of Science and Technology)
Optimization
🎯 What it does: This paper studies the fundamental advantages of alternating updates in min-max optimization, proposes the Alternating Extrapolated GDA (Alex-GDA) algorithm, and compares the convergence of Sim-GDA and Alt-GDA.
Fundamental Limitations of Alignment in Large Language Models
Yotam Wolf (Hebrew University of Jerusalem), Amnon Shashua (Hebrew University of Jerusalem)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Proposes the Behavioral Expectation Bound (BEB) theoretical framework for a quantitative analysis of the alignment limitations of large language models (LLMs), demonstrating that even when aligned, LLMs can still be induced to exhibit undesirable behavior through short texts.
Fundamental Limits of Distributed Covariance Matrix Estimation Under Communication Constraints
Mohammad Reza Rahmani (Sharif University of Technology), Mohammad Reza Aref (Sharif University of Technology)
OptimizationTabular
🎯 What it does: This paper studies the problem of distributed covariance matrix estimation under communication constraints, considering how two agents can effectively estimate the covariance matrix of high-dimensional random vectors with a limited communication budget.
FuRL: Visual-Language Models as Fuzzy Rewards for Reinforcement Learning
Yuwei Fu (McGill University), Benoit Boulet (McGill University)
Robotic IntelligenceTransformerReinforcement LearningVision Language ModelMultimodality
🎯 What it does: Utilizing a pre-trained Vision-Language Model (VLM) as a reward signal to improve sample efficiency in sparse reward reinforcement learning tasks, the FuRL method is proposed and validated on the Meta-world MT10.
GALA3D: Towards Text-to-3D Complex Scene Generation via Layout-guided Generative Gaussian Splatting
Xiaoyu Zhou (Peking University), Ming-Hsuan Yang (Google DeepMind)
GenerationData SynthesisTransformerLarge Language ModelDiffusion modelGaussian SplattingText
🎯 What it does: A method is proposed for text-driven complex scene 3D generation and interactive editing, which generates layouts based on large language models and utilizes layout-guided 3D Gaussian Splatting.
GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection
Jiawei Zhao (California Institute of Technology), Yuandong Tian (Meta AI)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: A gradient low-rank projection (GaLore) strategy is proposed, which utilizes the low-rank structure of gradients to significantly reduce memory usage during the pre-training and fine-tuning processes of LLMs, especially for optimizer states, without changing the training dynamics.
Gambling-Based Confidence Sequences for Bounded Random Vectors
Jongha Jon Ryu (Massachusetts Institute of Technology), Gregory W. Wornell (Massachusetts Institute of Technology)
TabularFinance Related
🎯 What it does: A general gambling framework is proposed for constructing time-uniform confidence sequences (CS) for bounded random vector means;
GATE: How to Keep Out Intrusive Neighbors
Nimrah Mustafa (CISPA Helmholtz Center for Information Security), Rebekka Burkholz (CISPA Helmholtz Center for Information Security)
Graph Neural NetworkGraph
🎯 What it does: Proposes GATE, an extension of the Graph Attention Network, which can turn off neighborhood aggregation when needed to alleviate over-smoothing.
Gated Linear Attention Transformers with Hardware-Efficient Training
Songlin Yang (Massachusetts Institute of Technology), Yoon Kim (Massachusetts Institute of Technology)
TransformerLarge Language ModelText
🎯 What it does: Proposes the FLASHLINEARATTENTION hardware-efficient linear attention algorithm and introduces data-dependent gating (GLA) in the Transformer architecture, achieving performance competitive with traditional softmax Transformers from linear attention.
Gaussian Plane-Wave Neural Operator for Electron Density Estimation
Seongsu Kim (Pohang University of Science and Technology), Sungsoo Ahn (Pohang University of Science and Technology)
Graph Neural NetworkTabularPhysics Related
🎯 What it does: This paper proposes a Gaussian Plane Wave Hybrid Basis Neural Operator (GPWNO) for efficient prediction of electronic density in molecules and solids.
Gaussian Processes on Cellular Complexes
Mathieu Alain (University College London), Marc Peter Deisenroth (California Institute of Technology)
GraphStochastic Differential Equation
🎯 What it does: Proposes the definition of higher-order Gaussian processes on cellular complexes and constructs new kernel functions.
GaussianPro: 3D Gaussian Splatting with Progressive Propagation
Kai Cheng (University of Science and Technology of China), Xuejin Chen (University of Science and Technology of China)
Autonomous DrivingOptimizationGaussian SplattingPoint Cloud
🎯 What it does: This paper proposes GaussianPro, a method that enhances the geometric and rendering quality of sparse textured scenes based on 3D Gaussian Splatting through advanced propagation strategies.
GeminiFusion: Efficient Pixel-wise Multimodal Fusion for Vision Transformer
Ding Jia (Peking University), Xinghao Chen (Huawei Noah's Ark Lab)
Image TranslationObject DetectionSegmentationTransformerImageMultimodality
🎯 What it does: A pixel-level multimodal fusion method named GeminiFusion is proposed, which enables efficient cross-modal information interaction in visual Transformers.
GenCO: Generating Diverse Designs with Combinatorial Constraints
Aaron M Ferber, Yuandong Tian (Meta)
GenerationOptimizationAuto EncoderGenerative Adversarial NetworkTabular
🎯 What it does: A generative framework named GenCO is proposed, which utilizes a differentiable combinatorial optimization solver to ensure that generated samples meet hard combinatorial constraints during training while maintaining a fit to the data distribution.
Generalist Equivariant Transformer Towards 3D Molecular Interaction Learning
Xiangzhe Kong (Tsinghua University), Yang Liu (Tsinghua University)
Drug DiscoveryGraph Neural NetworkTransformerGraphBiomedical Data
🎯 What it does: This paper proposes a unified representation of different molecules (proteins, small molecules, nucleic acids, etc.) as a geometric graph of sets, and designs a Generalist Equivariant Transformer (GET) to achieve efficient learning of this representation.
Generalization Analysis for Multi-Label Learning
Yifan Zhang, Min-Ling Zhang (Southeast University)
Classification
🎯 What it does: This paper derives the upper bound of generalization error for multi-label learning by introducing a class of projection functions and a new vector contraction inequality, significantly reducing the dependence on the number of labels, and systematically analyzes the impact of label correlation on generalization performance. It also defines the label-based ranking Rademacher complexity for macro-average AUC and provides its generalization bounds as well as its relationship with class imbalance.
Generalization Analysis of Deep Non-linear Matrix Completion
Antoine Ledent (Singapore Management University), Rodrigo Alves (Czech Technical University)
Recommendation SystemOptimizationTabular
🎯 What it does: This paper studies the matrix completion problem under the Schatten p pseudo-norm constraint and proposes a novel nonlinear model called Functionally Rescaled Matrix Completion (FRMC), which enhances prediction performance by adding a learnable scalar transformation to the low-rank latent matrix.
Generalization Analysis of Stochastic Weight Averaging with General Sampling
Peng Wang (Huazhong University of Science and Technology), Dacheng Tao (Nanyang Technological University)
OptimizationConvolutional Neural NetworkImageTabular
🎯 What it does: The theoretical analysis of the generalization ability of Stochastic Weight Averaging (SWA) under convex/non-convex objective functions and in both sampling with and without replacement is conducted, providing corresponding uniform stability and upper bounds on generalization error.
Generalization Bound and New Algorithm for Clean-Label Backdoor Attack
Lijia Yu (Institute of Software, Chinese Academy of Sciences), Lijun Zhang (Institute of Software, Chinese Academy of Sciences)
Adversarial AttackConvolutional Neural NetworkImage
🎯 What it does: A generalization theory for clean-label backdoor attacks is proposed, along with an algorithm-independent error upper bound. Based on this theory, a trigger combining adversarial noise and indiscriminate poisoning is designed.
Generalization Bounds for Causal Regression: Insights, Guarantees and Sensitivity Analysis
Daniel Csillag (Fundacao Getulio Vargas), Guilherme Tegoni Goedert (Fundacao Getulio Vargas)
Meta LearningTabularTime Series
🎯 What it does: This paper derives a generalization bound for causal regression, providing theoretical guarantees under finite samples, and is applicable to result regression and individualized treatment effect estimation in the presence of hidden confounding and missing positivity assumptions.
Generalization Bounds for Heavy-Tailed SDEs through the Fractional Fokker-Planck Equation
Benjamin Dupuis (Inria), Umut Simsekli (Inria)
ImageStochastic Differential Equation
🎯 What it does: This paper proves high-probability generalization bounds for heavy-tailed stochastic differential equations (SDEs), addressing issues related to expected generalization bounds and intractable information-theoretic terms found in previous studies.
Generalization Error of Graph Neural Networks in the Mean-field Regime
Gholamali Aminian (Alan Turing Institute), Samuel N. Cohen (Mathematical Institute)
ClassificationGraph Neural NetworkGraphBiomedical Data
🎯 What it does: This study investigates the generalization error of an over-parameterized single hidden layer Graph Convolutional Network (GCN) and a Graph Message Passing Neural Network (MPGNN) in graph classification tasks, providing an upper bound.
Generalization in Kernel Regression Under Realistic Assumptions
Daniel Barzilai (Weizmann Institute of Science), Ohad Shamir (Weizmann Institute of Science)
🎯 What it does: This paper provides a unified theoretical framework for bounding the excess risk of kernel regression in almost all common and realistic settings, analyzing the benign overfitting phenomenon of kernel regression in high-dimensional inputs and fixed dimensions.
Generalization to New Sequential Decision Making Tasks with In-Context Learning
Sharath Chandra Raparthy (AI at Meta), Roberta Raileanu (AI at Meta)
Robotic IntelligenceReinforcement Learning from Human FeedbackTransformerSequential
🎯 What it does: This paper studies how to leverage complete trajectory context to transfer to completely different new tasks by implementing weight-free few-shot learning using causal Transformers in sequential decision-making tasks.
Generalized Neural Collapse for a Large Number of Classes
Jiachen Jiang (Ohio State University), Zhihui Zhu (Ohio State University)
ClassificationImage
🎯 What it does: This paper conducts theoretical and experimental research on the last layer features and classifiers of deep learning models in scenarios with a large number of classes, proposing a concept called Generalized Neural Collapse (GNC).
Generalized Preference Optimization: A Unified Approach to Offline Alignment
Yunhao Tang (Google DeepMind), Bilal Piot (Google DeepMind)
OptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: This paper proposes Generalized Preference Optimization (GPO), unifying offline preference optimization into a framework based on convex loss functions, which encompasses and extends existing algorithms such as DPO, IPO, and SLiC;
Generalized Smooth Variational Inequalities: Methods with Adaptive Stepsizes
Daniil Vankov (Arizona State University), Lalitha Sankar (Arizona State University)
OptimizationGenerative Adversarial Network
🎯 What it does: The study investigates variational inequalities (VI) under non-monotonic and non-smooth conditions, proposing the convergence and convergence rates of projection, Korpelevich, and Popov methods using adaptive step sizes under α-symmetry (generalized smoothness) and p-quasi sharpness conditions.
Generalized Sobolev Transport for Probability Measures on a Graph
Tam Le (Institute of Statistical Mathematics), Kenji Fukumizu (Institute of Statistical Mathematics)
OptimizationGraph Neural NetworkTextGraph
🎯 What it does: This paper proposes a generalized Sobolev transport (GST) for calculating probability measures in metric spaces, combining Sobolev transport (ST) with Orlicz geometric structures to obtain a metric that can be solved through univariate optimization.
Generalizing Knowledge Graph Embedding with Universal Orthogonal Parameterization
Rui Li (Renmin University of China), Xu Chen (Renmin University of China)
Adversarial AttackGraph Neural NetworkGraph
🎯 What it does: A general knowledge graph embedding framework called GoldE has been designed and implemented. This framework achieves arbitrary dimensional and geometric (Euclidean, elliptical, hyperbolic) relationship transformations through unified orthogonal parameterization, and integrates different geometries on product manifolds to adapt to the heterogeneous topology of graphs.
Generalizing Orthogonalization for Models with Non-Linearities
David Rügamer (Ludwig Maximilian University of Munich), Thomas Nagler (Ludwig Maximilian University of Munich)
OptimizationSafty and PrivacyMultimodalityTabularElectronic Health Records
🎯 What it does: This paper proposes a general correction method that extends orthogonalization to nonlinear models and tensor predictions to eliminate implicit sensitive information in black-box models.
Generating Chain-of-Thoughts with a Pairwise-Comparison Approach to Searching for the Most Promising Intermediate Thought
Zhen-Yu Zhang (Center for Advanced Intelligence Project RIKEN), Masashi Sugiyama (Graduate School of Frontier Sciences University of Tokyo)
GenerationTransformerLarge Language ModelPrompt EngineeringTextChain-of-Thought
🎯 What it does: A chain-of-thought (CoT) generation framework based on pairwise comparison, called C-ToT, is proposed to select the most promising intermediate thoughts under LLM feedback noise.
Generating In-Distribution Proxy Graphs for Explaining Graph Neural Networks
Zhuomin Chen (Florida International University), Dongsheng Luo (Florida International University)
Explainability and InterpretabilityGraph Neural NetworkAuto EncoderGraph
🎯 What it does: This paper proposes an explanation framework based on proxy graphs, utilizing graph autoencoders to generate proxy graphs that are similar to the training distribution, thereby addressing the out-of-distribution (OOD) problem of traditional GNN explanation subgraphs.
Generative Active Learning for Long-tailed Instance Segmentation
Muzhi Zhu (Zhejiang University), Chunhua Shen (Ant Group)
Object DetectionSegmentationGenerative Adversarial NetworkImageBenchmark
🎯 What it does: This paper proposes a Generative Active Learning framework for the long-tail instance segmentation task, which filters beneficial data by estimating the contribution of generated samples to improve model performance.
Generative Conditional Distributions by Neural (Entropic) Optimal Transport
Bao Nguyen (VinUniversity), Viet Anh Nguyen (VinAI Research)
GenerationData SynthesisOptimizationGenerative Adversarial NetworkTabularBiomedical Data
🎯 What it does: This paper proposes a neural network-based entropy-regularized optimal transport method called GENTLE, designed to learn the conditional distribution of a one-dimensional response corresponding to multi-dimensional covariates in situations with scarce samples.
Generative Enzyme Design Guided by Functionally Important Sites and Small-Molecule Substrates
Zhenqiao Song (Carnegie Mellon University), Lei Li (Carnegie Mellon University)
GenerationDrug DiscoveryProtein Structure PredictionGraph Neural NetworkTransformerGraphBenchmark
🎯 What it does: A unified generative model, EnzyGen, has been designed to simultaneously generate the amino acid sequences and three-dimensional scaffolds of enzymes under the conditions of given functionally important sites and substrates.
Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Design
Andrew Campbell (University of Oxford), Tommi Jaakkola (Massachusetts Institute of Technology)
GenerationDrug DiscoveryTransformerFlow-based ModelTextBiomedical Data
🎯 What it does: This paper studies a new framework for implementing flow models on discrete state spaces, called DFM, and combines it with continuous space flow models for protein co-design.
Generative Marginalization Models
Sulin Liu (Princeton University), Ryan P Adams
GenerationData SynthesisReinforcement LearningImageText
🎯 What it does: A marginalized model (MAM) is proposed, which is a generative model for high-dimensional discrete data that can estimate the marginal probabilities of any subset with a single forward pass and supports generation in any order.
Generative Modeling on Manifolds Through Mixture of Riemannian Diffusion Processes
Jaehyeong Jo (Korea Advanced Institute of Science and Technology), Sung Ju Hwang (Korea Advanced Institute of Science and Technology)
GenerationData SynthesisDiffusion modelMeshStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: A generative model constructed using bridge process mixtures on Riemannian manifolds—Riemannian Diffusion Mixture—is proposed for learning the distribution of manifold data.
Genie: Generative Interactive Environments
Jake Bruce (Google DeepMind), Tim Rocktäschel (University of British Columbia)
GenerationRobotic IntelligenceTransformerReinforcement LearningWorld ModelVideo
🎯 What it does: Introducing Genie, a generative interactive environment based on unsupervised learning from internet videos, capable of generating controllable virtual worlds from prompts such as text, images, or sketches.
GeoAB: Towards Realistic Antibody Design and Reliable Affinity Maturation
Haitao Lin (Zhejiang University), Stan Z. Li (Westlake University)
OptimizationDrug DiscoveryGraph Neural NetworkNeural Radiance FieldGraphBiomedical Data
🎯 What it does: GeoAB proposes a generation-optimization framework for the co-design of antibody CDRs and affinity maturation, which includes Geo-Initializer, Geo-Refiner, and Geo-Optimizer. It can generate CDR structures that comply with physical geometric constraints and accurately predict mutation ΔΔG.
Geometric Active Exploration in Markov Decision Processes: the Benefit of Abstraction
Riccardo De Santi (ETH Zurich), Andreas Krause (ETH Zurich)
OptimizationComputational EfficiencyReinforcement Learning
🎯 What it does: Proposes the Geometric Active Exploration (GAE) algorithm, which transforms the active experimental design problem into convex RL, and enhances statistical and computational efficiency using MDP isomorphic abstraction.
Geometry-Aware Instrumental Variable Regression
Heiner Kremer (Max Planck Institute for Intelligent Systems), Bernhard Schölkopf (Max Planck Institute for Intelligent Systems)
Tabular
🎯 What it does: The Sinkhorn Method of Moments (SMM) is proposed, a geometric-aware IV regression estimator implemented using Sinkhorn distance;
Geometry-Calibrated DRO: Combating Over-Pessimism with Free Energy Implications
Jiashuo Liu (Tsinghua University), Peng Cui (Tsinghua University)
OptimizationGraph Neural NetworkTabular
🎯 What it does: This study investigates the overly pessimistic problem that arises in distributionally robust optimization (DRO) in the presence of subgroup shifts and noisy samples, and proposes a Geometric Calibrated DRO (GCDRO) scheme. This scheme suppresses the excessive weighting of noisy samples by incorporating geometric Wasserstein distance on the uncertainty set, along with data geometry-based total variation and entropy regularization.
GeoMFormer: A General Architecture for Geometric Molecular Representation Learning
Tianlang Chen (Peking University), Liwei Wang (Peking University)
Representation LearningDrug DiscoveryTransformerGraph
🎯 What it does: A two-stream Transformer framework called GeoMFormer is proposed, which can simultaneously learn invariant and equivariant representations of molecular systems.
GeoReasoner: Geo-localization with Reasoning in Street Views using a Large Vision-Language Model
Ling Li (Hong Kong University of Science and Technology), Wei Zeng (Hong Kong University of Science and Technology)
SegmentationRetrievalTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodality
🎯 What it does: This study investigates geographic localization based on street view images and embeds reasoning capabilities in a large visual-language model (LVLM), proposing the GeoReasoner framework.
Get More with LESS: Synthesizing Recurrence with KV Cache Compression for Efficient LLM Inference
Harry Dong (Carnegie Mellon University), Beidi Chen (Meta AI)
CompressionComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: A constant-size low-rank cache called LESS is proposed, which is integrated with an eviction-based sparse KV cache strategy to significantly compress the KV cache during LLM inference while maintaining performance.
Getting the most out of your tokenizer for pre-training and domain adaptation
Gautier Dagan (University of Edinburgh), Baptiste Roziere (Meta AI)
GenerationCompressionDomain AdaptationOptimizationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Research and optimize the tokenizer of code language models, including vocabulary size, pre-tokenization regular expressions, and training data ratio, evaluating their impact on model compression rate, inference speed, context length, and code generation performance.
GFlowNet Training by Policy Gradients
Puhua Niu (Texas A&M University), Xiaoning Qian (Texas A&M University)
OptimizationDrug DiscoveryReinforcement LearningGraphBiomedical Data
🎯 What it does: Proposes to transform the training framework of Generative Flow Networks (GFlowNet) into a policy gradient-based reinforcement learning method, integrating joint learning of forward and backward policies;
Gibbs Sampling of Continuous Potentials on a Quantum Computer
Arsalan Motamedi (University of Waterloo), Pooya Ronagh (University of Waterloo)
Physics Related
🎯 What it does: A Gibbs sampling algorithm for continuous potential functions is proposed, using quantum Fourier transform and the Fokker-Planck equation under high-dimensional periodic potentials.
GiLOT: Interpreting Generative Language Models via Optimal Transport
Xuhong Li (Baidu Inc), Haoyi Xiong (Baidu Inc)
GenerationExplainability and InterpretabilityTransformerLarge Language ModelText
🎯 What it does: The GILOT method is proposed, which explains the importance of input tokens by calculating the optimal transport distance on the output distribution generated by large language models.
GistScore: Learning Better Representations for In-Context Example Selection with Gist Bottlenecks
Shivanshu Gupta (University of California), Ethan R. Elenberg (Permanence AI)
RetrievalRepresentation LearningTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Two new methods, Example Gisting and GistScore, are proposed for training retrievers and measuring the relevance of candidate examples to test inputs, aimed at improving few-shot learning (ICL) example selection in large language models.
GliDe with a CaPE: A Low-Hassle Method to Accelerate Speculative Decoding
Cunxiao Du (Singapore Management University), Yang You (National University of Singapore)
GenerationComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningTextFinance Related
🎯 What it does: This paper proposes two modules, GLIDE and CAPE, which utilize the KV cache of the target LLM to improve the accuracy of draft model predictions and significantly accelerate the speculative decoding process through confidence-adaptive expansion of draft outputs.
Global Reinforcement Learning : Beyond Linear and Convex Rewards via Submodular Semi-gradient Methods
Riccardo De Santi (ETH Zurich), Andreas Krause (ETH Zurich)
OptimizationReinforcement LearningTabular
🎯 What it does: A global reinforcement learning (GRL) framework is proposed, which shifts the definition of rewards from local state definitions to global trajectory definitions, and formulates the problem into a series of classical RL planning problems using submodular function semi-gradient methods.
GLoRe: When, Where, and How to Improve LLM Reasoning via Global and Local Refinements
Alexander Havrilla, Roberta Raileanu (Meta)
TransformerLarge Language ModelReinforcement LearningText
🎯 What it does: A phased self-improvement framework GLoRe is proposed, which uses trained reward models (ORM and SORM) to determine when, where, and how to improve the reasoning process of large language models, combining global and local improvement strategies to enhance the accuracy of the final answers.
GNNs Also Deserve Editing, and They Need It More Than Once
Shaochen Zhong, Xia Hu (Rice University)
ClassificationAnomaly DetectionGraph Neural NetworkGraph
🎯 What it does: This paper researches and implements a new graph neural network model editing method called SEED-GNN, which defines and implements the editing task on GNNs in a scalable and sequentially robust manner for the first time.
Going beyond Compositions, DDPMs Can Produce Zero-Shot Interpolations
Justin Deschenaux (École Polytechnique Fédérale de Lausanne), Volkan Cevher (École Polytechnique Fédérale de Lausanne)
GenerationData SynthesisDiffusion modelImage
🎯 What it does: Train a DDPM that only includes extreme labels (such as obvious smile, obvious no smile) and achieve zero-shot interpolation of intermediate expressions through multi-guidance sampling;
GPT-4V(ision) is a Generalist Web Agent, if Grounded
Boyuan Zheng (Ohio State University), Yu Su (Ohio State University)
TransformerLarge Language ModelVision Language ModelVision-Language-Action ModelMultimodality
🎯 What it does: This paper proposes SEEACT, a general web agent based on large multimodal models (such as GPT-4V) that can complete tasks on any website through visual perception and natural language instructions.
GPTSwarm: Language Agents as Optimizable Graphs
Mingchen Zhuge (King Abdullah University of Science and Technology), Jürgen Schmidhuber (Swiss AI Lab IDSIA)
OptimizationReinforcement Learning from Human FeedbackGraph Neural NetworkLarge Language ModelReinforcement LearningPrompt EngineeringText
🎯 What it does: The GPTSwarm framework is proposed, abstracting language agents as an optimizable computational graph, and implementing automatic optimization at two levels: nodes (prompts) and edges (communication between agents).
Gradient Compressed Sensing: A Query-Efficient Gradient Estimator for High-Dimensional Zeroth-Order Optimization
Ruizhong Qiu (University of Illinois Urbana-Champaign), Hanghang Tong (University of Illinois Urbana-Champaign)
OptimizationGraph
🎯 What it does: A new zero-order gradient estimator, GraCe, is proposed to significantly reduce query complexity for sparse gradient problems under high-dimensional constraints.
Gradient-based Visual Explanation for Transformer-based CLIP
Chenyang ZHAO, Antoni B. Chan (City University of Hong Kong)
RetrievalExplainability and InterpretabilityTransformerVision Language ModelImage
🎯 What it does: A gradient visualization explanation method for the CLIP visual-language model, called Grad-ECLIP, is proposed to generate high-quality heatmaps for image-text matching results.
Gradual Divergence for Seamless Adaptation: A Novel Domain Incremental Learning Method
Kishaan Jeeveswaran (Eindhoven University of Technology), Bahram Zonooz (Eindhoven University of Technology)
Domain AdaptationContrastive LearningImage
🎯 What it does: A three-stage domain incremental learning method named DARE is proposed, aimed at alleviating catastrophic forgetting caused by representation drift.
Graph Adversarial Diffusion Convolution
Songtao Liu (Pennsylvania State University), Dinghao Wu (Pennsylvania State University)
ClassificationOptimizationAdversarial AttackGraph Neural NetworkGraph
🎯 What it does: A min-max optimization framework for graph signal denoising is proposed, and based on this, a new graph diffusion convolution model called Graph Adversarial Diffusion Convolution (GADC) is constructed.
Graph As Point Set
Xiyuan Wang (Peking University), Muhan Zhang (Peking University)
Graph Neural NetworkTransformerAuto EncoderGraph
🎯 What it does: A graph-point set conversion method is proposed to transform graph structures into point sets, and a point set encoder (such as Point Set Transformer) is used to learn graph representations.
Graph Automorphism Group Equivariant Neural Networks
Edward Pearce-Crump (Imperial College London), William Knottenbelt (Imperial College London)
Graph Neural NetworkGraph
🎯 What it does: Constructed a linear equivariant layer for the automorphism group Aut(G) of a graph, providing a complete basis for all learnable Aut(G)-equivariant linear mappings, using the combinatorics of double-labeled graphs to generate the basis matrix.
Graph Distillation with Eigenbasis Matching
Yang Liu (Beijing University of Posts and Telecommunication), Chuan Shi (Beijing University of Posts and Telecommunication)
OptimizationKnowledge DistillationGraph Neural NetworkGraph
🎯 What it does: This paper proposes a graph distillation method called GDEM based on spectral basis matching, which generates synthetic graphs that allow GNNs to achieve the same performance on small graphs as on the original large graphs without relying on specific GNNs.
Graph External Attention Enhanced Transformer
Jianqing Liang (Shanxi University), Jiye Liang (Shanxi University)
Representation LearningGraph Neural NetworkTransformerGraph
🎯 What it does: A Graph External Attention (GEA) mechanism is designed and implemented, and based on this, a Graph External Attention Enhanced Transformer (GEAET) model is proposed to simultaneously capture local structures within graphs, global interactions between graphs, and external graph-related information to enhance graph representation learning effectiveness.
Graph Generation with Diffusion Mixture
Jaehyeong Jo (Korea Advanced Institute of Science and Technology), Sung Ju Hwang (Korea Advanced Institute of Science and Technology)
GenerationData SynthesisDrug DiscoveryGraph Neural NetworkDiffusion modelGraphBiomedical DataStochastic Differential Equation
🎯 What it does: A graph generation framework called GruM based on diffusion mixing is proposed, which directly captures the graph topology by learning the mixture of the final graph structure, achieving fast convergence and high-quality generation.
Graph Geometry-Preserving Autoencoders
Jungbin Lim (Seoul National University), Frank C. Park
Representation LearningRobotic IntelligenceGraph Neural NetworkAuto EncoderImageGraph
🎯 What it does: A graph geometry-based autoencoder (GGAE) is proposed, which combines the Laplacian operator of the similarity graph with Riemannian geometric distortion metrics to perform geometric preservation regularization directly at the encoder side.
Graph Mixup on Approximate Gromov–Wasserstein Geodesics
Zhichen Zeng (University of Illinois Urbana-Champaign), Hanghang Tong (University of Illinois Urbana-Champaign)
ClassificationOptimizationGraph Neural NetworkGraph
🎯 What it does: A graph mixing method based on Gromov-Wasserstein (GW) geodesics, called GEOMIX, is proposed to generate consistently high-quality augmented samples in graph space.
Graph Neural Network Explanations are Fragile
Jiate Li (Nanchang University), Binghui Wang (Illinois Institute of Technology)
Explainability and InterpretabilityAdversarial AttackGraph Neural NetworkGraph
🎯 What it does: This study investigates the robustness of GNN interpreters against adversarial perturbations in graph structures and proposes two attack methods based on loss and deduction.
Graph Neural Networks Use Graphs When They Shouldn't
Maya Bechler-Speicher (Tel Aviv University), Amir Globerson (Google Research)
Graph Neural NetworkGraph
🎯 What it does: This study systematically evaluates and demonstrates that graph neural networks (GNNs) overly rely on graph structure in tasks that are independent of graph structure, and explores the theoretical and practical effects of regularized graphs in combating overfitting.
Graph Neural Networks with a Distribution of Parametrized Graphs
See Hian Lee (Nanyang Technological University), Wee Peng Tay (Nanyang Technological University)
Graph Neural NetworkGraph
🎯 What it does: Proposes a graph neural network based on the EM framework, utilizing the distribution of graph parameters to train the model, thereby improving performance under uncertain graph structures.
Graph Neural PDE Solvers with Conservation and Similarity-Equivariance
Masanobu Horie (RICOS Co. Ltd.), NAOTO MITSUME
Graph Neural NetworkGraph
🎯 What it does: A PDE solver called FluxGNN is proposed, which combines graph neural networks and the finite volume method (FVM), achieving local conservation and similarity invariance.
Graph Neural Stochastic Diffusion for Estimating Uncertainty in Node Classification
Xixun Lin (Institute of Information Engineering Chinese Academy of Sciences), Yanan Cao (School of Cyber Security University of Chinese Academy of Sciences)
ClassificationAnomaly DetectionGraph Neural NetworkGraphStochastic Differential Equation
🎯 What it does: This paper studies the prediction uncertainty estimation of graph neural networks and proposes the GNSD framework based on the graph stochastic diffusion equation.
Graph Out-of-Distribution Detection Goes Neighborhood Shaping
Tianyi Bao (Shanghai Jiao Tong University), Junchi Yan (Shanghai Jiao Tong University)
Anomaly DetectionGraph Neural NetworkGraph
🎯 What it does: A node-level anomaly distribution detection method based on graph neighborhood structure, TopoOOD, is proposed, along with a new evaluation setting based on topological distribution.
Graph Positional and Structural Encoder
Semih Cantürk (Université de Montréal), Ladislav Rampášek (Isomorphic Labs)
Drug DiscoveryGraph Neural NetworkGraph
🎯 What it does: This paper proposes and trains a general Graph Position and Structure Encoder (GPSE) that can learn and reconstruct various Position and Structure Encodings (PSE) from graph structures in a self-supervised manner, which can then be directly used as feature enhancement for any GNN or graph Transformer model.
Graph Structure Extrapolation for Out-of-Distribution Generalization
Xiner Li (Texas A&M University), Shuiwang Ji (Texas A&M University)
Domain AdaptationGraph Neural NetworkAuto EncoderGraph
🎯 What it does: The research proposes an adaptive data augmentation method based on graph structure extrapolation, G-Splice, to enhance the OOD generalization ability of graph data.
Graph-based Forecasting with Missing Data through Spatiotemporal Downsampling
Ivan Marisca (Universitá della Svizzera italiana), Filippo Maria Bianchi (UiT the Arctic University of Norway)
Graph Neural NetworkGraphTime Series
🎯 What it does: A hierarchical spatiotemporal downsampling-based graph neural network framework is proposed, which directly predicts on synchronized spatiotemporal sequences with missing values without the need for prior imputation.
Graph-based Time Series Clustering for End-to-End Hierarchical Forecasting
Andrea Cini (Universita della Svizzera italiana), Cesare Alippi (Politecnico di Milano)
Graph Neural NetworkTime Series
🎯 What it does: A graph-based hierarchical clustering and prediction framework (HiGP) is proposed, unifying relational and hierarchical priors into a time series prediction network.
Graph-enhanced Large Language Models in Asynchronous Plan Reasoning
Fangru Lin (University of Oxford), Janet B. Pierrehumbert (University of Oxford)
TransformerLarge Language ModelPrompt EngineeringTextGraphBenchmark
🎯 What it does: Proposes an asynchronous planning reasoning task, constructs a high-quality benchmark AsyncHow, and evaluates the performance of LLMs.
Graph-Triggered Rising Bandits
Gianmarco Genalti (Politecnico di Milano), Alberto Maria Metelli (Politecnico di Milano)
OptimizationReinforcement Learning from Human FeedbackGraph Neural NetworkGraph
🎯 What it does: A new graph-triggered rising multi-armed bandit (GTRB) model is proposed, which extends the sleeping and activation bandit framework by introducing a graph structure to describe the interactions between arms.
Graph2Tac: Online Representation Learning of Formal Math Concepts
Lasse Blaauwbroek (Institut des Hautes Études Scientifiques), Vasily Pestun (IBM Research)
Representation LearningGraph Neural NetworkGraph
🎯 What it does: Rapid representation learning of new definitions and proof steps has been achieved in the Coq proof assistant through online learning techniques, proposing the Graph2Tac model combined with online k-NN for proof reasoning.