ICLR 2023 Papers — Page 6
International Conference on Learning Representations · 1573 papers
Exploring Low-Rank Property in Multiple Instance Learning for Whole Slide Image Classification
Jinxi Xiang (Tencent AI Lab), Jun Zhang (Tencent AI Lab)
ClassificationConvolutional Neural NetworkContrastive LearningImageMagnetic Resonance Imaging
🎯 What it does: A multi-instance learning method based on low-rank properties is proposed, which enhances feature embedding through low-rank constrained contrastive learning (LRC) and designs an Iterative Low-Rank Attention Module (ILRA-MIL) to efficiently model the inter-instance correlations of whole slide images, achieving whole slide image classification.
Exploring perceptual straightness in learned visual representations
Anne Harrington (Massachusetts Institute of Technology), William T. Freeman (Massachusetts Institute of Technology)
Object DetectionSegmentationRepresentation LearningAdversarial AttackConvolutional Neural NetworkTransformerContrastive LearningImageTime Series
🎯 What it does: The study quantifies the representation curvature (linearity) of deep visual models on time series, exploring the impact of architecture, training methods, and robustness on curvature, and verifies that linearization can enhance temporal stability.
Exploring Temporally Dynamic Data Augmentation for Video Recognition
Taeoh Kim (NAVER Cloud), Sangyoun Lee (NAVER Cloud)
RecognitionObject DetectionSegmentationConvolutional Neural NetworkTransformerVideo
🎯 What it does: This paper proposes and implements DynaAugment, a dynamic data augmentation framework designed for video recognition, capable of generating smooth and diverse augmentation magnitudes for each frame.
Exploring the Limits of Differentially Private Deep Learning with Group-wise Clipping
Jiyan He (University of Science and Technology of China), Jiang Bian (Microsoft Research)
Safty and PrivacyComputational EfficiencyTransformerSupervised Fine-TuningText
🎯 What it does: This paper researches and implements a differential privacy deep learning method based on group clipping, proposing layer-wise clipping that can run in parallel with backpropagation and device-level clipping aimed at large models, validating its effectiveness on various tasks.
Exploring The Role of Mean Teachers in Self-supervised Masked Auto-Encoders
Youngwan Lee (Korea Advanced Institute of Science and Technology), Sung Ju Hwang (Korea Advanced Institute of Science and Technology)
Object DetectionSegmentationKnowledge DistillationRepresentation LearningTransformerAuto EncoderImage
🎯 What it does: A self-supervised masked autoencoder combined with EMA teacher (RC-MAE) is proposed, and a theoretical and empirical analysis of the gradient effect of the teacher during the training process is conducted.
Exponential Generalization Bounds with Near-Optimal Rates for $L_q$-Stable Algorithms
Xiaotong Yuan, Ping Li (LinkedIn)
Optimization
🎯 What it does: This paper proposes a new near-optimal exponential generalization and risk bounding method for Lq-stable learning algorithms, and applies it to incomplete L0-ERM (such as Iterative Hard Thresholding, IHT) to obtain exponential upper bounds for sparse models.
Expressive Monotonic Neural Networks
Niklas Nolte (Massachusetts Institute of Technology), Mike Williams (Massachusetts Institute of Technology)
ClassificationOptimizationImageTabularPhysics Related
🎯 What it does: This paper proposes a method that applies L1-Lipschitz constraints to fully connected layers and adds a single residual connection, which can strictly ensure that the network is monotonic for any specified input subset and possesses provable robustness.
ExpressivE: A Spatio-Functional Embedding For Knowledge Graph Completion
Aleksandar Pavlović (TU Wien), Emanuel Sallinger (TU Wien)
Knowledge DistillationRepresentation LearningGraph Neural NetworkGraphBenchmark
🎯 What it does: A new knowledge graph embedding model called ExpressivE is proposed, which embeds entities as points and relationships as hyper-parallelograms in a virtual triple space to achieve richer reasoning patterns.
Extracting Robust Models with Uncertain Examples
Guanlin Li (Nanyang Technological University), Tianwei Zhang (Nanyang Technological University)
Knowledge DistillationAdversarial AttackConvolutional Neural NetworkImage
🎯 What it does: This paper proposes a model extraction method based on uncertain samples (UE) called Boundary Entropy Searching Thief (BEST), which can recover the clarity accuracy and robustness of the target model in a black-box Machine Learning as a Service (MLaaS) environment using only hard label queries.
Extreme Q-Learning: MaxEnt RL without Entropy
Divyansh Garg (Stanford University), Stefano Ermon (Stanford University)
Reinforcement LearningTabular
🎯 What it does: Proposes an extreme value theory-driven extreme Q-learning framework that directly estimates the soft Bellman maximum without the need for policy entropy.
Extremely Simple Activation Shaping for Out-of-Distribution Detection
Andrija Djurisic (ML Collective), Rosanne Liu (Google Research)
ClassificationAnomaly DetectionConvolutional Neural NetworkImage
🎯 What it does: This paper proposes an extremely simple post-training activation shaping method called ASH, which is used to significantly prune and slightly adjust network features during inference to enhance OOD detection performance.
f-DM: A Multi-stage Diffusion Model via Progressive Signal Transformation
Jiatao Gu (Apple), Joshua M. Susskind (Apple)
GenerationDiffusion modelImage
🎯 What it does: This paper proposes f-DM, a multi-stage diffusion model that can perform progressive transformations (such as downsampling, blurring, or VAE encoding) on signals during the diffusion process, achieving efficient and semantic image generation.
Factorized Fourier Neural Operators
Alasdair Tran (Australian National University), Cheng Soon Ong (Data61, CSIRO)
Computational EfficiencyPoint CloudPhysics Related
🎯 What it does: An improved Fourier Neural Operator (F-FNO) is proposed for efficient learning and simulation of various PDEs (Navier-Stokes, elasticity, aerodynamics, plastic forming).
Fair Attribute Completion on Graph with Missing Attributes
Dongliang Guo (University of Georgia), Sheng Li (University of Virginia)
Graph Neural NetworkAuto EncoderGenerative Adversarial NetworkGraph
🎯 What it does: The FairAC framework is proposed, which can complete missing attributes in graphs while simultaneously suppressing unfairness at both feature and topological levels.
FaiREE: fair classification with finite-sample and distribution-free guarantee
Puheng Li (Peking University), Linjun Zhang (Rutgers University)
ClassificationTabular
🎯 What it does: A post-processing algorithm FaiREE is proposed to achieve group fair classification under finite samples and no distribution assumptions.
FairGBM: Gradient Boosting with Fairness Constraints
André Cruz, Pedro Bizarro (Feedzai)
ClassificationOptimizationSupervised Fine-TuningTabularFinance Related
🎯 What it does: Proposes the FairGBM framework, which implements fairness constraints in gradient boosting trees (GBDT) through dual ascent learning.
Fairness and Accuracy under Domain Generalization
Thai-Hoang Pham (Ohio State University), Ping Zhang (Ohio State University)
Domain AdaptationRepresentation LearningGenerative Adversarial NetworkImageBiomedical DataElectronic Health Records
🎯 What it does: The research focuses on maintaining both model accuracy and fairness in domain generalization scenarios, and proposes a two-stage training framework based on density matching called FATDM.
Fairness-aware Contrastive Learning with Partially Annotated Sensitive Attributes
Fengda Zhang (Zhejiang University), Jun Xiao (Zhejiang University)
Representation LearningGenerative Adversarial NetworkContrastive LearningImage
🎯 What it does: In the absence of target labels and with only partial sensitive attribute annotations, a FairCL framework is proposed for unsupervised fair representation learning.
Fake It Until You Make It : Towards Accurate Near-Distribution Novelty Detection
Hossein Mirzaei (University of Amsterdam), Mohammad Hossein Rohban (Sharif University of Technology)
Anomaly DetectionTransformerSupervised Fine-TuningDiffusion modelGenerative Adversarial NetworkImageStochastic Differential Equation
🎯 What it does: For anomaly detection in near-distribution scenarios, this paper proposes using a non-adversarial diffusion model to generate approximate anomalous samples and fine-tuning a pre-trained feature extractor on the generated samples to enhance detection performance.
Fantastic Rewards and How to Tame Them: A Case Study on Reward Learning for Task-oriented Dialogue Systems
Yihao Feng (Salesforce Research), Huan Wang (Salesforce Research)
TransformerReinforcement LearningText
🎯 What it does: Two learning-to-rank based reward function learning methods, RewardNet and RewardMLE, are proposed, and the learned reward functions are combined with Gumbel-softmax policy gradients for training end-to-end task-oriented dialogue systems.
Fast and Precise: Adjusting Planning Horizon with Adaptive Subgoal Search
Michał Zawalski (University of Warsaw), Piotr Miłoś (Polish Academy of Sciences)
OptimizationComputational EfficiencyConvolutional Neural NetworkTransformerTabularBenchmark
🎯 What it does: The Adaptive Subgoal Search (AdaSubS) algorithm is proposed, which can adaptively adjust the time window for subgoal search based on state complexity, thereby planning more efficiently in complex reasoning tasks.
Fast Nonlinear Vector Quantile Regression
Aviv A. Rosenberg (Technion), Alexander Bronstein
OptimizationTabular
🎯 What it does: Proposed a scalable Vector Quantile Regression (VQR) and its nonlinear extension (NL-VQR) and Vector Monotonic Rearrangement (VMR) methods for estimating the conditional quantile functions of multidimensional targets.
Fast Sampling of Diffusion Models with Exponential Integrator
Qinsheng Zhang (Georgia Institute of Technology), Yongxin Chen (Georgia Institute of Technology)
GenerationData SynthesisComputational EfficiencyDiffusion modelImageStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: A fast sampling method based on the Exponential Integrator, DEIS, has been developed to accelerate the generation process of diffusion models.
Faster federated optimization under second-order similarity
Ahmed Khaled (Princeton University), Chi Jin (Princeton University)
OptimizationFederated LearningTabular
🎯 What it does: Two federated optimization algorithms, SVRP and Catalyzed SVRP, utilizing client sampling are designed to significantly reduce communication complexity.
Faster Gradient-Free Methods for Escaping Saddle Points
Hualin Zhang (Nanjing University of Information Science and Technology), Bin Gu (MBZUAI)
OptimizationHyperparameter SearchReinforcement Learning
🎯 What it does: The paper proposes two zeroth-order (gradient-free) methods based on Nesterov accelerated gradient descent, which can quickly escape saddle points and converge to ε-approximate second-order stationary points under the condition of only accessing function values.
Faster Last-iterate Convergence of Policy Optimization in Zero-Sum Markov Games
Shicong Cen (Carnegie Mellon University), Lin Xiao (Meta AI Research)
OptimizationReinforcement Learning
🎯 What it does: A single-loop, symmetric update strategy optimization algorithm (optimistic multiplicative weight update based on entropy regularization) is proposed to solve the quantum reaction equilibrium (QRE) of two-player zero-sum Markov games and converge to Nash equilibrium (NE).
FastFill: Efficient Compatible Model Update
Florian Jaeckle (University of Oxford), Hadi Pouransari (Apple)
RetrievalOptimizationImage
🎯 What it does: This paper proposes FastFill, a method compatible with model updates and online local backfilling, which can quickly enhance retrieval performance.
Feature Reconstruction From Outputs Can Mitigate Simplicity Bias in Neural Networks
Sravanti Addepalli (Google Research India), Prateek Jain (Google Research India)
Domain AdaptationConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: This paper proposes a Feature Reconstruction Regularizer (FRR) to alleviate the simplification bias of deep networks, thereby enhancing robustness in out-of-distribution (OOD) reasoning.
Feature selection and low test error in shallow low-rotation ReLU networks
Matus Telgarsky (University of Illinois)
ClassificationOptimizationTabular
🎯 What it does: This paper studies two-layer ReLU networks during the gradient flow (GF) and stochastic gradient descent (SGD) training processes, proving that under conditions of limited but sufficient width, the network can learn low-rotation features, thereby achieving significantly improved classification margins and low test errors.
FedDAR: Federated Domain-Aware Representation Learning
Aoxiao Zhong (Harvard University), Quanzheng Li (Massachusetts General Hospital)
Federated LearningRepresentation LearningBiomedical Data
🎯 What it does: A new method called FedDAR is proposed for domain-mixed data distribution in cross-site federated learning, achieving model personalization through learning shared encoders and domain-specific prediction heads.
Federated Learning as Variational Inference: A Scalable Expectation Propagation Approach
Han Guo (Carnegie Mellon University), Eric Xing
Federated LearningImageText
🎯 What it does: Proposed FedEP: a federated learning framework based on expectation propagation, which improves global posterior approximation using distributed inference;
Federated Learning from Small Datasets
Michael Kamp (University Hospital Essen), Jilles Vreeken (CISPA Helmholtz Center for Information Security)
Federated LearningSafty and PrivacyConvolutional Neural NetworkImageBiomedical DataMagnetic Resonance Imaging
🎯 What it does: A federated learning method called FEDDC is proposed, which combines 'daisy chaining' and aggregation to achieve effective training in scenarios where each client has very few samples.
Federated Nearest Neighbor Machine Translation
Yichao Du (University of Science and Technology of China), Enhong Chen (University of Science and Technology of China)
Federated LearningSafty and PrivacyComputational EfficiencyText
🎯 What it does: Proposes the FedNN framework, which uses single-round memory-based interaction to replace traditional FedAvg, achieving efficient training and knowledge sharing for federated NMT.
Federated Neural Bandits
Zhongxiang Dai (National University of Singapore), Patrick Jaillet (Massachusetts Institute of Technology)
Recommendation SystemFederated LearningTabular
🎯 What it does: The first Federated Neural Upper Confidence Bound (FN-UCB) algorithm is proposed, which utilizes a weighted combination of two UCBs to accelerate exploration and achieve better reward prediction.
FedExP: Speeding Up Federated Averaging via Extrapolation
Divyansh Jhunjhunwala (Carnegie Mellon University), Gauri Joshi (Carnegie Mellon University)
OptimizationFederated LearningComputational EfficiencyConvolutional Neural NetworkImage
🎯 What it does: The FedExP algorithm is proposed, which accelerates global aggregation by adaptively adjusting the server learning rate in Federated Averaging.
FedFA: Federated Feature Augmentation
Tianfei Zhou (ETH Zurich), Ender Konukoglu (ETH Zurich)
Domain AdaptationFederated LearningImageBiomedical DataMagnetic Resonance Imaging
🎯 What it does: Proposes FEDFA, which achieves data augmentation in federated learning through Gaussian sampling of feature statistics to alleviate feature shift among clients.
FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracy
Yan Sun (University of Sydney), Dacheng Tao (JD Explore Academy)
OptimizationFederated LearningConvolutional Neural NetworkImage
🎯 What it does: A novel federated learning algorithm called FedSpeed is proposed, which combines prox-correction and gradient perturbation to alleviate the issues of local inconsistency and client drift caused by data heterogeneity, achieving faster communication efficiency and higher generalization performance.
Few-shot Backdoor Attacks via Neural Tangent Kernels
Jonathan Hayase (University of Washington), Sewoong Oh (University of Washington)
OptimizationAdversarial AttackConvolutional Neural NetworkImage
🎯 What it does: Construct a small number of powerful backdoor trigger samples and achieve a high attack success rate by predicting the training process through the Neural Tangent Kernel (NTK).
Few-shot Cross-domain Image Generation via Inference-time Latent-code Learning
Arnab Kumar Mondal (Indian Institute of Technology Delhi), Prathosh AP (Indian Institute of Science Bengaluru)
GenerationData SynthesisGenerative Adversarial NetworkImage
🎯 What it does: A few-shot cross-domain image generation method is proposed for learning latent codes during the inference phase without the need to retrain the generator;
Few-Shot Domain Adaptation For End-to-End Communication
Jayaram Raghuram (University of Wisconsin - Madison), Suman Banerjee (University of Wisconsin - Madison)
Domain AdaptationAuto Encoder
🎯 What it does: A few-shot domain adaptation method for end-to-end communication systems is proposed, achieving rapid channel model transfer through the adaptation of Gaussian Mixture Networks (MDN) without retraining the encoder/decoder.
FIFA: Making Fairness More Generalizable in Classifiers Trained on Imbalanced Data
Zhun Deng (Columbia University), James Zou (Stanford University)
ClassificationConvolutional Neural NetworkImageTabular
🎯 What it does: The FIFA (Flexible Imbalance-Fairness-Aware) method is proposed, which combines logits-based loss to address the fairness generalization issue of over-parameterized models on imbalanced data.
FIGARO: Controllable Music Generation using Learned and Expert Features
Dimitri von Rütte (ETH Zurich), Thomas Hofmann (ETH Zurich)
GenerationTransformerAuto EncoderAudio
🎯 What it does: The paper proposes a controllable music generation model FIGARO based on Transformer, which combines high-level features manually extracted by experts with vectors obtained through self-supervised learning to construct a self-supervised training framework from description to sequence.
Filter-Recovery Network for Multi-Speaker Audio-Visual Speech Separation
Haoyue Cheng (Nanjing University), Limin Wang (Nanjing University)
RecognitionRestorationConvolutional Neural NetworkMultimodalityAudio
🎯 What it does: This paper proposes a method for audio-visual speech separation aimed at multi-speaker scenarios, utilizing BFRNet to achieve speech separation for 2 to 5 speakers within a unified model, and specifically addressing the issues of noise residue and information loss in the separation results.
FINDE: Neural Differential Equations for Finding and Preserving Invariant Quantities
Takashi Matsubara (Osaka University), Takaharu Yaguchi (Kobe University)
Time SeriesPhysics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: This paper proposes FINDE (First Integral-preserving Neural Differential Equation), a neural network framework that can automatically discover and preserve unknown conserved quantities of dynamical systems from data.
Finding Actual Descent Directions for Adversarial Training
Fabian Latorre (École Polytechnique Fédérale de Lausanne), Volkan Cevher (École Polytechnique Fédérale de Lausanne)
OptimizationAdversarial AttackConvolutional Neural NetworkImage
🎯 What it does: This study investigates the theoretical foundation of adversarial training, reveals the error in Corollary C.2 proposed by Madry et al., and introduces the Danskin descent direction (DDi) algorithm using multiple adversarial samples to obtain the true descent direction.
Finding the Global Semantic Representation in GAN through Fréchet Mean
Jaewoong Choi (Korea Institute for Advanced Study), Myungjoo Kang (Seoul National University)
GenerationData SynthesisRepresentation LearningGenerative Adversarial NetworkImage
🎯 What it does: This paper proposes an unsupervised global semantic benchmark (Fréchet Basis) for finding global semantic directions in the intermediate latent space of GANs, and solves it in two steps using Fréchet means on the Grassmann and SO groups.
First Steps Toward Understanding the Extrapolation of Nonlinear Models to Unseen Domains
Kefan Dong (Stanford University), Tengyu Ma (Stanford University)
Domain Adaptation
🎯 What it does: This paper studies the extrapolation ability of nonlinear models (in the form of f(x)=∑f_i(x_i)) under domain shift for unseen test distributions.
Fisher-Legendre (FishLeg) optimization of deep neural networks
Jezabel R Garcia (MediaTek Research), Guillaume Hennequin (University of Cambridge)
OptimizationMeta LearningAuto EncoderImage
🎯 What it does: This paper proposes an adaptive natural gradient optimizer called FishLeg, based on the Legendre-Fenchel conjugate, which can learn the inverse of the Fisher matrix online and directly use it for parameter updates.
FIT: A Metric for Model Sensitivity
Ben Zandonati (University of Cambridge), Tal Kopetz (CEVA Inc.)
OptimizationComputational EfficiencyTransformerImage
🎯 What it does: This paper proposes and implements the FIT (Fisher Information Trace) metric, which predicts the performance of quantized deep networks without the need for retraining and can quickly compute mixed-precision quantization configurations.
FiT: Parameter Efficient Few-shot Transfer Learning for Personalized and Federated Image Classification
Aliaksandra Shysheya (University of Cambridge), Richard E Turner
ClassificationFederated LearningMeta LearningConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: A method called FiLM Transfer (FIT) has been developed for achieving parameter-efficient image classification transfer learning in few-shot scenarios, and it can be applied in personalized models and federated learning settings.
FLIP: A Provable Defense Framework for Backdoor Mitigation in Federated Learning
Kaiyuan Zhang (Purdue University), Xiangyu Zhang (Purdue University)
Federated LearningAdversarial AttackImage
🎯 What it does: This paper proposes FLIP, a provable defense framework that combines trigger inversion and adversarial training in federated learning to reduce the success rate of backdoor attacks while maintaining the accuracy of clean samples.
Flow Annealed Importance Sampling Bootstrap
Laurence Illing Midgley, José Miguel Hernández-Lobato (University of Cambridge)
Flow-based Model
🎯 What it does: A Flow Annealed Importance Sampling Bootstrap (FAB) method is proposed, which combines α-divergence with α=2 and Annealed Importance Sampling to train Normalizing Flows for approximating multimodal target distributions.
Flow Matching for Generative Modeling
Yaron Lipman (Meta AI), Matthew Le (Meta AI)
GenerationData SynthesisDiffusion modelFlow-based ModelImageOrdinary Differential Equation
🎯 What it does: Proposes the Flow Matching framework for training continuous normalizing flows (CNF) without simulation, making it scalable to high-dimensional image generation.
Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow
Xingchao Liu (University of Texas at Austin), qiang liu
GenerationData SynthesisDomain AdaptationScore-based ModelFlow-based ModelRectified FlowNeural Radiance FieldImageOrdinary Differential Equation
🎯 What it does: A 'Rectified Flow' model based on ODE is proposed, which learns the transport mapping from one empirical distribution to another and can be used for both unsupervised generation and domain transfer.
FluidLab: A Differentiable Environment for Benchmarking Complex Fluid Manipulation
Zhou Xian (Carnegie Mellon University), Chuang Gan (Massachusetts Institute of Technology IBM Watson AI Lab)
Robotic IntelligenceReinforcement LearningBenchmarkPhysics Related
🎯 What it does: This paper presents the FluidLab simulation platform and the FluidEngine differential physics engine, which supports interactions between various fluids, solids, and gases, and provides 10 complex daily fluid manipulation tasks.
Fooling SHAP with Stealthily Biased Sampling
gabriel laberge, Foutse Khomh (Polytechnique Montreal)
OptimizationExplainability and InterpretabilityAdversarial AttackTabular
🎯 What it does: This paper proposes an attack method called Fool SHAP, which manipulates SHAP explanations through covertly weighted sampling of background data points without altering the model.
Formal Mathematics Statement Curriculum Learning
Stanislas Polu (OpenAI), Ilya Sutskever (OpenAI)
TransformerLarge Language ModelReinforcement LearningText
🎯 What it does: An Expert Iteration framework is proposed for training language models to complete theorem proofs in Lean formalized mathematics, achieving self-improvement through alternating search and learning.
Forward Super-Resolution: How Can GANs Learn Hierarchical Generative Models for Real-World Distributions
Zeyuan Allen-Zhu (Allen-Zhu Research), Yuanzhi Li (Mohamed bin Zayed University of Artificial Intelligence)
RestorationGenerationSuper ResolutionGenerative Adversarial NetworkImage
🎯 What it does: This paper proposes and proves that in hierarchical generative models with forward super-resolution structures, using Stochastic Gradient Descent-Ascent (SGDA) to train Generative Adversarial Networks (GANs) can efficiently learn the target distribution within polynomial time and sample complexity.
FoSR: First-order spectral rewiring for addressing oversquashing in GNNs
Kedar Karhadkar (University of California Los Angeles), Guido Montufar
ClassificationGraph Neural NetworkGraph
🎯 What it does: This paper proposes a First-Order Spectral Rewiring (FoSR) algorithm based on spectral augmentation, and implements reconnection of GNNs through Relational Graph Convolutional Networks (R-GCN), which reduces oversquashing and avoids oversmoothing.
Free Lunch for Domain Adversarial Training: Environment Label Smoothing
YiFan Zhang, Tieniu Tan (Alibaba Group)
ClassificationDomain AdaptationImageTextTime Series
🎯 What it does: In Domain Adversarial Training (DAT), Environment Label Smoothing (ELS) is introduced to reduce the overconfidence of the discriminator and alleviate environmental label noise, enhancing training stability and convergence speed;
FreeMatch: Self-adaptive Thresholding for Semi-supervised Learning
Yidong Wang (Microsoft Research), Xing Xie (Microsoft Research)
ClassificationData-Centric LearningImage
🎯 What it does: Proposes the FreeMatch framework, which combines adaptive thresholds and category fairness regularization to achieve more efficient semi-supervised learning.
From $t$-SNE to UMAP with contrastive learning
Sebastian Damrich (Heidelberg University), Dmitry Kobak (University of Tübingen)
OptimizationRepresentation LearningContrastive LearningImageBiomedical Data
🎯 What it does: This study investigates the essential relationship between t-SNE and UMAP within the contrastive learning framework, and proposes a tunable negative sampling spectral method to adjust the clustering tightness and global structure of embeddings by correlating Noise Contrastive Estimation (NCE) with Negative Sampling (NEG).
From Play to Policy: Conditional Behavior Generation from Uncurated Robot Data
Zichen Jeff Cui (New York University), Lerrel Pinto (New York University)
GenerationAutonomous DrivingRobotic IntelligenceTransformerReinforcement LearningContrastive LearningVideoMultimodality
🎯 What it does: This paper proposes Conditional Behavior Transformers (C-BeT), a multimodal behavior generation model based on Transformers that can learn task-specific control strategies from unlabeled play data.
Function-Consistent Feature Distillation
Dongyang Liu (Chinese Academy of Sciences), Xilin CHEN
ClassificationObject DetectionKnowledge DistillationConvolutional Neural NetworkImage
🎯 What it does: The Function-Consistent Feature Distillation (FCFD) method is proposed, which enhances the performance of the student model by using the functional consistency of intermediate features between the teacher and student in the later part of the network as supervision.
Function-space regularized Rényi divergences
Jeremiah Birrell (University of Massachusetts), Markos Katsoulakis
Generative Adversarial NetworkBiomedical Data
🎯 What it does: A new family of regularized Rényi divergences is proposed, which is parameterized not only by the order α but also by the variational function space. These new objects are defined by minimizing the convolution of the standard Rényi divergence with an integral probability metric (IPM) related to the chosen function space.
Fundamental Limits in Formal Verification of Message-Passing Neural Networks
Marco Sälzer (University of Kassel), Martin Lange (University of Kassel)
Graph Neural NetworkGraph
🎯 What it does: This paper studies the formal verification problem of Message Passing Neural Networks (MPNNs) and proves that when the input graph size is infinite, the node degree is non-trivial, and the labels are sufficiently expressive, the reachability and adversarial robustness of graph classifiers are undecidable; however, when the input graph degree is finite, these two safety properties for node classifiers are decidable.
Fundamental limits on the robustness of image classifiers
Zheng Dai (Massachusetts Institute of Technology), David Gifford
ClassificationAdversarial AttackImage
🎯 What it does: It is proven that there exists a fundamental robustness limit for image classifiers in any discrete image space, specifically providing a non-robust upper bound regarding the size of p-norm perturbations, and demonstrating that this upper bound is asymptotically optimal in dimension N; further analysis of the impact of bit depth on robustness shows that higher bit depth leads to weaker robustness; finally, the semantic information carried by fragile features is discussed.
FunkNN: Neural Interpolation for Functional Generation
AmirEhsan Khorashadizadeh (University of Basel), Ivan Dokmanić (University of Illinois)
RestorationGenerationSuper ResolutionConvolutional Neural NetworkGenerative Adversarial NetworkImage
🎯 What it does: A convolution-based continuous super-resolution network called FunkNN is proposed, which combines a discrete generator to generate continuous images at any scale and is differentiable.
Fuzzy Alignments in Directed Acyclic Graph for Non-Autoregressive Machine Translation
Zhengrui Ma (Institute of Computing Technology, Chinese Academy of Sciences), Yang Feng (Institute of Computing Technology, Chinese Academy of Sciences)
GenerationOptimizationTransformerTextMultimodality
🎯 What it does: A target function based on fuzzy alignment (n-gram expected matching) is proposed to train a non-autoregressive translation model with a Directed Acyclic Graph (DAG), thereby better addressing multimodal issues.
GAIN: On the Generalization of Instructional Action Understanding
Junlong Li (Tsinghua University), Jiwen Lu (Tsinghua University)
Domain AdaptationRecurrent Neural NetworkVideo
🎯 What it does: This paper studies the generalization ability of instruction action understanding models in out-of-domain (OOD) scenarios by constructing the GAIN dataset.
GAMR: A Guided Attention Model for (visual) Reasoning
Mohit Vaishnav (Universite de Toulouse), Thomas Serre (Universite de Toulouse)
Convolutional Neural NetworkRecurrent Neural NetworkImage
🎯 What it does: A guided attention module named GAMR is proposed, which learns to dynamically locate and store key information in visual reasoning tasks through a serialized attention movement and memory mechanism.
gDDIM: Generalized denoising diffusion implicit models
Qinsheng Zhang (Georgia Institute of Technology), Yongxin Chen (Georgia Institute of Technology)
GenerationData SynthesisComputational EfficiencyDiffusion modelImageStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: The paper proposes a general acceleration method for diffusion models called gDDIM, which can achieve nearly perfect sampling in a limited number of steps.
GEASS: Neural causal feature selection for high-dimensional biological data
Mingze Dong (Yale University), Yuval Kluger (Yale University)
Graph Neural NetworkTime SeriesBiomedical Data
🎯 What it does: Proposes the GEASS method, which utilizes a single neural network for sparse Granger causal feature selection in high-dimensional spatiotemporal biological data.
GeneFace: Generalized and High-Fidelity Audio-Driven 3D Talking Face Synthesis
Zhenhui Ye (Zhejiang University), Zhou Zhao (Zhejiang University)
GenerationData SynthesisDomain AdaptationNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkVideoAudio
🎯 What it does: This paper presents a three-stage NeRF-driven universal high-fidelity audio-driven 3D talking face generation system called GeneFace.
General Neural Gauge Fields
Fangneng Zhan (Max Planck Institute for Informatics), Christian Theobalt (Max Planck Institute for Informatics)
Data SynthesisOptimizationNeural Radiance FieldPoint Cloud
🎯 What it does: A general Neural Gauge Fields framework is proposed, which learns a learnable coordinate transformation from 3D coordinates to another measurement system (such as a 2D plane or hash codebook) and builds a neural field based on this.
Generalization and Estimation Error Bounds for Model-based Neural Networks
Avner Shultzman (Weizmann Institute of Science), Yonina C. Eldar (Weizmann Institute of Science)
🎯 What it does: This paper derives the upper bounds of generalization error and estimation error for learning-based ISTA and ADMM networks using Rademacher complexity and local Rademacher complexity theory.
Generalization Bounds for Federated Learning: Fast Rates, Unparticipating Clients and Unbounded Losses
Xiaolin Hu (Renmin University of China), Yong Liu (Renmin University of China)
Federated LearningTabular
🎯 What it does: Under a two-layer distribution framework, a generalization error theory for federated learning is proposed, along with a high-probability fast convergence rate for participating and non-participating clients.
Generalize Learned Heuristics to Solve Large-scale Vehicle Routing Problems in Real-time
Qingchun Hou (Alibaba Group), Yuming Deng (Alibaba Group)
OptimizationTransformerReinforcement LearningTabular
🎯 What it does: This paper proposes a two-stage segmentation method (TAM) that generates sub-path sequences instead of node sequences. It utilizes a trained data-driven model to decompose large-scale vehicle routing problems into several small-scale TSPs, which are then solved in parallel using traditional or learned solvers, achieving completion in seconds for VRPs with over 5000 nodes.
Generalized Precision Matrix for Scalable Estimation of Nonparametric Markov Networks
Yujia Zheng (Carnegie Mellon University), Kun Zhang (Carnegie Mellon University)
Score-based ModelTabular
🎯 What it does: A scalable regularized score matching framework is designed and implemented, utilizing the Generalized Precision Matrix (GPM) for model-free estimation of Markov network structures with continuous, discrete, and mixed-type variables.
Generalizing and Decoupling Neural Collapse via Hyperspherical Uniformity Gap
Weiyang Liu (Max Planck Institute for Intelligent Systems), Bernhard Schölkopf (Max Planck Institute for Intelligent Systems)
ClassificationOptimizationConvolutional Neural NetworkSupervised Fine-TuningImageText
🎯 What it does: The Generalized Neural Collapse (GNC) hypothesis is proposed, and a decoupled classification loss function is designed based on the Hyper-Spherical Uniformity Gap (HUG) for training deep networks.
Generate rather than Retrieve: Large Language Models are Strong Context Generators
Wenhao Yu (University of Notre Dame), Meng Jiang (Microsoft Cognitive Service Research)
GenerationRetrievalTransformerLarge Language ModelPrompt EngineeringTextRetrieval-Augmented Generation
🎯 What it does: A generate-then-read (GENREAD) framework is proposed, which directly uses large language models to generate context documents related to questions, and then a reading model provides answers based on these documents, replacing the traditional retrieve-read pipeline.
Generating Diverse Cooperative Agents by Learning Incompatible Policies
Rujikorn Charakorn (Vistec), Nat Dilokthanakul (KMITL)
Robotic IntelligenceReinforcement Learning
🎯 What it does: A method for generating diverse cooperative agents by learning incompatible strategies is proposed, aimed at enhancing the cooperation ability and diversity of the agents.
Generating Sequences by Learning to Self-Correct
Sean Welleck (Allen Institute for Artificial Intelligence), Yejin Choi (Allen Institute for Artificial Intelligence)
GenerationReinforcement LearningTextSequential
🎯 What it does: The SELF-CORRECTION framework is proposed, which splits the generation task into a basic generator and a self-correction module, achieving multi-round iterative improvements.
Generative Augmented Flow Networks
Ling Pan (Mila Quebec AI Institute), Yoshua Bengio (Mila Quebec AI Institute)
GenerationReinforcement LearningFlow-based ModelGraph
🎯 What it does: A Generative Augmented Flow Network (GAFlowNet) framework is proposed, which introduces intermediate rewards (through edge, state, or joint methods) into the traditional GFlowNet to enhance exploration efficiency and generation diversity in sparse reward environments.
Generative Modeling Helps Weak Supervision (and Vice Versa)
Benedikt Boecking (Carnegie Mellon University), Artur Dubrawski (Carnegie Mellon University)
ClassificationGenerationData SynthesisGenerative Adversarial NetworkImage
🎯 What it does: This paper proposes a Weakly Supervised Generative Adversarial Network (WSGAN) that combines programmatic weak supervision with GANs, utilizing weak labels and generative models to mutually calibrate and enhance each other.
Generative Modelling with Inverse Heat Dissipation
Severi Rissanen (Aalto University), Arno Solin (Aalto University)
GenerationData SynthesisConvolutional Neural NetworkDiffusion modelImage
🎯 What it does: A generative model based on the Inverse Heat Dissipation Model (IHDM) is designed and implemented, achieving multi-scale image generation through the inverse process of the heat equation.
Geometrically regularized autoencoders for non-Euclidean data
Cheongjae Jang (Hanyang University), Frank C. Park
Anomaly DetectionRepresentation LearningAuto EncoderTextPoint Cloud
🎯 What it does: This paper proposes geometric regularized autoencoders (Geometric DAE and Geometric RCAE), constructing coordinate-invariant regularization terms on Riemannian manifolds, deriving a theory applicable for estimating the logarithmic probability density gradient (geometric score), and applying it to sampling, clustering, noise filtering, and representation learning.
GFlowNets and variational inference
Nikolay Malkin (Mila), Yoshua Bengio (Mila)
GenerationDrug DiscoveryReinforcement LearningGraph
🎯 What it does: This paper connects two types of probabilistic algorithms: Variational Inference (VI) and Generative Flow Networks (GFlowNet). It proves that they are equivalent in terms of gradient expectations under certain conditions and experimentally compares the performance of the two methods in discrete structure generation tasks.
Git Re-Basin: Merging Models modulo Permutation Symmetries
Samuel Ainsworth, Siddhartha Srinivasa (Paul G. Allen School of Computer Science and Engineering University of Washington)
OptimizationFederated LearningSafty and PrivacyConvolutional Neural NetworkImage
🎯 What it does: This paper proposes aligning two independently trained models by finding permutations of hidden layer units, thereby achieving linear mode connectivity (LMC) in weight space and enabling model merging.
GLM-130B: An Open Bilingual Pre-trained Model
Aohan Zeng (Tsinghua University), Jie Tang (Tsinghua University)
TransformerLarge Language ModelText
🎯 What it does: Trained and released a 130 billion parameter bilingual (English-Chinese) large language model GLM-130B, providing complete code, training logs, and quantization implementation.
Global Explainability of GNNs via Logic Combination of Learned Concepts
Steve Azzolin (University of Trento), Andrea Passerini (University of Trento)
Explainability and InterpretabilityGraph Neural NetworkGraph
🎯 What it does: This paper proposes GLGExplainer, which can generate global explanations through logical combinations of local explanations, thereby revealing the overall decision-making process of Graph Neural Networks (GNNs).
Globally Optimal Training of Neural Networks with Threshold Activation Functions
Tolga Ergen (Stanford University), Mert Pilanci (Stanford University)
OptimizationImageTabular
🎯 What it does: Research on the global optimal training of threshold activated neural networks, constructing an equivalent convex optimization problem.
GNNDelete: A General Strategy for Unlearning in Graph Neural Networks
Jiali Cheng (University of Massachusetts Lowell), Marinka Zitnik (Harvard University)
Graph Neural NetworkGraph
🎯 What it does: A general graph neural network deletion operator GNNDELETE is proposed, which can effectively delete nodes, edges, and node features without retraining the model.
GNNInterpreter: A Probabilistic Generative Model-Level Explanation for Graph Neural Networks
Xiaoqi Wang (Ohio State University), Han Wei Shen
ClassificationExplainability and InterpretabilityGraph Neural NetworkGraph
🎯 What it does: This paper proposes GNNInterpreter, which generates a probabilistic explanation graph distribution to reveal the high-level decision-making process of GNN in graph classification tasks.
GoBigger: A Scalable Platform for Cooperative-Competitive Multi-Agent Interactive Simulation
Ming Zhang (SenseTime Research), Yu Liu (Shanghai AI Laboratory)
Robotic IntelligenceReinforcement Learning
🎯 What it does: Designed and implemented GoBigger, a scalable multi-agent interactive simulation platform that supports M×N teams and can simulate the cooperation and competition of multiple teams within the same game.
GOGGLE: Generative Modelling for Tabular Data by Learning Relational Structure
Tennison Liu (University of Cambridge), Mihaela van der Schaar (University of Cambridge)
GenerationData SynthesisGraph Neural NetworkAuto EncoderGenerative Adversarial NetworkTabular
🎯 What it does: GOGGLE is proposed, an end-to-end framework that can jointly learn the relational structure and generative model of tabular data.
GOOD: Exploring geometric cues for detecting objects in an open world
Haiwen Huang (University of Tübingen), Dan Zhang (Bosch)
Object DetectionAuto EncoderImage
🎯 What it does: A pseudo-labeling method based on geometric cues (depth, normals) is proposed to train an open-world category-free object detector called GOOD.
GPViT: A High Resolution Non-Hierarchical Vision Transformer with Group Propagation
Chenhongyi Yang (University of Edinburgh), Xiaolong Wang (UC San Diego)
ClassificationObject DetectionSegmentationTransformerImage
🎯 What it does: A non-hierarchical high-resolution visual Transformer (GPViT) is proposed, which achieves low computational cost global information exchange through the Group Propagation Block while maintaining high resolution of image features.
GRACE-C: Generalized Rate Agnostic Causal Estimation via Constraints
Mohammadsajad Abavisani (Georgia Institute of Technology), Sergey Plis
GraphTime SeriesMagnetic Resonance Imaging
🎯 What it does: A new constraint satisfaction-based algorithm sRASL is proposed for recovering causal structures from under-sampled time series data at unknown sampling rates;
Gradient Boosting Performs Gaussian Process Inference
Aleksei Ustimenko (ShareChat), Liudmila Prokhorenkova (Yandex Research)
Tabular
🎯 What it does: View gradient boosting trees (GBDT) as kernel ridge regression in an implicit RKHS, prove its convergence to the posterior mean of Gaussian processes, and propose a KGB method based on random tree sampling and gradient boosting for sampling from the posterior distribution and obtaining knowledge uncertainty estimates.