NeurIPS 2024 Papers with AI Summaries
Conference on Neural Information Processing Systems · 4035 papers
→ NeurIPS 2024 papers with code (1874)
Each paper below shows an AI-generated one-line summary. Get the full 6-part summary (innovation, method, data, results, limitations) and search all 4035 NeurIPS 2024 papers by keyword, author or institution —
free trial on arXivSub.
(FL)$^2$: Overcoming Few Labels in Federated Semi-Supervised Learning
Seungjoo Lee (Korea Advanced Institute of Science and Technology), Sung-Ju Lee (Korea Advanced Institute of Science and Technology)
Federated LearningSupervised Fine-TuningImage
🎯 What it does: In the scenario of federated semi-supervised learning where labels are only on the server, a new federated training framework (FL²) is proposed, which significantly reduces confirmation bias and improves model performance through adaptive thresholds, sharpness consistency regularization, and learning state-aware aggregation.
$\beta$-DPO: Direct Preference Optimization with Dynamic $\beta$
Junkang Wu (University of Science and Technology of China), Xiangnan He (University of Science and Technology of China)
OptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Dynamic β calibration and data filtering for DPO are conducted, proposing the β-DPO framework to enhance the alignment of LLM with human feedback.
$\boldsymbol{\mu}\mathbf{P^2}$: Effective Sharpness Aware Minimization Requires Layerwise Perturbation Scaling
Moritz Haas (University of Tübingen), Leena Chennuru Vankadara (Amazon)
OptimizationConvolutional Neural NetworkTransformerImage
🎯 What it does: This study investigates the learning dynamics of Sharpness Aware Minimization (SAM) in infinitely wide neural networks and proposes a maximum update and perturbation parameterization (µP2) that achieves layer-wise effective perturbations while maintaining feature learning.
$\epsilon$-Softmax: Approximating One-Hot Vectors for Mitigating Label Noise
Jialiang Wang (Harbin Institute of Technology), Xianming Liu (Harbin Institute of Technology)
ClassificationRecognitionConvolutional Neural NetworkImage
🎯 What it does: A simple ε-softmax layer is designed in deep learning to approximate one-hot vectors, thereby alleviating label noise and enhancing model robustness.
$\text{Di}^2\text{Pose}$: Discrete Diffusion Model for Occluded 3D Human Pose Estimation
Weiquan Wang (Zhejiang University), Long Chen (Hong Kong University of Science and Technology)
Pose EstimationTransformerDiffusion modelImage
🎯 What it does: A 3D human pose estimation framework called Di Pose 2 is proposed, which quantizes the pose into discrete tokens and then uses a discrete diffusion model for conditional inference.
$\text{ID}^3$: Identity-Preserving-yet-Diversified Diffusion Models for Synthetic Face Recognition
Jianqing Xu (Tencent Youtu Lab), Bryan Hooi (National University of Singapore)
RecognitionGenerationData SynthesisDiffusion modelImage
🎯 What it does: A conditional diffusion model named ID3 and its sampling algorithm have been designed and implemented to automatically generate diverse, identity-preserving synthetic face data for training facial recognition models.
$\textit{Bifr\"ost}$: 3D-Aware Image Compositing with Language Instructions
Lingxiao Li (Chinese University of Hong Kong), Xiangyu Yue (Chinese University of Hong Kong)
GenerationData SynthesisDepth EstimationTransformerLarge Language ModelSupervised Fine-TuningDiffusion modelImageVideoTextMultimodality
🎯 What it does: This paper proposes the Bifröst framework, which achieves 3D perception image synthesis based on language instructions, precisely inserting reference objects into the background image while maintaining lighting, occlusion, and other three-dimensional spatial relationships.
$\textit{NeuroPath}$: A Neural Pathway Transformer for Joining the Dots of Human Connectomes
Ziquan Wei (University of North Carolina at Chapel Hill), Guorong Wu (University of North Carolina at Chapel Hill)
ClassificationRecognitionAnomaly DetectionGraph Neural NetworkTransformerMultimodalityGraphBiomedical DataAlzheimer's Disease
🎯 What it does: A graph neural network based on the NeuroPath Transformer is proposed, which uses multimodal structure-function connectivity (SC-FC) coupling to learn brain network features and perform neural activity recognition and cognitive disease diagnosis.
$\textit{Read-ME}$: Refactorizing LLMs as Router-Decoupled Mixture of Experts with System Co-Design
Ruisi Cai (University of Texas at Austin), Zhangyang Wang (University of Texas at Austin)
OptimizationComputational EfficiencyKnowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningMixture of ExpertsText
🎯 What it does: Transforming a pre-trained dense large language model into a smaller sparse expert network (Mixture-of-Experts)
$\textit{Trans-LoRA}$: towards data-free Transferable Parameter Efficient Finetuning
Runqian Wang (Massachusetts Institute of Technology), Leonid Karlinsky (IBM)
Data SynthesisKnowledge DistillationTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: A near-data-free LoRA model transfer method called Trans-LoRA is proposed, which can transfer LoRA to new base models or other PEFT methods without accessing the original task data.
$C^2M^3$: Cycle-Consistent Multi-Model Merging
Donato Crisostomi (Sapienza University of Rome), Emanuele Rodolà (Sapienza University of Rome)
OptimizationConvolutional Neural NetworkImage
🎯 What it does: A data-independent weight matching and merging method is proposed, achieving multi-model periodic consistency merging through global optimization of neuron arrangement.
$SE(3)$ Equivariant Ray Embeddings for Implicit Multi-View Depth Estimation
Yinshuang Xu (University of Pennsylvania), Vitor Campagnolo Guizilini
Depth EstimationTransformerPoint Cloud
🎯 What it does: A SE(3) equivariant multi-view depth estimation model is proposed, embedding geometric entities (rays) into Perceiver IO, and achieving global equivariance through equivariant positional encoding and attention mechanisms, enabling depth prediction from new viewpoints without relying on explicit geometric constraints.
2D-OOB: Attributing Data Contribution Through Joint Valuation Framework
Yifan Sun (University of Illinois Urbana-Champaign), Yongchan Kwon (Columbia University)
Explainability and InterpretabilityComputational EfficiencyData-Centric LearningImageTabular
🎯 What it does: Jointly estimate data contributions and assign scores to each feature cell of every sample.
2DQuant: Low-bit Post-Training Quantization for Image Super-Resolution
Kai Liu (Shanghai Jiao Tong University), Yulun Zhang (Shanghai Jiao Tong University)
RestorationSuper ResolutionKnowledge DistillationTransformerImage
🎯 What it does: This paper proposes a low-bit post-training quantization method for image super-resolution models based on the Transformer architecture—2DQuant.
3-in-1: 2D Rotary Adaptation for Efficient Finetuning, Efficient Batching and Composability
Baohao Liao (University of Amsterdam), Christof Monz (University of Amsterdam)
OptimizationComputational EfficiencyTransformerSupervised Fine-TuningTextBenchmark
🎯 What it does: This paper proposes a parameter-efficient fine-tuning method based on two-dimensional rotation, called RoAd, and evaluates its performance on tasks such as GLUE, commonsense reasoning, and arithmetic reasoning, demonstrating its advantages in batch processing and composability.
3D Equivariant Pose Regression via Direct Wigner-D Harmonics Prediction
Jongmin Lee (LG AI Research), Minsu Cho (Pohang University of Science and Technology)
Pose EstimationConvolutional Neural NetworkPoint Cloud
🎯 What it does: Directly predicting 3D poses in the frequency domain using Wigner-D coefficients, avoiding the discontinuity issues of traditional Euler angles and quaternions.
3D Focusing-and-Matching Network for Multi-Instance Point Cloud Registration
Liyuan Zhang (Northwestern Polytechnical University), Yuchao Dai (Northwestern Polytechnical University)
Object DetectionSegmentationConvolutional Neural NetworkPoint Cloud
🎯 What it does: A 3D Focusing and Matching Network (3DFMNet) is proposed, which first locates the instance center of the model point cloud in the scene, and then performs point cloud registration for each instance proposal, achieving the multi-instance point cloud registration task.
3D Gaussian Rendering Can Be Sparser: Efficient Rendering via Learned Fragment Pruning
Zhifan Ye (Georgia Institute of Technology), Yingyan Celine Lin
Computational EfficiencyGaussian SplattingImageVideo
🎯 What it does: This paper addresses the low rendering efficiency of 3D Gaussian splatting on edge devices by proposing a Fragment Pruning method based on adaptive truncation thresholds for each Gaussian;
3D Gaussian Splatting as Markov Chain Monte Carlo
Shakiba Kheradmand (University of British Columbia), Kwang Moo Yi (University of British Columbia)
Gaussian SplattingPoint CloudStochastic Differential Equation
🎯 What it does: Treats the training of 3D Gaussian Splatting as a Markov Chain Monte Carlo (MCMC) process sampling from a latent distribution, and updates using Stochastic Gradient Langevin Dynamics (SGLD), eliminating the reliance on manual heuristics such as cloning/splitting/cropping;
3D Structure Prediction of Atomic Systems with Flow-based Direct Preference Optimization
Rui Jiao (Tsinghua University), Yang Liu (Tsinghua University)
OptimizationProtein Structure PredictionFlow-based ModelPoint Cloud
🎯 What it does: The FlowDPO framework is proposed, which combines flow matching models and Direct Preference Optimization (DPO) to automatically construct preference datasets to improve the quality of 3D atomic structure predictions.
3DET-Mamba: Causal Sequence Modelling for End-to-End 3D Object Detection
Mingsheng Li (Fudan University), Tao Chen (Fudan University)
Object DetectionTransformerPoint Cloud
🎯 What it does: Proposed the 3DET-Mamba model for end-to-end 3D object detection.
3DGS-Enhancer: Enhancing Unbounded 3D Gaussian Splatting with View-consistent 2D Diffusion Priors
Xi Liu (Clemson University), Siyu Huang (Clemson University)
RestorationGenerationDiffusion modelGaussian SplattingImageVideo
🎯 What it does: This paper proposes the 3DGS-Enhancer system, which enhances low-quality views generated by 3D Gaussian Smoothing (3DGS) through a video diffusion model, followed by fine-tuning of 3DGS, significantly improving the rendering quality of undefined scenes from sparse viewpoints.
4-bit Shampoo for Memory-Efficient Network Training
Sike Wang (Beijing Normal University), Hua Huang (Beijing Normal University)
OptimizationConvolutional Neural NetworkTransformerImage
🎯 What it does: Proposes 4-bit Shampoo, which compresses the state of second-order optimizers to 4 bits for memory efficiency while maintaining training performance close to the 32-bit version.
4+3 Phases of Compute-Optimal Neural Scaling Laws
Elliot Paquette (McGill University), Jeffrey Pennington (Google DeepMind)
Tabular
🎯 What it does: Analyzed a three-parameter random feature model (PLRF), derived a deterministic expression for training dynamics, and based on this, obtained the optimal scaling law for computation;
4D Gaussian Splatting in the Wild with Uncertainty-Aware Regularization
Mijeong Kim (Seoul National University), Bohyung Han (Seoul National University)
RestorationData SynthesisDepth EstimationDiffusion modelGaussian SplattingOptical FlowVideo
🎯 What it does: This paper proposes a dynamic scene reconstruction method for casually shot monocular videos—Uncertainty-Aware 4D Gaussian Expansion (UA-4DGS). It combines uncertainty estimation for adaptive regularization during the training process, balancing the reconstruction quality of training images with the performance of novel view synthesis.
4Diffusion: Multi-view Video Diffusion Model for 4D Generation
Haiyu Zhang (Beihang University), Yu Qiao (Shanghai AI Laboratory)
GenerationData SynthesisDiffusion modelNeural Radiance FieldVideoMesh
🎯 What it does: A 4Diffusion framework is proposed, which generates spatial-temporal consistent 4D content from monocular videos.
4M-21: An Any-to-Any Vision Model for Tens of Tasks and Modalities
Roman Bachmann (Swiss Federal Institute of Technology Lausanne), Amir Zamir (Apple)
GenerationRetrievalTransformerVision Language ModelTextMultimodality
🎯 What it does: Trained a multimodal model 4M-21 that can take arbitrary inputs and produce arbitrary outputs, supporting interactions and generation across 21 different modalities.
4Real: Towards Photorealistic 4D Scene Generation via Video Diffusion Models
Heng Yu (Snap Inc), Hsin-Ying Lee (Snap Inc)
GenerationData SynthesisDiffusion modelGaussian SplattingVideoText
🎯 What it does: A novel text-to-4D scene generation pipeline named 4Real is proposed, achieving near-realistic dynamic scene generation.
A Bayesian Approach for Personalized Federated Learning in Heterogeneous Settings
Disha Makhija (University of Texas at Austin), Nhat Ho (University of Texas at Austin)
Federated LearningKnowledge DistillationImage
🎯 What it does: A personalized federated learning framework FedBNN based on Bayesian learning is proposed, which can train local Bayesian neural networks of different scales and achieve collaboration in environments with small data volumes, heterogeneous computing resources, and non-uniform data distributions.
A Bayesian Approach to Data Point Selection
Xinnuo Xu (Microsoft Research), Timothy Hospedales (University of Edinburgh)
OptimizationData-Centric LearningImageTextStochastic Differential Equation
🎯 What it does: This paper proposes a point selection method based on Bayesian inference called BADS, which utilizes SGLD to simultaneously learn model parameters and sample weights, thereby addressing the issues of slow convergence and high memory consumption in traditional two-layer optimization.
A Best-of-both-worlds Algorithm for Bandits with Delayed Feedback with Robustness to Excessive Delays
Saeed Masoudian (Churney ApS), Yevgeny Seldin (University of Copenhagen)
OptimizationReinforcement Learning from Human Feedback
🎯 What it does: A best-of-both-worlds algorithm suitable for multi-armed bandits with variable delay feedback is proposed, which maintains good performance even without prior knowledge of the maximum delay and in the face of extreme delay outliers.
A Boosting-Type Convergence Result for AdaBoost.MH with Factorized Multi-Class Classifiers
Xin Zou (Wuhan University), Weiwei Liu (Wuhan University)
ClassificationOptimization
🎯 What it does: Proposed and proved the convergence properties of the decomposed ADABOOST.MH, addressing a previously open theoretical problem.
A Canonicalization Perspective on Invariant and Equivariant Learning
George Ma (Peking University), Yisen Wang (Peking University)
Graph Neural NetworkGraph
🎯 What it does: Research on framework-based averaging methods, proposing a unified and optimized approach to symbol/basis invariance learning from the perspective of canonicalization.
A Cat Is A Cat (Not A Dog!): Unraveling Information Mix-ups in Text-to-Image Encoders through Causal Analysis and Embedding Optimization
Chieh-Yun Chen (National Yang Ming Chiao Tung University), Hong-Han Shuai (National Yang Ming Chiao Tung University)
Object DetectionGenerationOptimizationTransformerVision Language ModelDiffusion modelImageText
🎯 What it does: This paper studies the impact of the causal attention mechanism in text encoders on text-to-image diffusion models, proposing a training-independent text embedding balance optimization method (TEBOpt) to eliminate information bias and loss, and presents new automatic evaluation metrics.
A Closer Look at AUROC and AUPRC under Class Imbalance
Matthew B.A. McDermott, Jack Gallifant (Massachusetts Institute of Technology)
TabularSequentialBenchmark
🎯 What it does: The paper evaluates and refutes the widespread notion that AUPRC is superior to AUROC under conditions of class imbalance, providing theoretical proof and empirical experiments, and revealing that AUPRC may lead to fairness bias.
A Closer Look at the CLS Token for Cross-Domain Few-Shot Learning
Yixiong Zou (Huazhong University of Science and Technology), Ruixuan Li (Huazhong University of Science and Technology)
Domain AdaptationMeta LearningTransformerContrastive LearningImage
🎯 What it does: This study investigates the role of the CLS token in Vision Transformer in cross-domain few-shot learning and proposes enhancing model generalization performance by decoupling domain information.
A Combinatorial Algorithm for the Semi-Discrete Optimal Transport Problem
Pankaj K Agarwal, Keegan Yao (Virginia Tech)
Optimization
🎯 What it does: The paper studies the semi-discrete optimal transport problem and proposes a new combinatorial algorithm to achieve ε-approximate solutions.
A Compositional Atlas for Algebraic Circuits
Benjie Wang (University of California), YooJung Choi (Arizona State University)
🎯 What it does: This paper proposes a circuit reasoning combinatorial framework based on semiring algebraic structures to unify the description and analysis of various reasoning queries.
A Comprehensive Analysis on the Learning Curve in Kernel Ridge Regression
Tin Sum Cheng (University of Basel), David Belius (McMaster University)
Tabular
🎯 What it does: A comprehensive learning curve analysis of Kernel Ridge Regression (KRR) is conducted under minimal assumptions, and a unified theory is proposed.
A Concept-Based Explainability Framework for Large Multimodal Models
Jayneel Parekh (Sorbonne Université), Matthieu Cord (Valeo)
Explainability and InterpretabilityTransformerVision Language ModelImageTextMultimodality
🎯 What it does: This paper proposes an explanation framework CoX-LMM based on vocabulary concept dictionary learning, aimed at interpreting the internal representations of large multimodal models (LMM) and conducting multimodal concept mining.
A Consistency-Aware Spot-Guided Transformer for Versatile and Hierarchical Point Cloud Registration
Renlang Huang (Zhejiang University), Liang Li (Zhejiang University)
Autonomous DrivingOptimizationTransformerPoint Cloud
🎯 What it does: This paper proposes a Consistency-Aware Spot-Guided Transformer (CAST) that achieves semi-dense, geometrically consistent coarse matching and designs a lightweight sparse-to-dense refinement module for efficient and accurate point cloud registration.
A Critical Evaluation of AI Feedback for Aligning Large Language Models
Archit Sharma (Stanford University), Thomas Kollar (Toyota Research Institute)
OptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
🎯 What it does: This paper explores the necessity of the RL stage in aligning large language models by comparing two alignment methods: Supervised Fine-Tuning (SFT) and Reinforcement Learning with AI Feedback (LAIF).
A Decision-Language Model (DLM) for Dynamic Restless Multi-Armed Bandit Tasks in Public Health
Nikhil Behari (Massachusetts Institute of Technology), Milind Tambe (Harvard University)
OptimizationTransformerLarge Language ModelReinforcement LearningTabularBiomedical Data
🎯 What it does: This paper proposes the Decision-Language Model (DLM), which utilizes large language models to generate reward functions for RMAB based on human language instructions, aiming to achieve dynamic optimization of public health resource allocation.
A distributional simplicity bias in the learning dynamics of transformers
Riccardo Rende (International School for Advanced Studies), Sebastian Goldt (International School for Advanced Studies)
TransformerLarge Language ModelText
🎯 What it does: This paper proposes and verifies that transformers learn different orders of many-body interactions sequentially in self-supervised learning (MLM and next-token prediction), first learning low-order interactions and then high-order interactions.
A Fast Convoluted Story: Scaling Probabilistic Inference for Integer Arithmetics
Lennert De Smet (KU Leuven), Pedro Zuidberg Dos Martires (Örebro University)
Computational EfficiencyImageBenchmark
🎯 What it does: A differentiable probabilistic integer linear arithmetic framework PLIAt based on tensorization and Fast Fourier Transform (FFT) is proposed for efficient integer probabilistic inference and learning.
A Flexible, Equivariant Framework for Subgraph GNNs via Graph Products and Graph Coarsening
Guy Bar-Shalom (Technion - Israel Institute of Technology), Haggai Maron (NVIDIA Research)
Drug DiscoveryGraph Neural NetworkGraphBiomedical Data
🎯 What it does: This paper proposes a Subgraph GNN framework with a variable subgraph set (CS-GNN), which first refines the original graph to obtain a set of supernodes, then constructs a product graph through the Cartesian product with the original graph, and performs message passing on this graph, supporting subgraph bags of arbitrary size; it also introduces symmetry-based equivariant linear layers and various node labeling strategies.
A Foundation Model for Zero-shot Logical Query Reasoning
Mikhail Galkin (Intel AI Lab), Zhaocheng Zhu (Mila - Quebec AI Institute)
Graph Neural NetworkGraph
🎯 What it does: This paper proposes ULTRAQUERY, the first zero-shot knowledge graph complex logic query answering foundational model, capable of reasoning over any new knowledge graph.
A Framework for Bilevel Optimization on Riemannian Manifolds
Andi Han (RIKEN AIP), Akiko Takeda (RIKEN AIP)
Domain AdaptationOptimizationMeta LearningImage
🎯 What it does: A general framework for solving bilevel optimization on Riemannian manifolds is proposed, along with the Riemannian supergradient estimation method and the corresponding RHGD algorithm;
A Full-duplex Speech Dialogue Scheme Based On Large Language Model
Peng Wang (MThreads AI), Yuanjun Xiong (MThreads AI)
Large Language ModelPrompt EngineeringTextAudio
🎯 What it does: A full-duplex speech dialogue system based on a large language model has been designed, allowing the system to seamlessly switch between listening and speaking while generating speech output in real-time.
A Functional Extension of Semi-Structured Networks
David Rügamer (Ludwig Maximilian University of Munich), Almond Stöcker (École Polytechnique Fédérale de Lausanne)
Explainability and InterpretabilityComputational EfficiencyAuto EncoderTime Series
🎯 What it does: Proposes a Semi-Structured Function Network (SSFNN) that combines functional linear regression with the residual structure of deep networks, achieving an interpretable functional linear part and a flexible deep part;
A General Protocol to Probe Large Vision Models for 3D Physical Understanding
Guanqi Zhan (University of Oxford), Andrew Zisserman (University of Oxford)
Depth EstimationRepresentation LearningTransformerDiffusion modelContrastive LearningImage
🎯 What it does: This paper proposes a general and lightweight detection protocol for evaluating the representational capabilities of large-scale pre-trained visual models (such as CLIP, DINO, VQGAN, and Stable Diffusion) on various 3D physical attributes (scene geometry, material, support relations, shadows, occlusion, depth), and verifies the learnability of these attributes through linear probes.
A generalized neural tangent kernel for surrogate gradient learning
Luke Eilers (University of Bern), Sven Goedeke (University of Bonn)
Tabular
🎯 What it does: This study theoretically analyzes the generalized NTK obtained through the use of substitute gradient learning (SGL) and the extension of the neural tangent kernel (NTK) under activation functions without gradient information (such as sign), and verifies its predictive capability.
A Generative Model of Symmetry Transformations
James Urquhart Allingham (University of Cambridge), José Miguel Hernández-Lobato (University of Cambridge)
GenerationData SynthesisAuto EncoderImage
🎯 What it does: A Symmetry-Aware Generative Model (SGM) is proposed, which learns the distribution of naturally occurring symmetric transformations in the data, representing observations as prototypes plus transformations, and applies this model for data augmentation and generation.
A Geometric View of Data Complexity: Efficient Local Intrinsic Dimension Estimation with Diffusion Models
Hamidreza Kamkari (Layer 6 AI), Gabriel Loaiza-Ganem (Layer 6 AI)
Computational EfficiencyData-Centric LearningDiffusion modelScore-based ModelImage
🎯 What it does: A local intrinsic dimension (LID) estimation method based on diffusion models, FLIPD, is proposed, utilizing the Fokker-Planck equation to achieve efficient and differentiable complexity measurement.
A Global Depth-Range-Free Multi-View Stereo Transformer Network with Pose Embedding
Yitong Dong (Zhejiang University), Guofeng Zhang (Zhejiang University)
Depth EstimationTransformerImage
🎯 What it does: A global depth range unconstrained multi-view stereo vision Transformer network is proposed, which can estimate the depth map of the reference image by utilizing all source images simultaneously.
A Globally Optimal Portfolio for m-Sparse Sharpe Ratio Maximization
Yizun Lin (Jinan University), Cheng Li (Jinan University)
OptimizationTabularTime SeriesFinance Related
🎯 What it does: This paper proposes an m-sparse (at most m active assets) Sharpe ratio maximization model and provides its equivalent quadratic programming form. Subsequently, a Proximal Gradient Algorithm (PGA) based on semi-algebraic properties is developed to solve this non-convex problem, theoretically achieving a global optimal solution under certain conditions.
A Gradient Accumulation Method for Dense Retriever under Memory Constraint
Jaehee Kim (Seoul National University), Pilsung Kang (Seoul National University)
RetrievalSupervised Fine-TuningContrastive LearningText
🎯 What it does: A gradient accumulation method called CONTACCUM with a dual memory pool is proposed for stabilizing the training of dense retrievers in low-resource environments.
A hierarchical decomposition for explaining ML performance discrepancies
Harvineet Singh (University of California, San Francisco), Jean Feng (University of California, San Francisco)
Domain AdaptationExplainability and InterpretabilityBiomedical DataElectronic Health Records
🎯 What it does: This paper proposes a hierarchical, non-parametric framework (HDPD) to explain the fundamental reasons for performance differences of machine learning models across different domains.
A Huber Loss Minimization Approach to Mean Estimation under User-level Differential Privacy
Puning Zhao (Zhejiang Lab), Zhe Liu (Zhejiang Lab)
OptimizationSafty and PrivacyTabular
🎯 What it does: A user-level differential privacy mean estimation method based on Huber loss minimization is proposed;
A Kernel Perspective on Distillation-based Collaborative Learning
Sejun Park (Korea Advanced Institute of Science and Technology), Ganguk Hwang (Korea Advanced Institute of Science and Technology)
Federated LearningKnowledge DistillationImageTabular
🎯 What it does: This paper conducts a non-parametric theoretical analysis of kernel regression based on distillation collaborative learning (FedMD) and proposes a practical algorithm DCL-NN that is compatible with heterogeneous neural networks based on this theory, proving that it can achieve near-optimal convergence rates in large-scale, statistically heterogeneous environments.
A Label is Worth A Thousand Images in Dataset Distillation
Tian Qin (Harvard University), David Alvarez-Melis (Harvard University)
ClassificationKnowledge DistillationImage
🎯 What it does: Through extensive ablation experiments and a simple soft label baseline, it is demonstrated that soft labels are the core factor for the success of data distillation methods, and that randomly sampling real images paired with soft labels from pre-trained experts can approach or even surpass existing state-of-the-art synthetic image distillation methods under significant data compression.
A Layer-Wise Natural Gradient Optimizer for Training Deep Neural Networks
Xiaolei Liu (Ant Group), Binfeng Wang (Ant Group)
OptimizationTransformerImage
🎯 What it does: This paper proposes a hierarchical natural gradient optimizer (LNGD) that utilizes a hierarchical sampling method and Kronecker decomposition to approximate the Fisher information matrix, and introduces an adaptive hierarchical learning rate and dynamic damping.
A Local Method for Satisfying Interventional Fairness with Partially Known Causal Graphs
Haoxuan Li (Peking University), Kun Zhang (Carnegie Mellon University)
OptimizationGraph Neural NetworkTabular
🎯 What it does: A local method is proposed to enumerate the possible parent sets of sensitive attributes, estimate propensity scores, and achieve causal intervention fairness through min-max joint optimization, under the condition of only observing data and having partial knowledge of the causal graph.
A Metalearned Neural Circuit for Nonparametric Bayesian Inference
Jake Snell, Thomas L. Griffiths (Princeton University)
Meta LearningRecurrent Neural NetworkSequential
🎯 What it does: A Neural Circuit based on metalearning is proposed, which can perform sequential nonparametric Bayesian inference for Dirichlet Process Mixture Models (DPMM);
A Method for Evaluating Hyperparameter Sensitivity in Reinforcement Learning
Jacob Adkins (University of Alberta), Adam White (University of Alberta)
Hyperparameter SearchReinforcement LearningTabular
🎯 What it does: A new empirical method is proposed to quantify the sensitivity of reinforcement learning algorithms to hyperparameter tuning in different environments, introducing two evaluation metrics: hyperparameter sensitivity and effective hyperparameter dimension. This method is then used to evaluate various normalized variants of PPO.
A Modular Conditional Diffusion Framework for Image Reconstruction
Magauiya Zhussip (MTS AI), Stamatios Lefkimmiatis
RestorationSuper ResolutionConvolutional Neural NetworkTransformerDiffusion modelImage
🎯 What it does: A modular conditional diffusion framework (DP-IR) is proposed, which combines a pre-trained image restoration network with a denoising module and achieves multi-task conditional modeling through a small fusion network.
A Motion-aware Spatio-temporal Graph for Video Salient Object Ranking
Hao Chen (Southeast University), Yongjian Deng (Beijing University of Technology)
Object DetectionSegmentationGraph Neural NetworkVideo
🎯 What it does: A motion-aware spatiotemporal graph model is proposed for video salient object ranking, and video redirection is achieved based on the ranking results.
A Near-optimal Algorithm for Learning Margin Halfspaces with Massart Noise
Ilias Diakonikolas (University of Wisconsin Madison), Nikos Zarifis (University of Wisconsin Madison)
OptimizationSupervised Fine-Tuning
🎯 What it does: This paper studies the problem of learning γ-margin half-spaces under Massart noise and proposes a computationally efficient learning algorithm.
A Nearly Optimal and Low-Switching Algorithm for Reinforcement Learning with General Function Approximation
Heyang Zhao (University of California, Los Angeles), Quanquan Gu (University of California, Los Angeles)
Reinforcement Learning
🎯 What it does: An algorithm named MQL-UCB is proposed for achieving low switching costs and approximately optimal returns in reinforcement learning with general function approximation.
A Neural Network Approach for Efficiently Answering Most Probable Explanation Queries in Probabilistic Models
Shivvrat Arya (University of Texas at Dallas), Vibhav Giridhar Gogate
Knowledge DistillationTabular
🎯 What it does: A method for arbitrary MPE query solving based on neural networks is proposed, achieving the most probable explanation (MPE) inference for any variable partition in probabilistic models (PC, PGM, NAM) through self-supervised loss, inference-time gradient optimization (ITSELF), and teacher-student pre-training (GUIDE).
A New Neural Kernel Regime: The Inductive Bias of Multi-Task Learning
Julia B Nakhleh, Robert D Nowak
Tabular
🎯 What it does: This study investigates the solutions of multi-task shallow ReLU neural networks trained with weight decay, minimizing the sum of squared weights when fitting data. It proves that multi-task solutions are almost always unique and equivalent to the minimum norm interpolation in Sobolev spaces.
A Non-parametric Direct Learning Approach to Heterogeneous Treatment Effect Estimation under Unmeasured Confounding
Xinhai Zhang (State University of New York at Binghamton), Xingye Qiao (State University of New York at Binghamton)
Tabular
🎯 What it does: A non-parametric direct learning framework (IV-DL) for estimating heterogeneous treatment effects (CATE) using instrumental variables (IV) is proposed, and based on this, two efficient and robust estimation methods (IV-RDL1, IV-RDL2) are constructed through residualization.
A Novel Unified Architecture for Low-Shot Counting by Detection and Segmentation
Jer Pelhan (University of Ljubljana), Matej Kristan (University of Ljubljana)
Object DetectionSegmentationConvolutional Neural NetworkImage
🎯 What it does: A single-stage low-shot counting framework called GeCo is proposed, which achieves unified reasoning across detection, segmentation, and counting.
A Pairwise Pseudo-likelihood Approach for Matrix Completion with Informative Missingness
Jiangyuan Li (Texas A&M University), Kwun Chuen Gary Chan (University of Washington)
Tabular
🎯 What it does: This paper proposes a matrix completion method suitable for missing values that may depend on unobserved quantities.
A Phase Transition between Positional and Semantic Learning in a Solvable Model of Dot-Product Attention
Hugo Cui (École Polytechnique Fédérale de Lausanne), Lenka Zdeborova
OptimizationTransformerText
🎯 What it does: A solvable low-rank dot-product attention model is proposed and analyzed, studying the emergence of global optimal solutions and mechanisms in the high-dimensional limit.
A PID Controller Approach for Adaptive Probability-dependent Gradient Decay in Model Calibration
Siyuan Zhang (Jiangnan University), Linbo Xie (Jiangnan University)
ClassificationOptimizationImage
🎯 What it does: This paper proposes a technique that dynamically adjusts the Softmax gradient decay rate using a PID controller, thereby optimizing both model accuracy and calibration performance during training.
A Polar coordinate system represents syntax in large language models
Pablo J. Diego Simon (PSL University), Jean-Remi King
TransformerLarge Language ModelContrastive LearningText
🎯 What it does: The study investigates whether there exists a polar coordinate system in the activation space of large language models that can fully represent syntactic trees, and proposes a linear probe (Polar Probe) that can simultaneously read the existence and type/direction of syntactic relationships.
A Primal-Dual-Assisted Penalty Approach to Bilevel Optimization with Coupled Constraints
Liuyuan Jiang (Rensselaer Polytechnic Institute), Tianyi Chen (Rensselaer Polytechnic Institute)
OptimizationTabular
🎯 What it does: A penalty reconstruction based on Lagrangian duality is proposed, and a full-step algorithm BLOCC is designed to solve the bilevel optimization problem with coupling constraints.
A probability contrastive learning framework for 3D molecular representation learning
Jiayu Qin (University at Buffalo), Changyou Chen (University at Buffalo)
Representation LearningDrug DiscoveryGraph Neural NetworkTransformerContrastive LearningGraph
🎯 What it does: This paper proposes a probabilistic contrastive learning framework aimed at addressing the issue of incorrect positive-negative sample pairs caused by graph data augmentation in molecular contrastive learning.
A Prompt-Based Knowledge Graph Foundation Model for Universal In-Context Reasoning
Yuanning Cui (Nanjing University), Wei Hu (Nanjing University)
Graph Neural NetworkPrompt EngineeringGraph
🎯 What it does: A KG-based model KG-ICL is proposed, which implements context learning without parameter updates to complete cross-KG reasoning tasks.
A provable control of sensitivity of neural networks through a direct parameterization of the overall bi-Lipschitzness
Yuri Kinoshita (University of Tokyo), Taro Toyoizumi (RIKEN Center for Brain Science)
OptimizationTabular
🎯 What it does: A Bi-Lipschitz Neural Network (BLNN) that can directly control the constant based on convex networks and the Legendre-Fenchel transform is proposed.
A Recipe for Charge Density Prediction
Xiang Fu (Massachusetts Institute of Technology), Tommi Jaakkola (Massachusetts Institute of Technology)
OptimizationComputational EfficiencyTabularPhysics Related
🎯 What it does: A new method (SCDP) is proposed for the rapid and accurate prediction of molecular charge density, capable of directly obtaining charge distribution without the need for self-consistent field iterations, and can be used for further calculations of properties such as energy and force.
A robust inlier identification algorithm for point cloud registration via $\mathbf{\ell_0}$-minimization
Yinuo Jiang (Huazhong University of Science and Technology), Ye Yuan (Huazhong University of Science and Technology)
Autonomous DrivingOptimizationPoint Cloud
🎯 What it does: A robust inlier identification algorithm is proposed that transforms the point cloud registration problem into an ℓ0-minimization problem of alignment error for each local set.
A scalable generative model for dynamical system reconstruction from neuroimaging data
Eric Volkmann (Heidelberg University), Georgia Koppe (Heidelberg University)
GenerationData SynthesisOptimizationRecurrent Neural NetworkSupervised Fine-TuningReinforcement LearningTime SeriesBiomedical DataMagnetic Resonance Imaging
🎯 What it does: This paper proposes a scalable generative model that utilizes control theory's teacher forcing (GTF) and Wiener deconvolution techniques to reconstruct dynamic systems from convolutional observational data such as BOLD fMRI, and generates interpretable generative models.
A Separation in Heavy-Tailed Sampling: Gaussian vs. Stable Oracles for Proximal Samplers
Ye He (Georgia Institute of Technology), Murat A Erdogdu
Optimization
🎯 What it does: This paper studies the complexity of sampling under heavy-tailed target distributions, proving that the Gaussian oracle-based proximal sampler can only achieve polynomial accuracy, while proposing and analyzing a stable proximal sampler based on α-stable oracle, which can achieve logarithmic accuracy.
A Siamese Transformer with Hierarchical Refinement for Lane Detection
Zinan Lv (Shanghai JiaoTong University), Danny Chen
SegmentationAutonomous DrivingTransformerImage
🎯 What it does: This paper proposes a hierarchical refinement framework (LATR) based on Siamese Transformer for lane line detection.
A Simple and Adaptive Learning Rate for FTRL in Online Learning with Minimax Regret of $\Theta(T^{2/3})$ and its Application to Best-of-Both-Worlds
Taira Tsuchiya (University of Tokyo and RIKEN), Shinji Ito (University of Tokyo and RIKEN)
OptimizationReinforcement LearningGraph
🎯 What it does: A new adaptive learning rate framework is proposed, suitable for online learning problems with Θ(T^2/3) minimum maximum regret, and applied to issues such as partial monitoring, graph bandwidth, and multi-armed bandits with paid observations.
A Simple and Optimal Approach for Universal Online Learning with Gradient Variations
Yu-Hu Yan (Nanjing University), Zhi-Hua Zhou (Nanjing University)
Optimization
🎯 What it does: This study investigates the problem of general online learning with gradient variation regret and designs a simple universal method that can achieve optimal gradient variation regret for strongly convex, exponentially concave, and convex functions simultaneously.
A Simple Framework for Generalization in Visual RL under Dynamic Scene Perturbations
Wonil Song (Yonsei University), Dongbo Min (Ewha Womans University)
Convolutional Neural NetworkReinforcement LearningVideo
🎯 What it does: To address the generalization problem of visual reinforcement learning under dynamic background disturbances, this paper proposes the SimGRL framework, which tackles two major flaws: significant imbalance and observation overfitting, and analyzes the essence of the problem through gradient attribution mask.
A Simple Image Segmentation Framework via In-Context Examples
Yang Liu (Zhejiang University), Chunhua Shen (Ant Group)
Object DetectionSegmentationTransformerSupervised Fine-TuningImageVideo
🎯 What it does: This paper studies a general in-context example-based image segmentation framework called SINE, which can simultaneously output masks at three granularities: object, instance, and semantic, addressing the task ambiguity problem in traditional in-context segmentation.
A Simple Remedy for Dataset Bias via Self-Influence: A Mislabeled Sample Perspective
Yeonsung Jung (Korea Advanced Institute of Science and Technology), Eunho Yang (Korea Advanced Institute of Science and Technology)
Domain AdaptationData-Centric LearningSupervised Fine-TuningImageText
🎯 What it does: This paper proposes a method to detect and utilize mislabelled samples through self-influence to identify biased opposing samples, and constructs a small central sample set to fine-tune existing biased models, thereby further calibrating the model without the need for biased labels or an unbiased validation set.
A Simple yet Scalable Granger Causal Structural Learning Approach for Topological Event Sequences
Mingjia Li (East China Normal University), Aimin Zhou (East China Normal University)
Time SeriesSequential
🎯 What it does: A simple and scalable Granger causal structure learning method (S2GCSL) is proposed to identify causal relationships of telecom network alarms from topological event sequences.
A Simple yet Universal Framework for Depth Completion
Jin-Hwi Park (Gwangju Institute of Science and Technology), Hae-Gon Jeon (Gwangju Institute of Science and Technology)
RestorationDepth EstimationConvolutional Neural NetworkSupervised Fine-TuningImagePoint Cloud
🎯 What it does: A unified depth completion (UniDC) problem is proposed, constructing a lightweight framework based on deep foundational models. It utilizes the relative depth features from a monocular camera, hyperplane geometry, and multi-scale feature fusion to achieve rapid conversion from sparse depth to dense depth, and completes depth refinement through pixel-level affinity graphs, supporting few-shot learning with very few labeled data.
A Single-Step, Sharpness-Aware Minimization is All You Need to Achieve Efficient and Accurate Sparse Training
Jie Ji (Clemson University), Xiaolong Ma (Clemson University)
ClassificationOptimizationConvolutional Neural NetworkSupervised Fine-TuningImage
🎯 What it does: This paper proposes a single-step Sharpness-Aware Minimization (S2-SAM) with no additional computational cost, which is applied as a plugin to various sparse training methods, significantly enhancing the generalization performance and robustness of sparse networks.
A Sober Look at the Robustness of CLIPs to Spurious Features
Qizhou Wang (Hong Kong Baptist University), Tong Zhang (University of Illinois Urbana-Champaign)
ClassificationRecognitionTransformerVision Language ModelContrastive LearningImage
🎯 What it does: A new evaluation dataset called CounterAnimal is proposed to test the robustness of the CLIP model on animal images in real-world backgrounds.
A Structure-Aware Framework for Learning Device Placements on Computation Graphs
Shukai Duan (University of Southern California), Paul Bogdan (University of Southern California)
OptimizationComputational EfficiencyGraph Neural NetworkTransformerReinforcement LearningGraph
🎯 What it does: Designed and implemented an end-to-end device placement framework HSDAG to optimize the inference time of computation graphs on heterogeneous hardware (CPU, GPU).
A Study of Plasticity Loss in On-Policy Deep Reinforcement Learning
Arthur Juliani (Microsoft Research), Jordan T. Ash (Microsoft Research)
Reinforcement LearningSequential
🎯 What it does: This paper systematically studies the phenomenon of plasticity loss in on-policy deep reinforcement learning and experimentally evaluates various existing mitigation methods (intermittent reset, regularization, network structure modification, etc.).
A Surprisingly Simple Approach to Generalized Few-Shot Semantic Segmentation
Tomoya Sakai (IBM Research), Tadanobu Inoue (IBM)
SegmentationSupervised Fine-TuningImage
🎯 What it does: A method for base class mining (BCM) based on simple rules is proposed, utilizing standard supervised learning to identify new categories in generalized few-shot semantic segmentation (GFSS) while maintaining the segmentation performance of most base classes.
A Swiss Army Knife for Heterogeneous Federated Learning: Flexible Coupling via Trace Norm
Tianchi Liao (Sun Yat-sen University), Chuan Chen (Sun Yat-sen University)
Federated LearningImage
🎯 What it does: A federated multi-task learning framework FedSAK based on tensor trace norm is proposed, which can simultaneously handle the heterogeneity of data, models, and tasks within the same framework.
A teacher-teacher framework for clinical language representation learning
Feiqing Huang (Harvard University), Tianxi Cai (Harvard University)
Representation LearningTransformerLarge Language ModelTextBiomedical DataElectronic Health Records
🎯 What it does: A teacher-teacher framework is proposed, utilizing two pre-trained large language models (LLMs) to achieve mutual knowledge alignment through a lightweight LINE module, generating a unified representation across data forms (text and concept lists).