NeurIPS 2023 Papers — Page 24
Conference on Neural Information Processing Systems · 3218 papers
PRODIGY: Enabling In-context Learning Over Graphs
Qian Huang (Stanford University), Jure Leskovec (Stanford University)
ClassificationGraph Neural NetworkSupervised Fine-TuningContrastive LearningGraph
🎯 What it does: The research implements unsupervised In-Context learning on graph data, proposing the PRODIGY framework, which allows pre-trained models to perform classification tasks directly on new graphs with a small number of examples.
Progressive Ensemble Distillation: Building Ensembles for Efficient Inference
Don Dennis (Carnegie Mellon University), Virginia Smith (Carnegie Mellon University)
Computational EfficiencyKnowledge DistillationRecurrent Neural NetworkImageAudio
🎯 What it does: Decomposes a large-scale teacher model into a set of low inference cost weak learners, forming a scalable ensemble model on demand;
Projection Regret: Reducing Background Bias for Novelty Detection via Diffusion Models
Sungik Choi (LG AI Research), Moontae Lee (LG AI Research)
Anomaly DetectionDiffusion modelImage
🎯 What it does: Proposes the Projection Regret method, which maps any sample to a training distribution sample similar to the background through diffusion model projection, and detects anomalies using perceptual distance, further eliminating background bias through recursive projection;
Projection-Free Methods for Solving Nonconvex-Concave Saddle Point Problems
Morteza Boroun (University of Arizona), Afrooz Jalilzadeh (University of Arizona)
OptimizationTabular
🎯 What it does: This paper proposes two single-loop projection-free primal-dual algorithms to solve non-convex-concave saddle point problems, namely the fully projection-free R-PDCG (using linear minimization oracles to handle both primal and dual ends) and the one-sided projection-free CG-RPGA (using LMO for the primal end and projection for the dual end), and provides convergence rate proofs.
Projection-Free Methods for Stochastic Simple Bilevel Optimization with Convex Lower-level Problem
Jincheng Cao (University of Texas at Austin), Aryan Mokhtari (University of Texas at Austin)
OptimizationTabular
🎯 What it does: This paper proposes two projection-free stochastic bilevel optimization methods (SBCGI and SBCGF) to solve stochastic simple bilevel optimization problems where the reachable solution set of the lower-level problem is convex and the upper-level objective may be non-convex.
Projection-Free Online Convex Optimization via Efficient Newton Iterations
Khashayar Gatmiry (Massachusetts Institute of Technology), Zakaria Mhammedi (Massachusetts Institute of Technology)
OptimizationFinance Related
🎯 What it does: This paper proposes a new projection-free online convex optimization algorithm called BARONS, which generates feasible iterations using approximate Newton steps of self-concordant barrier functions and efficiently solves linear systems on polyhedra through the Lee-Sidford barrier, achieving O(√T) asymptotic optimal regret without the need for Euclidean projection.
ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation
Zhengyi Wang (Tsinghua University), Jun Zhu (Tsinghua University)
GenerationData SynthesisDiffusion modelNeural Radiance FieldTextMesh
🎯 What it does: Based on a pre-trained text-to-image diffusion model, a Variational Score Distillation (VSD) method is proposed, utilizing particle sampling to learn the distribution of 3D representations, thereby achieving high fidelity, rich details, and diverse text-to-3D generation; subsequently, the ProlificDreamer framework is constructed, combining high-resolution rendering, scene initialization, and two-stage temporal scheduling to realize 512×512 NeRF and high-quality textured meshes.
Promises and Pitfalls of Threshold-based Auto-labeling
Harit Vishwakarma (University of Wisconsin Madison), Ramya Korlakai Vinayak (University of Wisconsin Madison)
ClassificationData-Centric LearningSupervised Fine-TuningImageText
🎯 What it does: This paper conducts a theoretical analysis and empirical evaluation of the Threshold-Based Automatic Labeling (TBAL) algorithm, providing sample complexity bounds between the amount of validation data and the quality of automatic labeling.
Prompt Pre-Training with Twenty-Thousand Classes for Open-Vocabulary Visual Recognition
Shuhuai Ren (Peking University), Xu Sun (Peking University)
ClassificationObject DetectionSegmentationTransformerPrompt EngineeringContrastive LearningImage
🎯 What it does: Pre-train a general soft prompt POMP to learn semantics on the large-scale categories of ImageNet-21K, and then directly perform zero-shot transfer to tasks such as image classification, semantic segmentation, and object detection.
Prompt-augmented Temporal Point Process for Streaming Event Sequence
Siqiao Xue (Ant Group), JUN ZHOU
Prompt EngineeringSequential
🎯 What it does: This paper proposes a framework called PromptTPP for continual learning in streaming event sequences without replay buffers and task labels;
PromptIR: Prompting for All-in-One Image Restoration
Vaishnav Potlapalli (Mohamed bin Zayed University of AI), Fahad Khan
RestorationTransformerPrompt EngineeringImage
🎯 What it does: This paper proposes PromptIR, a versatile blind image restoration framework that utilizes prompt learning to simultaneously restore various degraded images such as denoising, deraining, and defogging under unknown degradation conditions.
PromptRestorer: A Prompting Image Restoration Method with Degradation Perception
Cong Wang (Hong Kong Polytechnic University), Junyang Chen (Shenzhen University)
RestorationTransformerPrompt EngineeringImage
🎯 What it does: This paper proposes PromptRestorer, a framework that utilizes prompt learning to extract raw degradation features from pre-trained models to guide image restoration.
Propagating Knowledge Updates to LMs Through Distillation
Shankar Padmanabhan (University of Texas at Austin), Eunsol Choi (University of Texas at Austin)
Knowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Inject entity definition information into the parameters of pre-trained language models through context distillation, allowing the model to reason based on the injected knowledge.
ProPILE: Probing Privacy Leakage in Large Language Models
Siwon Kim (Seoul National University), Seong Joon Oh (Tübingen AI Center)
Safty and PrivacyTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: ProPILE is a tool for detecting personal identity information (PII) leaks in large language models (LLMs), allowing data subjects to verify whether their PII has been leaked and helping LLM service providers assess the privacy risks of their models.
Proportional Response: Contextual Bandits for Simple and Cumulative Regret Minimization
Sanath Kumar Krishnamurthy (Stanford University), Emma Brunskill (Stanford University)
OptimizationReinforcement LearningTabular
🎯 What it does: This paper proposes a regression-based contextual multi-armed bandit algorithm named RAPR (Risk Adjusted Proportional Response), which can output the best personalized decision strategy after the experiment ends.
Protein Design with Guided Discrete Diffusion
Nate Gruver (New York University), Andrew Gordon Wilson (New York University)
OptimizationDrug DiscoveryReinforcement LearningDiffusion modelBiomedical Data
🎯 What it does: This paper proposes NOS (Diffusion-Optimized Sampling) and an improved version LaMBO-2, utilizing discrete diffusion models and gradient guidance for direct antibody design in the protein sequence space, ultimately achieving high expression rates and high binding rates in laboratory validation.
ProteinNPT: Improving Protein Property Prediction and Design with Non-Parametric Transformers
Pascal Notin (University of Oxford), Yarin Gal (University of Oxford)
Protein Structure PredictionTransformerSupervised Fine-TuningBiomedical Data
🎯 What it does: A semi-supervised conditional pseudo-generative model named ProteinNPT is proposed for protein property prediction and design in scenarios with scarce labels and the need for multi-task learning.
PROTES: Probabilistic Optimization with Tensor Sampling
Anastasia Batsheva (Skolkovo Institute of Science and Technology), Ivan Oseledets (Skolkovo Institute of Science and Technology)
Optimization
🎯 What it does: A probability sampling optimization method called PROTES based on Tensor Train (TT) format is proposed to solve black-box multi-dimensional discrete optimization problems.
ProtoDiff: Learning to Learn Prototypical Networks by Task-Guided Diffusion
Yingjun Du (AIM Lab University of Amsterdam), Cees G. M. Snoek (AIM Lab University of Amsterdam)
ClassificationMeta LearningTransformerDiffusion modelImageTabular
🎯 What it does: The ProtoDiff framework is proposed, which utilizes a task-oriented diffusion model during the meta-learning phase to gradually generate overfitting prototypes from naive prototypes, enhancing few-shot classification performance.
Prototype-based Aleatoric Uncertainty Quantification for Cross-modal Retrieval
Hao Li (University of Electronic Science and Technology of China), Heng Tao Shen (University of Electronic Science and Technology of China)
RetrievalContrastive LearningImageVideoTextMultimodality
🎯 What it does: This study investigates a prototype-based framework for quantifying perceptual uncertainty, aimed at assessing data uncertainty and enhancing prediction credibility in cross-modal retrieval.
Prototypical Variational Autoencoder for 3D Few-shot Object Detection
Weiliang Tang (Chinese University of Hong Kong), Chi-Wing Fu (Chinese University of Hong Kong)
Object DetectionAuto EncoderPoint Cloud
🎯 What it does: This paper proposes a Prototypical Variational Autoencoder (P-VAE) framework for few-shot 3D point cloud object detection, achieving high-quality learning of geometric and class-level prototypes by learning a latent space in the form of a multi-center Gaussian Mixture Model (GMM). It introduces two dedicated modules, GP-VAE and CP-VAE, in the detection network to handle scene-level and object-level reconstruction and prototype correction, respectively.
Provable Advantage of Curriculum Learning on Parity Targets with Mixed Inputs
Emmanuel Abbe (École Polytechnique Fédérale de Lausanne), Aryo Lotfi (École Polytechnique Fédérale de Lausanne)
OptimizationRecurrent Neural NetworkSupervised Fine-TuningTabular
🎯 What it does: This paper proposes and theoretically proves that a curriculum learning strategy, which first trains on sparse samples (i.e., low Hamming weight) and then on complete samples, can significantly reduce the number of training steps and sample size required to learn k-th order singular functions (Parity) under a mixed sparse-dense input distribution.
Provable Adversarial Robustness for Group Equivariant Tasks: Graphs, Point Clouds, Molecules, and More
Jan Schuchardt (Technical University of Munich), Stephan Günnemann (Technical University of Munich)
Adversarial AttackDrug DiscoveryGraph Neural NetworkReinforcement LearningPoint CloudGraph
🎯 What it does: A provable definition of adversarial robustness for equivariant tasks is proposed, and based on this, an equivariance-preserving random smoothing framework is constructed, further deriving various robustness certificates such as graph edit distance.
Provable benefits of annealing for estimating normalizing constants: Importance Sampling, Noise-Contrastive Estimation, and beyond
Omar Chehab (University Paris-Saclay), Andrej Risteski (Carnegie Mellon University)
Contrastive Learning
🎯 What it does: A class of general 'annealed Bregman estimators' (ABE) is proposed for estimating the normalization constant of a target distribution by sampling along a path composed of several intermediate distributions.
Provable benefits of score matching
Chirag Pabbaraju (Stanford University), Andrej Risteski (Carnegie Mellon University)
Computational EfficiencyScore-based Model
🎯 What it does: This study investigates the computational and statistical efficiency of parameter estimation using score matching and maximum likelihood estimation (MLE) within a class of exponential families (polynomial energy functions).
Provable convergence guarantees for black-box variational inference
Justin Domke (University of Massachusetts Amherst), Guillaume Garrigos (Université Paris Cité and Sorbonne Université)
Optimization
🎯 What it does: In this paper, the author conducts a rigorous convergence analysis of the stochastic optimization process in black-box variational inference (black-box VI), providing non-asymptotic convergence guarantees specifically for the Gaussian variational family.
Provable Guarantees for Generative Behavior Cloning: Bridging Low-Level Stability and High-Level Behavior
Adam Block (Massachusetts Institute of Technology), Russ Tedrake (Massachusetts Institute of Technology)
GenerationRobotic IntelligenceDiffusion modelMultimodality
🎯 What it does: This paper proposes a theoretical framework that combines the stabilization characteristics of low-level controllers with generative models to achieve provable behavior cloning of complex expert demonstrations.
Provable Guarantees for Neural Networks via Gradient Feature Learning
Zhenmei Shi (University of Wisconsin), Yingyu Liang (University of Wisconsin)
Optimization
🎯 What it does: A unified gradient feature learning framework is proposed to analyze the learning process of two-layer neural networks under gradient descent, providing provable error guarantees.
Provable Guarantees for Nonlinear Feature Learning in Three-Layer Neural Networks
Eshaan Nichani (Princeton University), Jason D. Lee (Princeton University)
OptimizationRepresentation LearningTabular
🎯 What it does: This paper studies the feature learning capability of three-layer neural networks, proving that they have a richer ability to learn hierarchical features compared to two-layer networks.
Provable Training for Graph Contrastive Learning
Yue Yu (Beijing University of Posts and Telecommunications), Chuan Shi (Beijing University of Posts and Telecommunications)
OptimizationRepresentation LearningGraph Neural NetworkContrastive LearningGraph
🎯 What it does: This paper addresses the issue of uneven node training in Graph Contrastive Learning (GCL) by proposing a 'node compactness' metric to measure the degree to which each node adheres to the InfoNCE principle under all possible graph augmentations. Based on this, a provably optimal training (POT) regularization method is designed to enhance the quality of node embeddings.
Provably (More) Sample-Efficient Offline RL with Options
Xiaoyan Hu (Chinese University of Hong Kong), Ho-fung Leung
Reinforcement Learning
🎯 What it does: This paper provides the first theoretical analysis of the sample complexity of using options in offline reinforcement learning and proposes an algorithm based on lazy value iteration called PEVIO.
Provably Bounding Neural Network Preimages
Suhas Kotha (Carnegie Mellon University), Huan Zhang (University of Illinois at Urbana-Champaign)
Object DetectionOptimizationReinforcement LearningImageBenchmark
🎯 What it does: An algorithm INVPROP is proposed for approximating the upper bound of neural network preimages, which can efficiently solve the feasible region of the input space under given output linear constraints.
Provably Efficient Algorithm for Nonstationary Low-Rank MDPs
Yuan Cheng (National University of Singapore), Yingbin Liang (Ohio State University)
OptimizationReinforcement Learning
🎯 What it does: An end-to-end policy optimization algorithm for non-stationary low-rank Markov decision processes, called PORTAL, and its non-parametric version, Ada-PORTAL, are proposed, along with a theoretical upper bound on the average dynamic suboptimality gap.
Provably Efficient Offline Goal-Conditioned Reinforcement Learning with General Function Approximation and Single-Policy Concentrability
Hanlin Zhu (University of California Berkeley), Amy Zhang (University of Texas Austin)
Robotic IntelligenceReinforcement Learning
🎯 What it does: This paper studies an algorithm for offline goal-oriented reinforcement learning called VP-learning. It provides a theoretical sample complexity and suboptimality analysis under the assumptions of general function approximation and single policy concentration, and validates its effectiveness on tasks such as robot grasping/pushing.
Provably Efficient Offline Reinforcement Learning in Regular Decision Processes
Roberto Cipollone (Sapienza University of Rome), Mohammad Sadegh Talebi (University of Copenhagen)
Reinforcement Learning
🎯 What it does: This paper proposes an RDP algorithm for offline reinforcement learning, RegORL, which converts the non-Markov decision process (RDP) into a Markov decision process (MDP) and utilizes existing MDP offline RL methods to achieve near-optimal policies.
Provably Fast Convergence of Independent Natural Policy Gradient for Markov Potential Games
Youbang Sun (Northeastern University), Shahin Shahrampour (Northeastern University)
OptimizationReinforcement Learning
🎯 What it does: This paper studies the convergence properties of the Independent Natural Policy Gradient (NPG) algorithm in Markov Potential Games, proving that under certain technical assumptions, its average NE-gap can converge to ε-Nash equilibrium at a rate of O(1/ε);
Provably Fast Finite Particle Variants of SVGD via Virtual Particle Stochastic Approximation
Aniket Das (Google Research), Dheeraj Mysore Nagaraj
TabularStochastic Differential Equation
🎯 What it does: Two realizable SVGD variants (VP-SVGD and GB-SVGD) are proposed, along with their convergence rates and computational complexities in the case of a finite number of particles.
Provably Robust Temporal Difference Learning for Heavy-Tailed Rewards
Semih Cayci (RWTH Aachen University), Atilla Eryilmaz (Ohio State University)
Reinforcement LearningSequential
🎯 What it does: This study investigates the TD learning and Natural Actor-Critic (NAC) algorithm improved by a dynamic gradient clipping mechanism in reinforcement learning environments with heavy-tailed reward distributions and potentially infinite variance, proposing provable robustness and convergence.
Provably Safe Reinforcement Learning with Step-wise Violation Constraints
Nuoya Xiong (Tsinghua University), Longbo Huang (Tsinghua University)
Safty and PrivacyReinforcement LearningTabular
🎯 What it does: This study investigates the reinforcement learning problem that must satisfy safety constraints at every step, proposing two algorithms: SUCBVI and SRF-UCRL, which are used for safe reinforcement learning with immediate violation constraints (Safe‑RL‑SW) and reward-agnostic safe exploration (Safe‑RFE‑SW), respectively.
Proximity-Informed Calibration for Deep Neural Networks
Miao Xiong (National University of Singapore), Bryan Hooi (Google)
ClassificationConvolutional Neural NetworkImage
🎯 What it does: The concept of 'proximity bias' is proposed, studying the issue of deep models being overconfident in sparse regions, and its universality is validated on 504 ImageNet pre-trained models;
Pruning vs Quantization: Which is Better?
Andrey Kuzmin (Qualcomm AI Research), Tijmen Blankevoort (Qualcomm AI Research)
CompressionLarge Language ModelSupervised Fine-TuningImageText
🎯 What it does: This paper provides a systematic comparison of two compression techniques: neural network pruning and quantization, covering theoretical error analysis, hierarchical error lower bounds, global solutions, and fine-tuning results of complete models.
Pseudo-Likelihood Inference
Theo Gruner (Technical University of Darmstadt), Jan Peters (German Research Center for AI)
🎯 What it does: A novel SBI method called Pseudo-Likelihood Inference (PLI) is proposed for inferring the posterior distribution of black-box simulators under multiple observation conditions.
PTQD: Accurate Post-Training Quantization for Diffusion Models
Yefei He (Zhejiang University), Bohan Zhuang (Monash University)
GenerationCompressionDiffusion modelImage
🎯 What it does: This paper proposes the PTQD method for untrained low-bit quantization of diffusion models, significantly reducing model size and inference costs.
Public Opinion Field Effect Fusion in Representation Learning for Trending Topics Diffusion
Junliang Li (Tianjin University), Hong Gao (Zhejiang Normal University)
Representation LearningGraph Neural NetworkGraph
🎯 What it does: A heterogeneous graph representation learning framework POFD is proposed and implemented, which integrates the effects of public opinion fields and social circle influences for the analysis of popular topic diffusion.
PUCA: Patch-Unshuffle and Channel Attention for Enhanced Self-Supervised Image Denoising
Hyemi Jang (Seoul National University), Sungroh Yoon (Seoul National University)
RestorationConvolutional Neural NetworkImage
🎯 What it does: A self-supervised image denoising model called PUCA based on J-invariant U-Net is proposed, which can train high-quality denoising results using only noisy images.
PUe: Biased Positive-Unlabeled Learning Enhancement by Causal Inference
Xutao Wang (Huawei Noah's Ark Lab), Yunhe Wang (Huawei Noah's Ark Lab)
ClassificationAnomaly DetectionConvolutional Neural NetworkImageBiomedical DataAlzheimer's Disease
🎯 What it does: A PUe algorithm based on causal inference is proposed to address the imbalance of positive and negative labels in PU learning. By using normalized inverse probability weighting for positive samples, the loss function is corrected to improve classifier performance in scenarios with label bias.
Punctuation-level Attack: Single-shot and Single Punctuation Can Fool Text Models
Wenqiang Wang (Nanjing University), Xiaochun Cao
Adversarial AttackTransformerText
🎯 What it does: This study proposes a punctuation-level attack method that utilizes a single punctuation mark to perform black-box text model attacks, and designs two techniques: TPPE and TPPEP.
Puzzlefusion: Unleashing the Power of Diffusion Models for Spatial Puzzle Solving
Sepidehsadat Hosseini (Simon Fraser University), Yasutaka Furukawa (Simon Fraser University)
GenerationTransformerDiffusion modelImage
🎯 What it does: Proposes an end-to-end neural architecture called PuzzleFusion, which utilizes diffusion models as a conditional generation process to solve spatial puzzle problems (including puzzle assembly and room layout arrangement);
PyNeRF: Pyramidal Neural Radiance Fields
Haithem Turki (Carnegie Mellon University), Deva Ramanan (Carnegie Mellon University)
GenerationData SynthesisComputational EfficiencyNeural Radiance FieldImagePoint Cloud
🎯 What it does: This paper proposes a multi-scale grid-based NeRF rendering method called PyNeRF, which significantly improves anti-aliasing effects and maintains high rendering speed by training multiple model heads on grids of different resolutions and selecting the appropriate scale based on sample volume size during rendering.
Q-DM: An Efficient Low-bit Quantized Diffusion Model
Yanjing Li (Beihang University), Baochang Zhang (Beihang University)
GenerationData SynthesisComputational EfficiencyDiffusion modelImage
🎯 What it does: A low-bit quantization diffusion model (Q-DM) is proposed, significantly reducing inference costs while maintaining generation quality.
QLoRA: Efficient Finetuning of Quantized LLMs
Tim Dettmers (University of Washington), Luke Zettlemoyer (University of Washington)
OptimizationComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: A QLORA method is proposed, which uses a 4-bit quantized model for efficient fine-tuning on a single GPU, maintaining the same performance as 16-bit fine-tuning.
QuACK: Accelerating Gradient-Based Quantum Optimization with Koopman Operator Learning
Di Luo (Massachusetts Institute of Technology), Marin Soljacic (Massachusetts Institute of Technology)
OptimizationNeural Architecture SearchTabularPhysics Related
🎯 What it does: This paper proposes a framework called QuACK, which treats the optimization trajectory of variational quantum algorithms as a dynamical system, utilizing the Koopman operator to learn linear predictions of gradient evolution, thereby significantly reducing gradient steps.
QuadAttac$K$: A Quadratic Programming Approach to Learning Ordered Top-$K$ Adversarial Attacks
Thomas Paniagua (North Carolina State University), Tianfu Wu (North Carolina State University)
OptimizationAdversarial AttackConvolutional Neural NetworkTransformerImage
🎯 What it does: This paper proposes a quadratic programming-based ordered top-K adversarial attack method called QuadAttac K, which can generate specified ordered top-K prediction results in a clear box attack environment.
Quantification of Uncertainty with Adversarial Models
Kajetan Schweighofer (Johannes Kepler University Linz), Sepp Hochreiter (Johannes Kepler University Linz)
Anomaly DetectionAdversarial AttackConvolutional Neural NetworkMixture of ExpertsGenerative Adversarial NetworkImage
🎯 What it does: A new uncertainty quantification method called QUAM is proposed and implemented. This method identifies important patterns in the prior distribution by searching for adversarial models and uses mixed importance sampling to accurately approximate prior integrals, thereby better estimating the model's epistemic uncertainty.
Quantifying & Modeling Multimodal Interactions: An Information Decomposition Framework
Paul Pu Liang (Carnegie Mellon University), Louis-Philippe Morency (Carnegie Mellon University)
OptimizationRepresentation LearningMultimodalityBenchmark
🎯 What it does: Deconstruct and quantify information for multimodal interaction, proposing a scalable PID estimation method.
Quantifying the Cost of Learning in Queueing Systems
Daniel Freund (Massachusetts Institute of Technology), Wentao Weng (Massachusetts Institute of Technology)
OptimizationReinforcement Learning
🎯 What it does: This paper proposes a new metric called the Cost of Learning in Queueing Systems (CLQ), providing lower and nearly optimal upper bounds for learning costs in single queue multi-server systems and more general queue networks, along with corresponding learning algorithms.
Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing
Yelysei Bondarenko (Qualcomm AI Research), Tijmen Blankevoort (Qualcomm AI Research)
TransformerLarge Language ModelImageText
🎯 What it does: This paper analyzes the causes of extreme outliers in Transformer activations and proposes two structural modifications: clipped softmax and gated attention, thereby suppressing extreme values during the pre-training phase and achieving a Transformer model that is easier to quantize post-training.
QuantSR: Accurate Low-bit Quantization for Efficient Image Super-Resolution
Haotong Qin (Beihang University), Fisher Yu (ETH Zurich)
Super ResolutionCompressionComputational EfficiencyConvolutional Neural NetworkTransformerImage
🎯 What it does: This paper proposes QuantSR, a low-bit (2-4 bit) image super-resolution network that significantly reduces model size and computational load while maintaining high accuracy.
Quantum Bayesian Optimization
Zhongxiang Dai (National University of Singapore), Patrick Jaillet (Massachusetts Institute of Technology)
OptimizationTabularPhysics Related
🎯 What it does: The first quantum Bayesian optimization algorithm Q-GP-UCB is proposed, which uses quantum Monte Carlo (QMC) to achieve high-precision evaluation of black-box functions.
Quantum speedups for stochastic optimization
Aaron Sidford (Stanford University), Chenyi Zhang (Stanford University)
OptimizationPhysics Related
🎯 What it does: Proposes a quantum variance reduction technique and uses this technique to construct a quantum random optimization algorithm, achieving acceleration in both convex and non-convex scenarios;
Quasi-Monte Carlo Graph Random Features
Isaac Reid (University of Cambridge), Adrian Weller (University of Cambridge)
Graph Neural NetworkGraph
🎯 What it does: This paper proposes the q-GRF (quasi-graph random features) with resistance termination, which reduces the variance of kernel estimation through negatively correlated random walk lengths while maintaining unbiased estimation, improving upon the previous GRFs method.
Query-based Temporal Fusion with Explicit Motion for 3D Object Detection
Jinghua Hou (Huazhong University of Science and Technology), Xiang Bai (Huazhong University of Science and Technology)
Object DetectionAutonomous DrivingTransformerPoint Cloud
🎯 What it does: A query-based temporal fusion network QTNet is proposed, which achieves temporal information fusion for 3D object detection using sparse query features.
QuIP: 2-Bit Quantization of Large Language Models With Guarantees
Jerry Chee (Cornell University), Christopher De Sa (Cornell University)
CompressionOptimizationTransformerLarge Language ModelText
🎯 What it does: A post-training quantization method named QuIP is proposed, which can compress large language models to just 2 bits and has theoretical guarantees.
R-divergence for Estimating Model-oriented Distribution Discrepancy
Zhilin Zhao (Macquarie University), Longbing Cao (Macquarie University)
Domain AdaptationSupervised Fine-TuningImage
🎯 What it does: Proposes the R-divergence metric for model-driven distribution difference assessment;
RADAR: Robust AI-Text Detection via Adversarial Learning
Xiaomeng Hu (Chinese University of Hong Kong), Tsung-Yi Ho (Chinese University of Hong Kong)
GenerationAdversarial AttackTransformerLarge Language ModelReinforcement LearningGenerative Adversarial NetworkText
🎯 What it does: An AI text detection framework called RADAR based on adversarial learning is proposed, which jointly trains a generator (rewriter) and a detector, making the detector robust to both LLM-generated text and its rewritten versions.
Random Cuts are Optimal for Explainable k-Medians
Konstantin Makarychev (Northwestern University), Liren Shan (TTIC)
OptimizationExplainability and Interpretability
🎯 What it does: The paper studies the interpretable k-medians clustering problem, proposing and analyzing the Random Coordinate Cut algorithm (RANDOMCOORDINATECUT), and proving that it has an optimal competitive ratio under ℓ1 distance;
Random-Access Infinite Context Length for Transformers
Amirkeivan Mohtashami (École Polytechnique Fédérale de Lausanne), Martin Jaggi (École Polytechnique Fédérale de Lausanne)
TransformerLarge Language ModelSupervised Fine-TuningTextRetrieval-Augmented Generation
🎯 What it does: A landmark token-based attention mechanism is designed and implemented, allowing the Transformer to achieve infinite context length without increasing training costs while maintaining random access flexibility.
Randomized and Deterministic Maximin-share Approximations for Fractionally Subadditive Valuations
Hannaneh Akrami (Max Planck Institute for Informatics), Golnoosh Shahkarami (Max Planck Institute for Informatics)
Optimization
🎯 What it does: This paper studies the maximum minimum share (MMS) approximation problem for fractional subadditive (XOS) utility functions in the allocation of discrete items, proposing new deterministic and randomized allocation algorithms. The deterministic algorithm achieves a 3/13-MMS approximation, while the randomized algorithm achieves an expected (ex-ante) 1/4-MMS and a final (ex-post) guarantee of 1/8-MMS.
Randomized Sparse Neural Galerkin Schemes for Solving Evolution Equations with Deep Networks
Jules Berman (New York University), Benjamin Peherstorfer (New York University)
OptimizationComputational EfficiencyTime SeriesSequentialPhysics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: A sequential time-step training method based on random sparse updates, called RSNG, is proposed to maintain causality and reduce computational costs when solving partial differential equations.
RangePerception: Taming LiDAR Range View for Efficient and Accurate 3D Object Detection
Yeqi BAI, Yikang LI
Object DetectionAutonomous DrivingConvolutional Neural NetworkPoint Cloud
🎯 What it does: A 3D object detection framework called RangePerception based on Range View is proposed, addressing the issues of spatial misalignment and visual degradation.
Rank-1 Matrix Completion with Gradient Descent and Small Random Initialization
Daesung Kim (Samsung Electronics), Hye Won Chung (KAIST)
OptimizationTabular
🎯 What it does: It is proven that in the rank-1 symmetric matrix completion problem, ordinary gradient descent (GD) with small random initialization can converge to the true matrix.
Rank-DETR for High Quality Object Detection
Yifan Pu (Tsinghua University), Gao Huang (Tsinghua University)
Object DetectionTransformerImage
🎯 What it does: In response to DETR-based object detection, Rank-DETR is proposed, achieving high-quality detection through a series of ranking-based architectures and optimization designs.
Rank-N-Contrast: Learning Continuous Representations for Regression
Kaiwen Zha (Massachusetts Institute of Technology), Dina Katabi (Massachusetts Institute of Technology)
Representation LearningConvolutional Neural NetworkContrastive LearningTabular
🎯 What it does: Proposes the Rank-N-Contrast (RNC) framework to learn continuous, sequentially aware feature representations, thereby enhancing the performance of deep regression models.
RanPAC: Random Projections and Pre-trained Models for Continual Learning
Mark McDonnell, Anton van den Hengel (Australian Institute for Machine Learning)
ClassificationRepresentation LearningTransformerSupervised Fine-TuningImage
🎯 What it does: Utilizing features from pre-trained models, combined with a frozen random projection layer and class prototype accumulation, to achieve replay-free continual learning;
RAPHAEL: Text-to-Image Generation via Large Mixture of Diffusion Paths
Zeyue Xue (University of Hong Kong), Ping Luo (University of Hong Kong)
GenerationData SynthesisTransformerMixture of ExpertsDiffusion modelImageText
🎯 What it does: A text-to-image generation model called RAPHAEL is designed based on large-scale mixed diffusion paths, capable of generating highly artistic images that are highly consistent with text prompts.
RayDF: Neural Ray-surface Distance Fields with Multi-view Consistency
Zhuoman Liu (Hong Kong Polytechnic University), Jinxi Li (Hong Kong Polytechnic University)
GenerationData SynthesisNeural Radiance FieldPoint Cloud
🎯 What it does: Learned and implemented a multi-view consistency ray-surface distance field to represent 3D shapes, achieving efficient point cloud and view generation.
RDumb: A simple approach that questions our progress in continual test-time adaptation
Ori Press (University of Tübingen), Matthias Bethge (University of Tübingen)
ClassificationDomain AdaptationConvolutional Neural NetworkTransformerImageBenchmark
🎯 What it does: A Continuous Changing Interference (CCC) benchmark is proposed, demonstrating the collapse of existing Test-Time Adaptation (TTA) methods in long-term continuous adaptation, and a simple baseline RDumb is introduced, which only requires periodic resetting of pre-trained weights.
Re-Think and Re-Design Graph Neural Networks in Spaces of Continuous Graph Diffusion Functionals
Tingting Dan (University of North Carolina), Guorong Wu (University of North Carolina)
ClassificationRecommendation SystemGraph Neural NetworkGenerative Adversarial NetworkGraphBiomedical DataMagnetic Resonance ImagingAlzheimer's Disease
🎯 What it does: This paper proposes a GNN design framework based on continuous graph diffusion functions and variational theory, which can rethink and improve the inductive bias of existing GNNs.
Read and Reap the Rewards: Learning to Play Atari with the Help of Instruction Manuals
Yue Wu (Carnegie Mellon University), Tom Mitchell (Carnegie Mellon University)
Large Language ModelReinforcement LearningText
🎯 What it does: This paper proposes the Read and Reward framework, which utilizes game manuals to extract information through QA and employs LLM for reasoning, generating auxiliary rewards for Atari RL agents, thereby significantly improving sample efficiency and game scores.
Reading Relevant Feature from Global Representation Memory for Visual Object Tracking
Xinyu Zhou (Fudan University), Wenqiang Zhang (Fudan University)
Object TrackingTransformerSupervised Fine-TuningVideo
🎯 What it does: A novel visual object tracking framework RFGM is proposed, combining relevant attention and global representation memory;
Real-Time Motion Prediction via Heterogeneous Polyline Transformer with Relative Pose Encoding
Zhejun Zhang (ETH Zurich), Luc Van Gool (ETH Zurich)
Autonomous DrivingTransformerPoint Cloud
🎯 What it does: A real-time motion prediction framework HPTR based on aligned polygons is proposed, which efficiently integrates various traffic elements by combining KNARPE attention;
Real-World Image Super-Resolution as Multi-Task Learning
Wenlong Zhang (Hong Kong Polytechnic University), Chao Dong (Shanghai AI Laboratory)
RestorationSuper ResolutionImage
🎯 What it does: This paper re-examines the real-world image super-resolution (real-SR) problem from the perspective of multi-task learning and proposes a task grouping method to address the issue of task competition, fine-tuning the real-SR model through this method.
Real-World Image Variation by Aligning Diffusion Inversion Chain
Yuechen ZHANG, Jiaya Jia (Chinese University of Hong Kong)
GenerationData SynthesisDiffusion modelImageText
🎯 What it does: Without the need for additional training, high-quality real image variants are generated by aligning the inversion chain and generation chain of the diffusion model.
Recaptured Raw Screen Image and Video Demoiréing via Channel and Spatial Modulations
Huanjing Yue (Tianjin University), Jingyu Yang (Tianjin University)
RestorationConvolutional Neural NetworkImageVideo
🎯 What it does: For the task of removing moiré patterns from raw input images and videos, a channel and spatial modulation network combining color separation and mixed features is proposed.
Recasting Continual Learning as Sequence Modeling
Soochan Lee (Seoul National University), Gunhee Kim (Seoul National University)
Computational EfficiencyMeta LearningTransformerImageSequential
🎯 What it does: Reformulate Continual Learning as a sequence modeling problem, and utilize Transformers and their efficient variants for Meta-Continual Learning within this framework.
RECESS Vaccine for Federated Learning: Proactive Defense Against Model Poisoning Attacks
Haonan Yan (Xidian University), Xiaodong Lin (University of Guelph)
Federated LearningImage
🎯 What it does: Designed and implemented the RECESS framework, which actively constructs test gradient detection clients in federated learning and achieves robust aggregation through multi-round trust scoring to defend against the latest model poisoning attacks.
RECKONING: Reasoning through Dynamic Knowledge Encoding
Zeming Chen (École Polytechnique Fédérale de Lausanne), Antoine Bosselut (École Polytechnique Fédérale de Lausanne)
TransformerLarge Language ModelText
🎯 What it does: By performing one or more gradient updates on the language model during inference, external contextual knowledge is encoded into the model parameters, enabling multi-hop reasoning.
Recommender Systems with Generative Retrieval
Shashank Rajput (University of Wisconsin-Madison), Maheswaran Sathiamoorthy (Google)
RetrievalRecommendation SystemTransformerText
🎯 What it does: A recommendation framework called TIGER based on generative retrieval is proposed, which represents products as semantic IDs generated by hierarchical quantization, and directly predicts the next item's semantic ID using a Transformer sequence-to-sequence model.
Reconciling Competing Sampling Strategies of Network Embedding
Yuchen Yan (University of Illinois), Hanghang Tong (University of Illinois)
Recommendation SystemGraph Neural NetworkContrastive LearningGraph
🎯 What it does: This paper studies the positive and negative sampling strategies in network embedding and proposes a two-step sampling framework called SENSEI that balances discriminability and monotonicity.
Reconstructing the Mind's Eye: fMRI-to-Image with Contrastive Learning and Diffusion Priors
Paul Steven Scotti, Tanishq Mathew Abraham (Stability AI)
RestorationGenerationRetrievalDiffusion modelContrastive LearningImageMagnetic Resonance Imaging
🎯 What it does: MindEye proposes a system that simultaneously supports brain image decoding and retrieval, capable of reconstructing natural scene images viewed by humans from fMRI signals and accurately locating the original images;
ReContrast: Domain-Specific Anomaly Detection via Contrastive Reconstruction
Jia Guo (Beijing Institute of Technology), Huiqi Li (Beijing Institute of Technology)
Anomaly DetectionAuto EncoderContrastive LearningImageBiomedical Data
🎯 What it does: This study investigates unsupervised anomaly detection for specific domains and proposes the ReContrast method, optimizing the entire network to address the domain discrepancy issue of ImageNet pre-trained encoders.
Recovering from Out-of-sample States via Inverse Dynamics in Offline Reinforcement Learning
Ke Jiang (Nanjing University of Aeronautics and Astronautics), Xiaoyang Tan (Peking University)
Reinforcement LearningTabular
🎯 What it does: Proposes an Offline State Recovery (OSR) method in offline reinforcement learning, utilizing an inverse dynamics model to guide the policy in recovering states within the offline data distribution, alleviating the issue of state distribution shift.
Recovering Simultaneously Structured Data via Non-Convex Iteratively Reweighted Least Squares
Christian Kümmerle (University of North Carolina at Charlotte), Johannes Maly (Ludwig-Maximilians-Universität München)
OptimizationTabular
🎯 What it does: This paper studies a novel Iteratively Reweighted Least Squares (IRLS) algorithm for recovering data matrices that possess both row sparsity and low-rank structure from linear observations.
Recovering Unbalanced Communities in the Stochastic Block Model with Application to Clustering with a Faulty Oracle
Chandra Sekhar Mukherjee (University of Southern California), Jiapeng Zhang (University of Southern California)
OptimizationGraph
🎯 What it does: A spectral algorithm based on SVD is designed, capable of recovering large clusters in SBM with any number of small clusters, and achieving sublinear query clustering under erroneous prior oracle models.
Recurrent Hypernetworks are Surprisingly Strong in Meta-RL
Jacob Beck (University of Oxford), Shimon Whiteson (University of Oxford)
Meta LearningRecurrent Neural NetworkReinforcement LearningSequential
🎯 What it does: This study investigates the use of recursive hypernetworks (RNN+HN) in Meta-RL to achieve improvements in sample efficiency and performance.
Recurrent Temporal Revision Graph Networks
YIZHOU CHEN, Zhiming Zhou (Shanghai University of Finance and Economics)
Recommendation SystemRecurrent Neural NetworkGraph Neural NetworkGraphTime Series
🎯 What it does: A Recurrent Temporal Revision (RTR) layer is proposed, which aggregates historical neighbor information using hidden states to address the issue of incomplete information caused by temporal graph neighbor sampling.
Recursion in Recursion: Two-Level Nested Recursion for Length Generalization with Scalability
Jishnu Ray Chowdhury (University of Illinois Chicago), Cornelia Caragea (University of Illinois Chicago)
OptimizationComputational EfficiencyRecurrent Neural NetworkTextSequential
🎯 What it does: Proposed the 'Recursion in Recursion' (RIR) framework, which nests the outer k-ary balanced tree recursion with the inner Efficient Beam-Tree RvNN (EBT-RvNN), achieving logarithmic scalability for long sequences;
Red Teaming Deep Neural Networks with Feature Synthesis Tools
Stephen Casper (Massachusetts Institute of Technology), Dylan Hadfield-Menell (Massachusetts Institute of Technology)
Explainability and InterpretabilityAdversarial AttackConvolutional Neural NetworkImageBenchmark
🎯 What it does: This paper proposes a new evaluation framework for model debugging using interpretability tools—assessing the effectiveness of feature synthesis tools by rediscovering implanted network interpretability backdoors (trojans).
ReDS: Offline RL With Heteroskedastic Datasets via Support Constraints
Anikait Singh (University of California Berkeley), Sergey Levine (Google DeepMind)
Reinforcement LearningTabular
🎯 What it does: This paper proposes a method called ReDS that re-weights distribution constraints to support constraints, addressing the issue of distribution constraint failure caused by heteroskedasticity in offline reinforcement learning.
Reduced Policy Optimization for Continuous Control with Hard Constraints
Shutong Ding (ShanghaiTech University), Ye Shi (ShanghaiTech University)
OptimizationReinforcement LearningBenchmark
🎯 What it does: A Reduced Policy Optimization (RPO) algorithm is proposed for reinforcement learning that strictly satisfies equality and inequality hard constraints in continuous control tasks.