NeurIPS 2023 Papers — Page 23
Conference on Neural Information Processing Systems · 3218 papers
Path Regularization: A Convexity and Sparsity Inducing Regularization for Parallel ReLU Networks
Tolga Ergen (LG AI Research), Mert Pilanci (Stanford University)
OptimizationImage
🎯 What it does: This study investigates the training problem of deep parallel ReLU networks, proposing an analytical method to reveal the implicit convexity in the optimization landscape, and demonstrates that the path regularization training problem can be represented as an exact convex optimization problem.
Paxion: Patching Action Knowledge in Video-Language Foundation Models
Zhenhailong Wang (University of Illinois Urbana-Champaign), Heng Ji (University of North Carolina)
RecognitionRetrievalTransformerVision-Language-Action ModelContrastive LearningVideoTextMultimodalityBenchmark
🎯 What it does: The ActionBench benchmark is proposed to quantify the action knowledge of video-language models, and the PAXION framework is designed to fill the gaps in the model's action understanding through a lightweight Knowledge Patcher and Knowledge Fuser while maintaining its original capabilities.
Payoff-based Learning with Matrix Multiplicative Weights in Quantum Games
Kyriakos Lotidis (Stanford University), Jose Blanchet (Stanford University)
OptimizationReinforcement Learning from Human FeedbackReinforcement LearningTabularPhysics Related
🎯 What it does: This paper proposes a Minimum Information Quantity Learning algorithm based on Matrix Multiplication Weights (MMW), which achieves convergence in quantum games and semi-definite games using only scalar payoff feedback.
PCF-GAN: generating sequential data via the characteristic function of measures on the path space
Hang Lou (University College London), Hao Ni (University College London)
GenerationData SynthesisRecurrent Neural NetworkAuto EncoderGenerative Adversarial NetworkTime SeriesSequentialFinance Related
🎯 What it does: This paper proposes PCF-GAN, a generative adversarial network that utilizes Path Characteristic Functions (PCF) as a discriminator for high-fidelity time series data generation and reconstruction.
PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers
Phillip Lippe (Microsoft Research AI4Science), Johannes Brandstetter (Microsoft Research AI4Science)
Diffusion modelTime SeriesPhysics Related
🎯 What it does: This paper proposes a neural PDE solver called PDE-Refiner based on iterative refinement, addressing the issue of accuracy degradation caused by the loss of frequency information in long-term rolling predictions.
PDF: Point Diffusion Implicit Function for Large-scale Scene Neural Representation
Yuhan Ding (Fudan University), Tao Chen (Fudan University)
GenerationData SynthesisDiffusion modelNeural Radiance FieldMultimodalityPoint Cloud
🎯 What it does: This paper proposes an implicit function based on point diffusion (PDF), utilizing diffusion super-resolution of sparse point clouds to obtain a dense surface prior, thereby restricting the sampling space to the scene surface. It combines background area sampling from Mip-NeRF 360 to achieve viewpoint synthesis for large-scale outdoor scenes.
PDP: Parameter-free Differentiable Pruning is All You Need
Minsik Cho (Apple Inc), Devang Naik (Apple Inc)
OptimizationConvolutional Neural NetworkTransformerImageText
🎯 What it does: A parameter-free differentiable pruning method (Parameter-free Differentiable Pruning, PDP) is proposed, which generates soft pruning masks directly through a dynamic function, avoiding additional learnable parameters and complex training processes, and enabling adaptive weight pruning during training.
Penalising the biases in norm regularisation enforces sparsity
Etienne Boursier (INRIA), Nicolas Flammarion (EPFL)
Tabular
🎯 What it does: This study investigates the function space corresponding to the parameter norm of a single hidden layer ReLU network in one dimension and the minimum norm interpolator. It finds that penalizing the bias leads to a total variation of weights √(1+x²), resulting in uniqueness and sparsity.
Pengi: An Audio Language Model for Audio Tasks
Soham Deshmukh (Microsoft), Huaming Wang (Microsoft)
ClassificationRecognitionGenerationTransformerLarge Language ModelVision Language ModelMultimodalityAudio
🎯 What it does: Designed and trained an audio language model called Pengi, which unifies all audio tasks into an input of audio + text prompts, outputting a free-text generation task that can complete open and closed audio tasks without additional fine-tuning.
Penguin: Parallel-Packed Homomorphic Encryption for Fast Graph Convolutional Network Inference
Ran Ran (North Carolina State University), Wujie Wen (North Carolina State University)
OptimizationSafty and PrivacyComputational EfficiencyGraph Neural NetworkGraph
🎯 What it does: For encrypted inference of Graph Convolutional Networks (GCN), we propose Penguin—a method for homomorphic encryption ciphertext compression and operation optimization based on two-dimensional parallel packing and interleaved assembly.
Percentile Criterion Optimization in Offline Reinforcement Learning
Cyrus Cousins (University of Massachusetts Amherst), Yair Zick (University of Massachusetts Amherst)
OptimizationReinforcement LearningTabular
🎯 What it does: This paper proposes a new algorithm based on Value at Risk (VaR) dynamic programming to optimize quantile criteria in offline reinforcement learning with limited data, resulting in less conservative and robust policies with confidence guarantees.
Perceptual adjustment queries and an inverted measurement paradigm for low-rank metric learning
Austin Xu (Georgia Institute of Technology), Ashwin Pananjady (Georgia Institute of Technology)
OptimizationTabular
🎯 What it does: Proposed the 'Perception Adjustment Query' (PAQ) and applied it to learn low-rank Mahalanobis distance metrics from human feedback.
Perceptual Kalman Filters: Online State Estimation under a Perfect Perceptual-Quality Constraint
Dror Freirich (Technion Israel Institute of Technology), Ron Meir (Technion Israel Institute of Technology)
RestorationOptimizationVideo
🎯 What it does: This paper proposes an online temporal signal reconstruction method that ensures the filter output is completely consistent with the original natural signal in terms of spatiotemporal distribution, thereby achieving the goal of 'perfect perceptual quality'.
PERFOGRAPH: A Numerical Aware Program Graph Representation for Performance Optimization and Program Analysis
Ali TehraniJamsaz (Iowa State University), Ali Jannesari (Iowa State University)
OptimizationGraph Neural NetworkGraph
🎯 What it does: A program graph representation called PERFOGRAPH based on LLVM IR is proposed, which supports aggregate data types and is numerically aware, making it suitable for performance optimization and program analysis.
Performance Bounds for Policy-Based Average Reward Reinforcement Learning Algorithms
Yashaswini Murthy (University of Illinois), R. Srikant (University of Illinois)
Reinforcement Learning
🎯 What it does: This paper presents a finite-time error upper bound for approximate policy iteration (API) in the average reward Markov decision process (MDP), addressing the challenge that traditional discounted reward analysis cannot be generalized to average reward problems.
Performance Scaling via Optimal Transport: Enabling Data Selection from Partially Revealed Sources
Feiyang Kang (Virginia Tech), Ruoxi Jia (Virginia Tech)
OptimizationData-Centric LearningImage
🎯 What it does: A framework called projektor is proposed for predicting model performance based on a small number of samples and selecting data sources in partially revealed data sources.
Performance-optimized deep neural networks are evolving into worse models of inferotemporal visual cortex
Drew Linsley (Brown University), Thomas Serre (Brown University)
ClassificationOptimizationExplainability and InterpretabilityConvolutional Neural NetworkTransformerSupervised Fine-TuningImage
🎯 What it does: This paper evaluates the relationship between the classification accuracy of 135 different deep neural networks (CNNs, ViTs, SSLs, robust models, etc.) on ImageNet and their predictive accuracy for IT cortical neurons in macaques. It finds that as the model's performance on ImageNet improves, its predictive accuracy for IT neural responses actually decreases. Subsequently, by using the 'neural vocoder' training method, the model's representations were aligned with human visual features, successfully breaking this trade-off and significantly improving IT predictive accuracy.
Permutation Equivariant Neural Functionals
Allan Zhou (Stanford University), Chelsea Finn (Stanford University)
ClassificationOptimizationConvolutional Neural NetworkAuto EncoderImage
🎯 What it does: This paper proposes a neural network framework capable of handling 'weight space' data such as neural network weights and gradients—Permutation Equivariant Neural Functionals (NFNs). It achieves effective encoding of the weight space by constructing an NF-Layer that is equivariant under neuron permutation symmetry.
Personalized Dictionary Learning for Heterogeneous Datasets
Geyu Liang (University of Michigan), Salar Fattahi (University of Michigan)
OptimizationFederated LearningImageVideo
🎯 What it does: A personalized dictionary learning (PerDL) framework is proposed, which uses the federated matching and averaging algorithm (PerMA) to separate shared and local sparse features from heterogeneous datasets.
Persuading Farsighted Receivers in MDPs: the Power of Honesty
Martino Bernasconi (Bocconi University), Mirco Mutti (Technion)
OptimizationReinforcement Learning from Human FeedbackReinforcement Learning
🎯 What it does: This paper studies the Bayesian persuasion problem for farsighted receivers in finite-horizon Markov Decision Processes (MDP), exploring how to design information revelation strategies to maximize the sender's expected payoff.
Perturbation Towards Easy Samples Improves Targeted Adversarial Transferability
Junqi Gao (Harbin Institute of Technology), Dazhi Zhang (Harbin Institute of Technology)
Adversarial AttackConvolutional Neural NetworkGenerative Adversarial NetworkImage
🎯 What it does: This paper proposes to enhance the transferability of targeted adversarial attacks by utilizing perturbation directions aimed at high sample density regions (HSDR) of the target class.
PETAL: Physics Emulation Through Averaged Linearizations for Solving Inverse Problems
Jihui Jin (Georgia Institute of Technology), Justin Romberg
OptimizationTime SeriesPhysics Related
🎯 What it does: Designed and trained a physics-based linearized forward model proxy (PETAL) to solve inverse problems in ocean acoustic mapping.
PGDiff: Guiding Diffusion Models for Versatile Face Restoration via Partial Guidance
Peiqing Yang (Nanyang Technological University), Chen Change Loy (Nanyang Technological University)
RestorationDiffusion modelImage
🎯 What it does: This paper proposes a facial image restoration framework called PGDiff based on diffusion models, which achieves multi-task restoration by 'partially guiding' the attributes of high-quality images during the reverse diffusion process, without the need to pre-assume the degradation process.
Phase diagram of early training dynamics in deep neural networks: effect of the learning rate, depth, and width
Dayal Singh Kalra (University of Maryland), Maissam Barkeshli (University of Maryland)
OptimizationConvolutional Neural NetworkRecurrent Neural NetworkImage
🎯 What it does: The system studied the effects of learning rate, network depth, and width on the early training dynamics when using stochastic gradient descent (SGD) training in deep networks, and plotted the corresponding phase diagrams;
PHOTOSWAP: Personalized Subject Swapping in Images
Jing Gu (University of California), Xin Eric Wang (University of California)
GenerationData SynthesisDiffusion modelImage
🎯 What it does: A training-free attention exchange framework based on diffusion models, called Photoswap, is designed to achieve personalized theme replacement in images.
Physics-Driven ML-Based Modelling for Correcting Inverse Estimation
Ruiyuan Kang (Bayanat AI), Dimitrios Kyritsis
OptimizationGenerative Adversarial NetworkPhysics Related
🎯 What it does: A physics-driven machine learning error correction algorithm called GEESE is proposed to detect and correct failures in ML estimates for scientific and engineering inverse problems, significantly reducing the number of physical evaluation queries.
Physics-Informed Bayesian Optimization of Variational Quantum Circuits
Kim Andrea Nicoli, Shinichi Nakajima (Technische Universitaet Berlin)
OptimizationPhysics Related
🎯 What it does: This paper proposes a Bayesian optimization method that combines physical priors for parameter search in the Variational Quantum Eigensolver (VQE).
Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation
Yuval Kirstain (Tel Aviv University), Omer Levy (Stability AI)
GenerationRecommendation SystemTransformerReinforcement LearningPrompt EngineeringVision Language ModelImageText
🎯 What it does: By constructing a web application to collect real user preferences during the text-to-image generation process, a large-scale Pick-a-Pic dataset was generated, and based on this, the PickScore scoring model was trained to predict user preferences, evaluate models, and enhance generation quality.
PICProp: Physics-Informed Confidence Propagation for Uncertainty Quantification
Qianli Shen (National University of Singapore), Kenji Kawaguchi (National University of Singapore)
OptimizationComputational EfficiencyTabularPhysics RelatedOrdinary Differential Equation
🎯 What it does: A new method called PICProp is proposed for uncertainty quantification in physical information learning, particularly by propagating confidence intervals (CIs) from data locations to the entire domain.
PID-Inspired Inductive Biases for Deep Reinforcement Learning in Partially Observable Control Tasks
Ian Char (Carnegie Mellon University), Jeff Schneider (Carnegie Mellon University)
Recurrent Neural NetworkTransformerReinforcement LearningSequential
🎯 What it does: Proposed PID-inspired historical encoders PIDE and GPIDE to improve the robustness of deep reinforcement learning in partially observable control tasks.
Pitfall of Optimism: Distributional Reinforcement Learning by Randomizing Risk Criterion
Taehyun Cho (Seoul National University), Jungwoo Lee (Seoul National University)
Reinforcement LearningSequential
🎯 What it does: A distributed reinforcement learning algorithm is proposed that randomizes risk criteria to avoid biased exploration caused by one-sided risk preferences.
PlanE: Representation Learning over Planar Graphs
Radoslav Dimitrov (University of Oxford), Ismail Ilkan Ceylan (University of Oxford)
Representation LearningDrug DiscoveryGraph Neural NetworkGraphBiomedical Data
🎯 What it does: Designed the PLANE framework to achieve scalable graph representation learning on planar graphs and proved that it can obtain complete graph invariants.
PLANNER: Generating Diversified Paragraph via Latent Language Diffusion Model
Yizhe Zhang (Apple), Navdeep Jaitly (Apple)
GenerationTransformerDiffusion modelAuto EncoderText
🎯 What it does: PLANNER is proposed, a two-stage latent diffusion model: first, it learns fixed-length paragraph embeddings using VAE, then generates semantic distributions through latent diffusion, and finally produces fluent long texts using an autoregressive decoder;
PLASTIC: Improving Input and Label Plasticity for Sample Efficient Reinforcement Learning
Hojoon Lee (Korea Advanced Institute of Science and Technology), Chulhee Yun (Korea Advanced Institute of Science and Technology)
Reinforcement LearningImage
🎯 What it does: This study addresses the 'plasticity loss' problem in offline reinforcement learning and proposes the PLASTIC algorithm to enhance sample efficiency by improving the plasticity of inputs and labels.
Plug-and-Play Stability for Intracortical Brain-Computer Interfaces: A One-Year Demonstration of Seamless Brain-to-Text Communication
Chaofei Fan (Stanford University), Francis R Willett
Recurrent Neural NetworkLarge Language ModelText
🎯 What it does: This paper proposes a method for continuous online self-calibration of iBCI (CORP) using language models to generate pseudo-labels, and validates its long-term stability in writing-based BCI through over a year of clinical trials.
PoET: A generative model of protein families as sequences-of-sequences
Timothy Fei Truong Jr, Tristan Bepler (Open Protein AI)
GenerationData SynthesisProtein Structure PredictionTransformerSequentialBiomedical DataRetrieval-Augmented Generation
🎯 What it does: A self-regressive generative model called PoET is proposed and trained to generate new protein sequences from homologous sequences of a given protein family, and to predict the functional variants through conditional probabilities.
Point Cloud Completion with Pretrained Text-to-Image Diffusion Models
Yoni Kasten (NVIDIA Research), Gal Chechik (Bar-Ilan University)
RestorationGenerationDiffusion modelNeural Radiance FieldPoint Cloud
🎯 What it does: This paper proposes a testing-time optimization method called SDS-Complete, which utilizes a pre-trained text-to-image diffusion model to recover partial point clouds obtained from real sensors into complete surfaces.
PointGPT: Auto-regressively Generative Pre-training from Point Clouds
Guangyan Chen (Beijing Institute of Technology), Yufeng Yue (Peking University)
ClassificationSegmentationGenerationTransformerPoint Cloud
🎯 What it does: This paper proposes PointGPT, which achieves self-supervised pre-training of point clouds through autoregressive generation tasks, overcoming the challenges of unordered point clouds, low information density, and the gap between generation and downstream tasks.
Pointwise uncertainty quantification for sparse variational Gaussian process regression with a Brownian motion prior
Luke Travis (Imperial College London), Kolyan Ray (Imperial College London)
Time SeriesStochastic Differential Equation
🎯 What it does: This paper studies the theoretical properties of point estimation and uncertainty quantification (confidence intervals) under the frequentist framework when using sparse variational Gaussian process regression (SGPR) with feature vector induced variables, particularly the coverage of confidence sets.
Policy Finetuning in Reinforcement Learning via Design of Experiments using Offline Data
Ruiqi Zhang (University of California), Andrea Zanette (University of California)
OptimizationReinforcement LearningTabular
🎯 What it does: An algorithm is proposed that utilizes an offline dataset to design a single non-reactive exploration strategy to improve the policy fine-tuning effect in reinforcement learning.
Policy Gradient for Rectangular Robust Markov Decision Processes
Navdeep Kumar (Technion), Shie Mannor (NVIDIA Research)
OptimizationReinforcement Learning
🎯 What it does: A reinforcement learning method for rectangular robust Markov decision processes (MDP) is proposed—Robust Policy Gradient (RPG), which can directly obtain closed-form gradients from the model's 'worst' transitions and rewards without relying on expensive convex optimization solvers, thus training robust optimal policies.
Policy Optimization for Continuous Reinforcement Learning
Hanyang Zhao (Columbia University), David Yao
OptimizationReinforcement LearningSequential
🎯 What it does: A policy optimization framework for reinforcement learning is constructed in continuous time and space, including continuous versions of occupancy time, performance difference formulas, and TRPO/PPO methods.
Policy Optimization in a Noisy Neighborhood: On Return Landscapes in Continuous Control
Nathan Rahn, Marc G Bellemare
OptimizationRobotic IntelligenceReinforcement Learning
🎯 What it does: This study investigates the return landscape of deep reinforcement learning agents in continuous control, exploring how slight changes in policy parameters lead to variations in return distribution. It proposes using the return distribution (especially the left tail probability) to assess policy stability and designs a CVaR-based update rejection mechanism to reduce the left tail probability of the return distribution.
Policy Space Diversity for Non-Transitive Games
Jian Yao (Tencent AI Lab), Yang Wei
Reinforcement Learning
🎯 What it does: A PSRO (PSD-PSRO) framework based on multi-strategy space diversity is proposed, which further approximates the Nash equilibrium of zero-sum non-transitive games by incorporating diversity regularization focused on convex hull expansion in the best response solving.
PolyDiffuse: Polygonal Shape Reconstruction via Guided Set Diffusion Models
Jiacheng Chen (Simon Fraser University), Yasutaka Furukawa (Simon Fraser University)
RestorationObject DetectionGenerationTransformerDiffusion modelPoint CloudOrdinary Differential Equation
🎯 What it does: This paper proposes the Guided Set Diffusion Model (GS-DM), which achieves polygonal geometric reconstruction under sensor data conditions by injecting noise into each element of the set one by one during the diffusion process and guiding the denoising.
Polyhedron Attention Module: Learning Adaptive-order Interactions
Tan Zhu (University of Connecticut), Jinbo Bi (University of Connecticut)
Recommendation SystemTabularBiomedical Data
🎯 What it does: Designing the Polyhedron Attention Module (PAM) to achieve adaptive order feature interaction, improving the expressive power of traditional ReLU networks;
Polynomial-Time Linear-Swap Regret Minimization in Imperfect-Information Sequential Games
Gabriele Farina (Massachusetts Institute of Technology), Charilaos Pipis (Massachusetts Institute of Technology)
OptimizationReinforcement Learning from Human FeedbackSequential
🎯 What it does: A polynomial-time algorithm for achieving linear transformation backtracking no-linear-swap regret learning is proposed in any incomplete information sequential game.
Polynomially Over-Parameterized Convolutional Neural Networks Contain Structured Strong Winning Lottery Tickets
Arthur da Cunha (Université Côte d'Azur), Emanuele Natale (Université Côte d'Azur)
Convolutional Neural Network
🎯 What it does: It is proven that in multi-layer convolutional neural networks, randomly initialized over-parameterized networks almost certainly contain structured sub-networks that can approximate any smaller network without training.
POMDP Planning for Object Search in Partially Unknown Environment
Yongbo Chen (Australian National University), Hanna Kurniawati (Australian National University)
OptimizationRobotic IntelligenceReinforcement LearningImagePoint Cloud
🎯 What it does: A target object search method based on Partially Observable Markov Decision Process (POMDP) is proposed, along with a design for an expandable state space and guessing target objects.
POP-3D: Open-Vocabulary 3D Occupancy Prediction from Images
Antonín Vobecký (Valeo AI), Josef Sivic (Czech Institute of Informatics Robotics and Cybernetics)
SegmentationRetrievalAutonomous DrivingTransformerContrastive LearningImageTextPoint Cloud
🎯 What it does: POP-3D predicts 3D occupancy grids for open vocabulary using images from a single camera, achieving zero-shot 3D semantic segmentation and natural language-based 3D retrieval.
Post Hoc Explanations of Language Models Can Improve Language Models
Satyapriya Krishna (Harvard University), Himabindu Lakkaraju (Harvard University)
Explainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringTextBenchmarkChain-of-Thought
🎯 What it does: A framework named AMPLIFY is proposed, which automatically generates post hoc explanations as natural language interpretations to enhance the reasoning and understanding capabilities of large language models (LLMs).
Post-processing Private Synthetic Data for Improving Utility on Selected Measures
Hao Wang (MIT IBM Watson AI Lab), Akash Srivastava (MIT IBM Watson AI Lab)
Data SynthesisOptimizationSafty and PrivacyTabular
🎯 What it does: A post-processing method is proposed to enhance the performance of synthetic data on user-specified utility metrics through resampling, while maintaining differential privacy guarantees.
Posterior Contraction Rates for Matérn Gaussian Processes on Riemannian Manifolds
Paul Rosa (University of Oxford), Judith Rousseau (University of Oxford)
Mesh
🎯 What it does: This paper studies the posterior convergence rates of Matérn Gaussian processes on Riemannian manifolds, providing theoretical proofs and experimental validations for intrinsic, truncated, and extrinsic models.
Posterior Sampling for Competitive RL: Function Approximation and Partial Observation
Shuang Qiu (Hong Kong University of Science and Technology), Tong Zhang (Hong Kong University of Science and Technology)
Reinforcement Learning
🎯 What it does: A model-based algorithm based on posterior sampling is proposed for learning Nash equilibria in zero-sum Markov games (MG) under general function approximation, supporting both self-play and adversarial learning settings, and capable of handling fully observable and partially observable scenarios.
Posterior Sampling with Delayed Feedback for Reinforcement Learning with Linear Function Approximation
Nikki Lijing Kuang (University of California), Yian Ma
Reinforcement Learning
🎯 What it does: This paper proposes an algorithm for exploration using posterior sampling in linear MDPs with delayed feedback, capable of handling random delays while ensuring sublinear scheduling.
Posthoc privacy guarantees for collaborative inference with modified Propose-Test-Release
Abhishek Singh (Massachusetts Institute of Technology), Ramesh Raskar (Massachusetts Institute of Technology)
Representation LearningAdversarial AttackAuto EncoderContrastive LearningImage
🎯 What it does: A post-hoc privacy guarantee framework is proposed, providing formal reconstruction privacy protection for low-dimensional encodings generated by deep networks in collaborative inference.
PPi: Pretraining Brain Signal Model for Patient-independent Seizure Detection
Zhizhang Yuan (Zhejiang University), Yafeng Li (Nuozhu Technology Co., Ltd.)
ClassificationDomain AdaptationAnomaly DetectionConvolutional Neural NetworkTransformerContrastive LearningTime SeriesBiomedical Data
🎯 What it does: A patient-independent SEEG seizure detection model called PPi is proposed, which is based on self-supervised pre-training and employs two techniques, channel background subtraction and brain region enhancement, to improve generalization ability during the detection phase.
Practical and Asymptotically Exact Conditional Sampling in Diffusion Models
Luhuan Wu (Columbia University), John Patrick Cunningham
Image TranslationRestorationGenerationData SynthesisProtein Structure PredictionDiffusion modelScore-based ModelImageBiomedical Data
🎯 What it does: This paper proposes the Twisted Diffusion Sampler (TDS), a conditional sampling framework based on Sequential Monte Carlo (SMC) that allows for precise sampling from trained diffusion models without the need for task-specific training.
Practical Contextual Bandits with Feedback Graphs
Mengxiao Zhang (University of Southern California), Paul Mineiro (Microsoft Research)
Recommendation SystemOptimizationGraph Neural NetworkReinforcement LearningGraphTabular
🎯 What it does: A new algorithm SquareCB G is proposed, which utilizes feedback graph structures to simplify the contextual multi-armed bandit problem into a regression task, achieving a computationally feasible and scalable implementation.
Practical Differentially Private Hyperparameter Tuning with Subsampling
Antti Koskela (Nokia Bell Labs), Tejas Kulkarni (Nokia Bell Labs)
OptimizationSafty and PrivacyHyperparameter SearchImage
🎯 What it does: A practical differential privacy hyperparameter tuning method is proposed, which reduces privacy and computational costs by randomly subsampling sensitive data and extrapolating the optimized hyperparameter values to larger datasets.
Practical Equivariances via Relational Conditional Neural Processes
Daolang Huang (Aalto University), Luigi Acerbi (University of Helsinki)
OptimizationTime Series
🎯 What it does: A new Relational Conditional Neural Process (RCNP) model is proposed, which achieves intrinsic modeling of equivariance through relational encoding, suitable for high-dimensional inputs.
Practical Sharpness-Aware Minimization Cannot Converge All the Way to Optima
Dongkuk Si (Korea Advanced Institute of Science and Technology), Chulhee Yun (Korea Advanced Institute of Science and Technology)
Optimization
🎯 What it does: This study investigates the convergence properties of Sharpness-Aware Minimization (SAM) under constant perturbation size and gradient normalization, proving the convergence upper and lower bounds of deterministic and stochastic SAM under different smoothness/convexity conditions, and providing an unavoidable ρ² error example.
Pre-RMSNorm and Pre-CRMSNorm Transformers: Equivalent and Efficient Pre-LN Transformers
Zixuan Jiang (Google), David Z. Pan (University of Texas at Austin)
Computational EfficiencyTransformerImageText
🎯 What it does: Two equivalent and efficient variants of the pre-normalized Transformer, Pre-RMSNorm and Pre-CRMSNorm, are proposed, along with a cost-free conversion method from Pre-LN to these two variants.
Pre-training Contextualized World Models with In-the-wild Videos for Reinforcement Learning
Jialong Wu (Tsinghua University), Mingsheng Long (Tsinghua University)
Autonomous DrivingRobotic IntelligenceRecurrent Neural NetworkReinforcement LearningAuto EncoderWorld ModelVideo
🎯 What it does: Utilizing the rich wild video data from the internet for unsupervised pre-training, a world model is constructed that achieves higher sample efficiency in visual control tasks.
Pre-Training Protein Encoder via Siamese Sequence-Structure Diffusion Trajectory Prediction
Zuobai Zhang (Mila - Quebec AI Institute), Jian Tang (HEC Montreal)
Protein Structure PredictionDiffusion modelContrastive LearningBiomedical Data
🎯 What it does: Proposes two pre-training methods for protein encoders based on joint sequence-structure diffusion: DiffPreT and SiamDiff.
Precise asymptotic generalization for multiclass classification with overparameterized linear models
David Xing Wu, Anant Sahai (University of California Berkeley)
Classification
🎯 What it does: In high-dimensional linear multi-class models (Gaussian bi-level setting), the exact asymptotic generalization error and error rate of the minimum norm interpolation classifier are derived and proven, completing the generalization threshold conjecture proposed by Subramanian et al. in 2022.
Precision-Recall Divergence Optimization for Generative Modeling with GANs and Normalizing Flows
Alexandre Verine, Yann Chevaleyre
GenerationOptimizationFlow-based ModelGenerative Adversarial NetworkImage
🎯 What it does: This paper proposes a new PR-Divergence (D_λ-PR) to explicitly optimize the precision-recall trade-off of generative models, and implements user-specified λ values on GANs and Normalizing Flows.
Preconditioning Matters: Fast Global Convergence of Non-convex Matrix Factorization via Scaled Gradient Descent
Xixi Jia (Xidian University), Deyu Meng (Xian Jiaotong University)
Optimization
🎯 What it does: This paper studies the Low-Rank Matrix Factorization (LRMF) problem and proves that the preconditioned gradient descent (ScaledGD) and its alternating version (AltScaledGD) can achieve global convergence under both general random initialization and small-scale initialization, with the convergence rate independent of the matrix condition number.
PRED: Pre-training via Semantic Rendering on LiDAR Point Clouds
Hao Yang (Peking University), Liwei Wang (Peking University)
Object DetectionSegmentationAutonomous DrivingNeural Radiance FieldPoint Cloud
🎯 What it does: The PRED framework is proposed for pre-training outdoor LiDAR point clouds based on image semantic rendering, addressing issues of incomplete point clouds and occlusion.
Predict-then-Calibrate: A New Perspective of Robust Contextual LP
Chunlin Sun (Stanford University), Xiaocheng Li (Imperial College London)
OptimizationTabular
🎯 What it does: The paper proposes a 'predict-then-calibrate' framework for achieving risk-sensitive (robust) decision-making in linear programming with side information.
Predict, Refine, Synthesize: Self-Guiding Diffusion Models for Probabilistic Time Series Forecasting
Marcel Kollovieh (Technical University of Munich), Bernie Wang
GenerationData SynthesisOptimizationDiffusion modelTime Series
🎯 What it does: An unconditional diffusion model TSDiff is proposed, which achieves three main functions of time series forecasting, improvement, and data generation through observed self-guidance during the inference phase.
Predicting a Protein's Stability under a Million Mutations
Jeffrey Ouyang-Zhang (University of Texas at Austin), Philipp Kraehenbuehl
Protein Structure PredictionSupervised Fine-TuningBiomedical Data
🎯 What it does: A model named Mutate Everything has been constructed, capable of parallel prediction of thermodynamic stability changes (ΔΔG) for single and multiple mutations (including double mutations) in a single forward pass.
Predicting Global Label Relationship Matrix for Graph Neural Networks under Heterophily
Langzhang Liang (Harbin Institute of Technology), Irwin King (Chinese University of Hong Kong)
ClassificationAnomaly DetectionGraph Neural NetworkGraph
🎯 What it does: This paper proposes a Low-Rank Graph Neural Network (LRGNN) that predicts the label relationship matrix through robust low-rank matrix approximation and uses it for node classification in heterogeneous graphs.
Predicting mutational effects on protein-protein binding via a side-chain diffusion probabilistic model
Shiwei Liu (Institute of Computing Technology, Chinese Academy of Sciences), Haicang Zhang (Institute of Computing Technology, Chinese Academy of Sciences)
Protein Structure PredictionDiffusion modelScore-based ModelBiomedical Data
🎯 What it does: Utilizing a Riemannian diffusion probabilistic model to learn the generation and representation of protein side-chain conformations, and based on this representation, predicting the ΔΔG changes in protein-protein binding affinity due to mutations.
Prediction and Control in Continual Reinforcement Learning
Nishanth Anand (McGill University), Doina Precup (McGill University)
Reinforcement Learning
🎯 What it does: This paper proposes to decompose the value function into two components: permanent (slow) and transient (fast). Through the collaborative updates of these two components in continuous reinforcement learning, it achieves a balance between stability and plasticity, addressing the issues of forgetting and adaptation of the value function when tasks change.
PreDiff: Precipitation Nowcasting with Latent Diffusion Models
Zhihan Gao (Hong Kong University of Science and Technology), Bernie Wang
GenerationData SynthesisConvolutional Neural NetworkDiffusion modelMultimodalityTime Series
🎯 What it does: A two-stage forecasting framework called PreDiff is proposed, which generates multimodal forecasts using a latent diffusion model and introduces physical constraints through a knowledge alignment mechanism.
Preference-grounded Token-level Guidance for Language Model Fine-tuning
Shentao Yang (University of Texas), Mingyuan Zhou (University of Texas)
GenerationReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
🎯 What it does: An alternating training process is proposed to improve the fine-tuning of language models by transforming sequence-level preferences into token-level guidance.
Prefix-Tree Decoding for Predicting Mass Spectra from Molecules
Samuel Goldman (Massachusetts Institute of Technology), Connor W. Coley (Massachusetts Institute of Technology)
Graph Neural NetworkTransformer
🎯 What it does: The SCARF method is proposed, which splits mass spectrometry prediction into two steps: sub-formula generation (SCARF-Thread) and intensity prediction (SCARF-Weave). It efficiently enumerates sub-formulas using a prefix tree, enabling fast and physically reasonable predictions from molecules to mass spectrometry.
Pretraining task diversity and the emergence of non-Bayesian in-context learning for regression
Allan Raventos, Surya Ganguli (Stanford University)
TransformerTabular
🎯 What it does: This study investigates the impact of task diversity on in-context learning (ICL) of pre-trained Transformers, particularly exploring the task diversity threshold in linear regression tasks.
Primal-Attention: Self-attention through Asymmetric Kernel SVD in Primal Representation
Yingyi Chen (KU Leuven), Johan Suykens
Computational EfficiencyTransformerReinforcement LearningImageTime Series
🎯 What it does: This paper proposes viewing the self-attention of the Transformer as a singular value decomposition (KSVD) of an asymmetric kernel matrix, and implements the Primal-Attention mechanism in the original representation, avoiding expensive kernel matrix computations.
PrimDiffusion: Volumetric Primitives Diffusion for 3D Human Generation
Zhaoxi Chen (Nanyang Technological University), Ziwei Liu (Nanyang Technological University)
GenerationData SynthesisPose EstimationDiffusion modelImage
🎯 What it does: Proposes PrimDiffusion, the first 3D human generation framework based on diffusion models, which performs denoising diffusion directly on a set of voxel primitives.
Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision
Zhiqing Sun (Carnegie Mellon University), Chuang Gan (Massachusetts Institute of Technology)
GenerationExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: This paper proposes the SELF-ALIGN method, which self-aligns large language models from scratch to build the Dromedary AI assistant.
Principled Weight Initialisation for Input-Convex Neural Networks
Pieter-Jan Hoedt (Johannes Kepler University Linz), Günter Klambauer (Johannes Kepler University Linz)
OptimizationDrug DiscoveryConvolutional Neural NetworkImage
🎯 What it does: A weight initialization method for Input Convex Neural Networks (ICNN) is proposed, which enables stable signal propagation under non-negative weight conditions, thereby enhancing learning speed and generalization performance.
PRIOR: Personalized Prior for Reactivating the Information Overlooked in Federated Learning.
Mingjia Shi (Sichuan University), Jiancheng Lv (Sichuan University)
OptimizationFederated LearningMeta LearningImageText
🎯 What it does: This paper proposes a framework called pFedBreD that introduces personalized prior knowledge into federated learning. It utilizes Bayesian modeling, Bregman-Moreau envelopes, and Relaxed Mirror Descent (RMD) to achieve the injection and extraction of priors, and further personalizes local models through a meta-learning-based step size strategy.
PriorBand: Practical Hyperparameter Optimization in the Age of Deep Learning
Neeratyoy Mallik (University of Freiburg), Frank Hutter (University of Freiburg)
OptimizationHyperparameter SearchTabular
🎯 What it does: The PriorBand algorithm is proposed, which combines expert priors and low-cost proxy tasks to optimize hyperparameters of deep learning models within a multi-fidelity framework.
Prioritizing Samples in Reinforcement Learning with Reducible Loss
Shiva Kanth Sujit, Samira Ebrahimi Kahou (Mila Quebec AI Institute)
Reinforcement LearningTabular
🎯 What it does: This paper proposes a sampling method based on Reducible Loss (ReLo) for experience replay priority selection, aimed at more efficiently sampling experiences in reinforcement learning algorithms.
Privacy Amplification via Compression: Achieving the Optimal Privacy-Accuracy-Communication Trade-off in Distributed Mean Estimation
Wei-Ning Chen (Stanford University), Peter Kairouz (Google Research)
Federated LearningSafty and PrivacyTabular
🎯 What it does: This paper studies the use of compression techniques to achieve a three-way optimal trade-off between privacy, communication, and accuracy in federated learning and federated analysis, proposing a compressed privacy amplification scheme under central and shuffled differential privacy models.
Privacy Assessment on Reconstructed Images: Are Existing Evaluation Metrics Faithful to Human Perception?
Xiaoxiao Sun (Australian National University), Liang Zheng (Australian National University)
Safty and PrivacyConvolutional Neural NetworkContrastive LearningImage
🎯 What it does: This paper studies the evaluation method for image privacy leakage caused by reconstruction attacks by collecting human judgments on whether reconstructed images are recognizable.
Privacy Auditing with One (1) Training Run
Thomas Steinke (Google DeepMind), Matthew Jagielski (Google DeepMind)
Safty and PrivacyConvolutional Neural NetworkImage
🎯 What it does: A method is proposed for auditing differential privacy models that requires only a single training run, utilizing the parallelism of independently including or excluding multiple 'canary' samples in a single training session.
Private (Stochastic) Non-Convex Optimization Revisited: Second-Order Stationary Points and Excess Risks
Daogao Liu (University of Washington), Abhradeep Guha Thakurta
OptimizationSafty and Privacy
🎯 What it does: Revisits the non-convex optimization problem under differential privacy constraints and proposes a new framework that utilizes two types of gradient oracles to improve the efficiency of finding second-order stationary points.
Private Distribution Learning with Public Data: The View from Sample Compression
Shai Ben-David (University of Waterloo), Vikrant Singhal (University of Waterloo)
OptimizationSafty and Privacy
🎯 What it does: The study investigates how to perform distribution learning using differential privacy algorithms under the condition of simultaneously using public and private samples, and provides upper and lower bounds on the sample complexity of public and private sample sizes.
Private estimation algorithms for stochastic block models and mixture models
Hongjie Chen (ETH Zurich), Stefan Tiegel (ETH Zurich)
OptimizationSafty and Privacy
🎯 What it does: This paper proposes a general framework for designing differential privacy (DP) estimation algorithms in high-dimensional settings and applies it to two classic statistical learning problems: the Stochastic Block Model (SBM) and the Spherical Gaussian Mixture Model (GMM). It achieves statistical accuracy almost identical to that of the optimal non-private algorithms while maintaining polynomial time complexity.
Private Everlasting Prediction
Moni Naor (Weizmann Institute of Science), Chao Yan (Georgetown University)
Safty and Privacy
🎯 What it does: This paper proposes a 'Private Perpetual Prediction' model that can continuously provide answers to an infinitely long sequence of prediction queries without disclosing the model, while simultaneously satisfying differential privacy for the training samples and all query points.
Private Federated Frequency Estimation: Adapting to the Hardness of the Instance
Jingfeng Wu (Johns Hopkins University), Vladimir Braverman (Rice University)
Federated LearningSafty and PrivacyGaussian SplattingText
🎯 What it does: In response to the Federated Frequency Estimation (FFE) problem, the authors improved the CountSketch algorithm, proposed single-round and multi-round instance-adaptive schemes, and designed a new HybridSketch algorithm; they also provided a two-stage adaptive parameter tuning method and implemented differential privacy within the Secure Sum (SecSum) framework.
Probabilistic Exponential Integrators
Nathanael Bosch (University of Tübingen), Filip Tronarp (Lund University)
Time SeriesPhysics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: A new class of Probabilistic Exponential Integrators is proposed for efficient simulation and uncertainty quantification of stiff semi-linear ordinary differential equations.
Probabilistic Inference in Reinforcement Learning Done Right
Jean Tarbouriech (Google DeepMind), Brendan O'Donoghue (Google DeepMind)
Reinforcement LearningTabular
🎯 What it does: This paper proposes a new Bayesian inference method for optimal policy generation in reinforcement learning, focusing on the posterior probability P Γ ⋆ of state-action pairs being visited under the optimal policy.
Probabilistic Invariant Learning with Randomized Linear Classifiers
Leonardo Cotta (Vector Institute), Chris J. Maddison (University of Toronto)
ClassificationOptimizationComputational EfficiencyGraph
🎯 What it does: This paper proposes a Random Linear Classifier (RLC) for binary classification tasks, establishing a theoretical framework for probabilistic universal approximation and invariance, and designing specific RLC models for typical invariance tasks such as sets, shapes, and spheres.
Probabilistic inverse optimal control for non-linear partially observable systems disentangles perceptual uncertainty and behavioral costs
Dominik Straub (Technische Universität Darmstadt), Constantin A. Rothkopf (Technische Universität Darmstadt)
OptimizationRobotic IntelligenceReinforcement LearningTime Series
🎯 What it does: A probabilistic inverse optimal control method for partially observable nonlinear systems is proposed, which can infer the agent's cost function and perception/motion noise parameters from the observed state trajectories.
Probabilistic Weight Fixing: Large-scale training of neural network weight uncertainties for quantisation.
Chris Subia-Waud, Srinandan Dasmahapatra (University of Southampton)
CompressionOptimizationConvolutional Neural NetworkTransformerImage
🎯 What it does: This paper proposes PWFN, a probabilistic framework that combines Bayesian networks with weight-sharing clustering for weight quantization and compression of large-scale neural networks.
PrObeD: Proactive Object Detection Wrapper
Vishal Asnani (Michigan State University), Xiaoming Liu (Michigan State University)
Object DetectionSupervised Fine-TuningImage
🎯 What it does: Proposes an active object detection wrapper PrObeD, which encrypts input images using learnable image-dependent templates and then fine-tunes the detector to enhance detection performance.