ICML 2024 Papers — Page 19
International Conference on Machine Learning · 2610 papers
Position: Mission Critical – Satellite Data is a Distinct Modality in Machine Learning
Esther Rolf (Harvard University), Hannah Kerner (Arizona State University)
TransformerImageTime SeriesReview/Survey PaperBenchmark
🎯 What it does: This paper argues that satellite data is a unique data modality for machine learning, highlighting its significant differences from traditional data such as images and text in terms of spatial scale, spectral channels, time series, and annotation sparsity. It proposes a research agenda and methodological framework to address these differences.
Position: Near to Mid-term Risks and Opportunities of Open-Source Generative AI
Francisco Eiras (University of Oxford), Jakob Nicolaus Foerster
GenerationLarge Language ModelTextReview/Survey PaperBenchmark
🎯 What it does: This paper proposes a generative AI openness classification system, conducts an open evaluation of 40 mid-term large language models, and systematically analyzes the differences between open and closed-source models in terms of risks, opportunities, transparency, innovation, and economic and social impacts, ultimately providing five recommendations for responsible open release.
Position: On the Possibilities of AI-Generated Text Detection
Souradip Chakraborty (University of Maryland), Furong Huang (University of Maryland)
ClassificationGenerationTransformerLarge Language ModelText
🎯 What it does: The feasibility of distinguishing between human-written texts and texts generated by large language models has been studied, with both theoretical and empirical evidence provided.
Position: On the Societal Impact of Open Foundation Models
Sayash Kapoor (Princeton University), Arvind Narayanan (Princeton University)
Review/Survey Paper
🎯 What it does: This paper conducts a systematic analysis of the social impact of open foundational models, proposes five unique attributes, constructs a risk assessment framework, and discusses its benefits and risks.
Position: Open-Endedness is Essential for Artificial Superhuman Intelligence
Edward Hughes (Google DeepMind), Tim Rocktäschel
Reinforcement LearningReview/Survey Paper
🎯 What it does: A formal definition of open systems based on the observer perspective is proposed, and it is explained that the combination of openness and foundational models is the key path to achieving artificial superhuman intelligence.
Position: Opportunities Exist for Machine Learning in Magnetic Fusion Energy
Lucas Spangher (Massachusetts Institute of Technology), Cristina Rea (University of Maryland)
Convolutional Neural NetworkRecurrent Neural NetworkGraph Neural NetworkReinforcement LearningAuto EncoderTime SeriesReview/Survey PaperPhysics Related
🎯 What it does: This paper discusses six key challenges that machine learning can address in magnetic confinement fusion energy and looks ahead to future opportunities.
Position: Optimization in SciML Should Employ the Function Space Geometry
Johannes Müller (RWTH Aachen University), Marius Zeinhofer (Simula Research Laboratory)
OptimizationTabularPhysics Related
🎯 What it does: A unified framework is proposed and validated for solving optimization problems in scientific machine learning (SciML) first in function space and then discretizing it into the neural network parameter space;
Position: Quo Vadis, Unsupervised Time Series Anomaly Detection?
M. Saquib Sarfraz (Mercedes-Benz Tech Innovation), Marios Koulakis (Karlsruhe Institute of Technology)
Anomaly DetectionKnowledge DistillationGraph Neural NetworkTransformerTime SeriesReview/Survey PaperBenchmark
🎯 What it does: A critical analysis of the current state of research on time series anomaly detection is conducted, proposing and implementing various simple baselines (such as PCA reconstruction error, 1-NN distance, L2 norm, and a single layer MLP/MLPMixer/Transformer/GCN-LSTM), and comparing them with existing deep learning methods under standard and modified evaluation metrics.
Position: Reinforcement Learning in Dynamic Treatment Regimes Needs Critical Reexamination
Zhiyao Luo (University of Oxford), Tingting Zhu (University of Oxford)
Reinforcement LearningBiomedical DataElectronic Health Records
🎯 What it does: This paper systematically evaluates the performance of various offline reinforcement learning (RL) algorithms in dynamic treatment regimes (DTR) through 17,550 experiments on a sepsis dataset, focusing on the diversity of policy evaluation methods, reward design, and benchmark settings and their impact on the results.
Position: Relational Deep Learning - Graph Representation Learning on Relational Databases
Matthias Fey (Kumo.AI), Jure Leskovec (Stanford University)
Representation LearningGraph Neural NetworkGraphTabularTime SeriesBenchmark
🎯 What it does: This paper proposes the Relational Deep Learning (RDL) framework, which transforms relational databases into temporal heterogeneous graphs (Relational Entity Graphs) and achieves end-to-end predictions through Graph Neural Networks (GNN), eliminating traditional manual feature engineering.
Position: Rethinking Post-Hoc Search-Based Neural Approaches for Solving Large-Scale Traveling Salesman Problems
Yifan Xia (Nanjing University), Jiang Bian (Microsoft Research Asia)
OptimizationReinforcement LearningTabular
🎯 What it does: This paper systematically evaluates a machine learning-based heatmap-guided MCTS method for solving large-scale TSP and proposes a simple and effective SoftDist baseline and Score metric.
Position: Scaling Simulation is Neither Necessary Nor Sufficient for In-the-Wild Robot Manipulation
Homanga Bharadhwaj (Carnegie Mellon University)
Domain AdaptationRobotic IntelligenceVideo
🎯 What it does: This paper proposes and argues for the necessity and sufficiency of expanding simulation in real robot operations through systematic analysis and comparison, and suggests a shift towards training methods based on real data and passive video.
Position: Scarce Resource Allocations That Rely On Machine Learning Should Be Randomized
Shomik Jain (Massachusetts Institute of Technology), Ashia Camage Wilson
Tabular
🎯 What it does: This paper argues that when allocating scarce resources, the use of machine learning should incorporate randomization rather than complete determinism to better reflect individuals' claims to resources.
Position: Social Choice Should Guide AI Alignment in Dealing with Diverse Human Feedback
Vincent Conitzer (Carnegie Mellon University), William S. Zwicker (Union College)
Reinforcement Learning from Human FeedbackReinforcement Learning
🎯 What it does: This paper proposes the application of social choice theory to AI alignment, particularly in addressing the aggregation and decision-making issues related to diverse human feedback (such as RLHF and Constitutional AI), and introduces two new frameworks: RLCHF (Reinforcement Learning based on Social Welfare Functions) and Simulated Collective Decision-Making (embedding voting rules into training or inference).
Position: Social Environment Design Should be Further Developed for AI-based Policy-Making
Edwin Zhang (Harvard University), Yiling Chen (Harvard University)
OptimizationReinforcement Learning
🎯 What it does: A Social Environment Design framework is proposed, integrating voting, Stackelberg-Markov games, and POMG to achieve AI-driven policy-making.
Position: Standardization of Behavioral Use Clauses is Necessary for the Adoption of Responsible Licensing of AI
Daniel McDuff (University of Washington), Danish Contractor
🎯 What it does: Analyze and quantify the adoption of AI licenses (including behavioral usage terms) in AI models and software repositories, assess the differences in their terms, and propose standardization and tooling suggestions.
Position: Stop Making Unscientific AGI Performance Claims
Patrick Altmeyer (Delft University of Technology), Cynthia C. S. Liem (Delft University of Technology)
TransformerTabularTime SeriesFinance Related
🎯 What it does: This paper examines and questions the practice of misinterpreting correlations in the latent space of models as artificial general intelligence (AGI) or 'understanding' capabilities, arguing that even simple models can obtain useful information through linear probing without training, thus indicating that these correlations do not prove the model possesses AGI.
Position: Technical Research and Talent is Needed for Effective AI Governance
Anka Reuel (Stanford University), Trond Arne Undheim (Oxford University)
Review/Survey Paper
🎯 What it does: This paper identifies the gap between technology and policy objectives by reviewing AI governance policy documents from the EU, the US, and China. It proposes the necessary technological research and talent supplementation across four dimensions: data, computation, models, and deployment, and discusses the technical capability gaps in governance institutions.
Position: Tensor Networks are a Valuable Asset for Green AI
Eva Memmel (Delft University of Technology), kim batselier
CompressionComputational EfficiencyTabularReview/Survey Paper
🎯 What it does: The paper proposes the combination of tensor networks and green AI, demonstrating their potential for model compression and energy efficiency, and reviews relevant cases.
Position: The Causal Revolution Needs Scientific Pragmatism
Joshua R. Loftus (London School of Economics)
Review/Survey Paper
🎯 What it does: This paper proposes viewing causal models as tools, guided by the philosophy of 'scientific pragmatism', to promote a broader application of the causal revolution in machine learning and scientific research.
Position: The No Free Lunch Theorem, Kolmogorov Complexity, and the Role of Inductive Biases in Machine Learning
Micah Goldblum (New York University), Andrew Gordon Wilson (New York University)
ClassificationCompressionConvolutional Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningImageTabular
🎯 What it does: This paper explains the relationship between the 'No Free Lunch' theorem and Kolmogorov complexity from both theoretical and experimental perspectives, and demonstrates the low complexity preference of neural network models on real data, further proving the feasibility of a general learner that can generalize to various data types and different sample sizes.
Position: The Platonic Representation Hypothesis
Minyoung Huh (Massachusetts Institute of Technology), Phillip Isola (Massachusetts Institute of Technology)
Representation LearningContrastive LearningImageTextMultimodality
🎯 What it does: This study investigates the phenomenon of representation convergence in deep networks across different tasks, scales, and modalities, proposing the 'Platonic Representation Hypothesis.'
Position: The Reasonable Person Standard for AI
Sunayana Rane (Princeton University)
🎯 What it does: The paper discusses how the reasonable person standard in American law can be applied to the governance of artificial intelligence (AI) behavior, suggesting that the reasonable person standard can provide useful guidance for AI model behavior.
Position: Topological Deep Learning is the New Frontier for Relational Learning
Theodore Papamarkou (University of Manchester), Ghada Zamzmi (University of South Florida)
Graph Neural NetworkGraphReview/Survey PaperBenchmark
🎯 What it does: This paper presents Topological Deep Learning (TDL) as the next frontier of relational learning in the form of a position paper, systematically outlining its potential advantages in the fields of graphs, geometry, and topology, and listing open questions and future research directions.
Position: Towards Implicit Prompt For Text-To-Image Models
Yue Yang (Shanghai Jiao Tong University), Ping Luo (Hong Kong University)
GenerationTransformerLarge Language ModelPrompt EngineeringDiffusion modelImageTextMultimodalityBenchmark
🎯 What it does: Proposes the concept of implicit prompts, constructs the ImplicitBench benchmark, and conducts a systematic evaluation on six mainstream text-to-image models.
Position: Towards Unified Alignment Between Agents, Humans, and Environment
Zonghan Yang (Tsinghua University), Yang Liu (Tsinghua University)
Recommendation SystemReinforcement Learning from Human FeedbackReinforcement LearningAgentic AITabular
🎯 What it does: This paper proposes the UA 2 principle, constructing an agent model that aligns human intentions, environmental dynamics, and self-constraints in a unified three-dimensional manner, and validates its effectiveness on the modified WebShop.
Position: TrustLLM: Trustworthiness in Large Language Models
Yue Huang (University of Notre Dame), Yue Zhao (University of Southern California)
Explainability and InterpretabilityLarge Language ModelPrompt EngineeringTextBenchmark
🎯 What it does: The TRUSTLLM framework is proposed for the systematic evaluation of the trustworthiness of large language models (LLMs), covering eight dimensions of principles and six-dimensional benchmarks, and assessing the performance of 16 mainstream LLMs.
Position: Understanding LLMs Requires More Than Statistical Generalization
Patrik Reizinger (Max Planck Institute for Intelligent Systems), Ferenc Huszár (University of Cambridge)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper proposes a shift from the traditional interpolation paradigm to a saturation paradigm in the research of large language models (LLMs), exploring the identifiability and inductive biases of autoregressive (AR) language models, and demonstrating the diversity of model behavior under the same minimum loss through three case studies (rule extrapolation, context learning, and fine-tuning feasibility).
Position: Video as the New Language for Real-World Decision Making
Sherry Yang (Google DeepMind), Dale Schuurmans (Google DeepMind)
GenerationRobotic IntelligenceTransformerReinforcement LearningDiffusion modelVideoReview/Survey Paper
🎯 What it does: Proposes to view video generation as a new language for the physical world and reviews its applications in tasks such as decision-making, planning, and simulation.
Position: What Can Large Language Models Tell Us about Time Series Analysis
Ming Jin (Griffith University), Qingsong Wen (Squirrel AI)
Anomaly DetectionExplainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringTime Series
🎯 What it does: This paper proposes and elaborates on three key roles of large language models (LLMs) in time series analysis—enhancer, predictor, and intelligent agent—and systematically reviews existing research along with a roadmap for future studies.
Position: What makes an image realistic?
Lucas Theis (Google DeepMind)
🎯 What it does: This paper proposes defining image realism from the perspective of randomness deficiency and systematically discusses the shortcomings of existing methods such as probability, typicality, and divergence, constructing a universal critic framework that does not rely on adversarial training.
Position: Why Tabular Foundation Models Should Be a Research Priority
Boris van Breugel (University of Cambridge), Mihaela van der Schaar (Alan Turing Institute)
TransformerTabular
🎯 What it does: This paper proposes focusing on Tabular Foundation Models (Large-scale Tabular Models, LTM) and discusses their potential value and challenges.
Position: Why We Must Rethink Empirical Research in Machine Learning
Moritz Herrmann (Ludwig Maximilian University of Munich), Bernd Bischl (Ludwig Maximilian University of Munich)
Review/Survey Paper
🎯 What it does: This paper reviews and critiques the current state of empirical research in machine learning, pointing out common issues such as reproducibility problems, biased experimental design, and imbalanced method comparisons. It suggests improvements such as shifting from confirmatory research to exploratory research, strengthening neutral comparisons and replication studies, and enhancing infrastructure and education.
Position: Will we run out of data? Limits of LLM scaling based on human-generated data
Pablo Villalobos (Epoch AI), Marius Hobbhahn
Large Language ModelText
🎯 What it does: This paper predicts the scale of publicly available human-generated text data required for training large language models (LLMs) and its availability by constructing a database inventory and a training data demand model. It estimates that if current trends continue, the available data will be exhausted between 2026 and 2032.
Positional Knowledge is All You Need: Position-induced Transformer (PiT) for Operator Learning
Junfeng CHEN, Kailiang Wu (Southern University of Science and Technology)
OptimizationComputational EfficiencyTransformerTabularBenchmarkPhysics Related
🎯 What it does: This paper proposes a position-attention-based Transformer model, PiT, for operator learning of PDEs;
Positive and Unlabeled Learning with Controlled Probability Boundary Fence
Changchun Li (Jilin University), Jihong Ouyang (Jilin University)
ClassificationConvolutional Neural NetworkImageAlzheimer's Disease
🎯 What it does: A two-stage positive and negative label learning method PUL-CPBF is proposed, which utilizes different penalty coefficients to train weak classifiers to form a probability boundary fence, and then generates random labels for unlabeled samples to self-train a strong classifier.
Positive Concave Deep Equilibrium Models
Mateusz Gabor (Wrocław University of Science and Technology), Renato L. G. Cavalcante (Fraunhofer Heinrich Hertz Institute)
ClassificationImage
🎯 What it does: A positive concave depth equilibrium model (pcDEQ) is proposed, which ensures the existence and uniqueness of fixed points by constraining weights to be non-negative and activation functions to be positive concave, while guaranteeing geometric convergence of standard fixed point iteration.
Post-hoc Part-Prototype Networks
Andong Tan (Hong Kong University of Science and Technology), Hao Chen (Hong Kong University of Science and Technology)
Explainability and InterpretabilityConvolutional Neural NetworkTransformerImage
🎯 What it does: A Post-hoc Part-Prototype Network is proposed, which decomposes the classification head of a trained model into interpretable part prototypes, thereby explaining both the regions the model focuses on and the visual features it relies on, while ensuring the same predictive performance as the original model.
Posterior Sampling-Based Bayesian Optimization with Tighter Bayesian Regret Bounds
Shion Takeno (Nagoya University), Ichiro Takeuchi (Nagoya University)
Optimization
🎯 What it does: This paper studies the Thompson Sampling (TS) and the sampling-based improved PI (PIMS) algorithm in posterior sampling Bayesian optimization, providing a tighter upper bound on Bayesian cumulative return (BCR) and proving its theoretical equivalence to methods such as IRGP-UCB.
Potential Based Diffusion Motion Planning
Yunhao Luo (Brown University), Yilun Du (Massachusetts Institute of Technology)
OptimizationRobotic IntelligenceDiffusion modelSequential
🎯 What it does: This paper proposes a motion planning method that utilizes diffusion models to learn potential energy functions, generating collision-safe paths through gradient optimization on the potential landscape, and supporting the synthesis of multiple constraints and path correction.
PPFLOW: Target-Aware Peptide Design with Torsional Flow Matching
Haitao Lin (Zhejiang University), Stan Z. Li (Westlake University)
OptimizationDrug DiscoveryFlow-based ModelNeural Radiance FieldBiomedical DataBenchmarkOrdinary Differential Equation
🎯 What it does: A target-oriented peptide molecular design method called PPFLOW is proposed, which utilizes Riemannian flow matching to model the torsion angles of the peptide backbone on a circular manifold, achieving conditional generation. At the same time, a large-scale protein-peptide binding database, PPBench2024, has been constructed to support training.
Practical Hamiltonian Monte Carlo on Riemannian Manifolds via Relativity Theory
Kai Xu (MIT IBM Watson AI Lab), Hong Ge (University of Cambridge)
Physics RelatedStochastic Differential Equation
🎯 What it does: This paper proposes a new Hamiltonian Monte Carlo (HMC) method called General Relativistic Hamiltonian Monte Carlo (GR-HMC), aimed at stabilizing numerical integration on Riemannian manifolds by constructing position-dependent relativistic momentum distributions.
Practical Performance Guarantees for Pipelined DNN Inference
Aaron Archer (Google), Prakash Prabhu (Google)
OptimizationComputational EfficiencyTransformerText
🎯 What it does: This paper studies the problem of dividing deep neural networks into k pipeline stages during the inference phase, proposing the use of Mixed Integer Programming (MIP) to generate instance lower bounds, and presents an efficient partitioning algorithm called SliceGraph based on dynamic programming and genetic algorithms.
Pragmatic Feature Preferences: Learning Reward-Relevant Preferences from Human Input
Andi Peng (Massachusetts Institute of Technology), David Abel (Google DeepMind)
Recommendation SystemReinforcement Learning from Human FeedbackReinforcement LearningText
🎯 What it does: This study investigates how to learn more accurate reward models by having users provide feature-level preferences and their language descriptions, enhancing user alignment.
PrE-Text: Training Language Models on Private Federated Data in the Age of LLMs
Charlie Hou (Carnegie Mellon University), Daniel Lazar (Meta)
Data SynthesisFederated LearningSafty and PrivacyComputational EfficiencyTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: The PrE-Text method is proposed, which generates synthetic text through differential privacy in a federated learning environment, serving as a replacement for traditional on-device training to train small and large models.
Pre-Training Protein Bi-level Representation Through Span Mask Strategy On 3D Protein Chains
Zhao jiale, Shuqi Lu (DP Technology)
Drug DiscoveryProtein Structure PredictionGraph Neural NetworkBiomedical Data
🎯 What it does: In this study, the authors proposed the Vabs-Net model and designed the Span Mask Protein Chain (SMPC) pre-training strategy, aiming to learn representations at both the residue and atom levels simultaneously, and evaluated it on various downstream tasks (function prediction, binding site prediction, molecular docking, etc.).
Precise Accuracy / Robustness Tradeoffs in Regression: Case of General Norms
Elvis Dohmatob (Meta), Meyer Scetbon (Microsoft Research)
OptimizationAdversarial AttackTabular
🎯 What it does: This study investigates the balance between accuracy and robustness of linear regression under adversarial attacks, providing an exact optimal robustness formula and phase transition diagram.
Predicting and Interpreting Energy Barriers of Metallic Glasses with Graph Neural Networks
Haoyu Li (University of California), Yizhou Sun (University of California)
Graph Neural NetworkGraphPhysics Related
🎯 What it does: The study investigates the prediction and explanation of energy barriers in metallic glasses, constructing a SymGNN model based on graph neural networks;
Predicting Dose-Response Curves with Deep Neural Networks
Pedro Alonso Campana, Tobias Scheffer (University of Potsdam)
Drug DiscoveryGraph Neural NetworkBiomedical Data
🎯 What it does: The ARCANet deep neural network is proposed to predict dose-response curves using the interaction embeddings of drug molecules and cell transcriptomes.
Predicting Lagrangian Multipliers for Mixed Integer Linear Programs
Francesco Demelas (Universite Sorbonne Paris Nord), Axel Parmentier (CERMICS Ecole des Ponts)
OptimizationGraph Neural NetworkTabular
🎯 What it does: Using a deep learning framework to predict Lagrange multipliers for mixed-integer linear programming, aiming to improve the quality of the Lagrange lower bound and accelerate the subsequent optimization process.
Prediction Accuracy of Learning in Games : Follow-the-Regularized-Leader meets Heisenberg
Yi Feng (Shanghai University of Finance and Economics), Xiao Wang (Shanghai University of Finance and Economics)
Reinforcement LearningTabularPhysics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Analyzes the prediction accuracy of Follow-the-Regularized-Leader (FTRL) learning dynamics in zero-sum games, explores the observational uncertainty caused by random initialization, and provides the covariance growth rates of continuous and discrete (Euler and Symplectic) schemes as well as Heisenberg-type inequalities using covariance as a metric.
Prediction-powered Generalization of Causal Inferences
Ilker Demirel (Massachusetts Institute of Technology), David Sontag (Massachusetts Institute of Technology)
Machine LearningTabular
🎯 What it does: This paper studies how to combine limited randomized controlled trials (RCTs) with large-scale observational data to enhance the external validity of causal inference for the target population.
Predictive Coding beyond Correlations
Tommaso Salvatori (VERSES Research Lab), Thomas Lukasiewicz (Institute of Logic and Computation)
ClassificationGenerationGraph Neural NetworkContrastive LearningImage
🎯 What it does: This paper proposes a predictive coding-based model that achieves causal inference, intervention, and counterfactual inference without disrupting the graph structure, and automatically discovers causal structures from observational data through structural learning, enhancing image classification and generation performance.
Predictive Dynamic Fusion
Bing Cao (Tianjin University), Qinghua Hu (Tianjin University)
ClassificationRecognitionMultimodality
🎯 What it does: A prediction-based dynamic multimodal fusion framework (PDF) is proposed, achieving robust fusion through the prediction of co-belief and relative calibration (RC);
Predictive Linear Online Tracking for Unknown Targets
Anastasios Tsiamis (ETH Zurich), John Lygeros (ETH Zurich)
Object TrackingReinforcement LearningTime Series
🎯 What it does: This paper studies the problem of online tracking of unknown targets and proposes a new algorithm called Predictive Linear Online Tracking (PLOT) for tracking dynamic targets in non-stationary conditions.
Predictive Performance Comparison of Decision Policies Under Confounding
Luke Guerdan (Carnegie Mellon University), Steven Wu
Biomedical DataElectronic Health Records
🎯 What it does: A method is proposed to compare the predictive performance of decision policies in the presence of unobserved confounding factors, utilizing bias-robust estimation and non-parametric machine learning to achieve δ-regret intervals.
Preference Fine-Tuning of LLMs Should Leverage Suboptimal, On-Policy Data
Fahim Tajwar (Carnegie Mellon University), Aviral Kumar (Google DeepMind)
OptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningContrastive LearningText
🎯 What it does: This paper evaluates and compares various preference fine-tuning methods through systematic experiments and theoretical analysis, focusing on whether online sampling (on-policy) and negative gradient are needed to enhance the alignment effect of large language models.
Preference Optimization for Molecule Synthesis with Conditional Residual Energy-based Models
Songtao Liu (Pennsylvania State University), Peng Liu
OptimizationDrug DiscoveryTransformerTabular
🎯 What it does: This paper proposes a framework based on the Conditional Residual Energy Model (CREBM), which optimizes the quality of synthetic routes by adjusting the energy function of the synthetic route on top of the existing single-step restoration prediction + search strategy.
Premier-TACO is a Few-Shot Policy Learner: Pretraining Multitask Representation via Temporal Action-Driven Contrastive Loss
Ruijie Zheng (University of Maryland), Furong Huang (Microsoft Research)
Robotic IntelligenceConvolutional Neural NetworkReinforcement LearningContrastive LearningSequential
🎯 What it does: A multi-task offline pre-training framework named Premier-TACO is proposed, which learns general visual representations through time-action driven contrastive learning, thereby enhancing the efficiency of few-shot policy learning.
Premise Order Matters in Reasoning with Large Language Models
Xinyun Chen (Google DeepMind), Denny Zhou (Google DeepMind)
TransformerLarge Language ModelTextBenchmark
🎯 What it does: This study investigates the impact of premise order on the reasoning performance of large language models (LLMs) and creates the R-GSM benchmark based on GSM8K to validate the order effect in mathematical reasoning.
Preventing Model Collapse in Gaussian Process Latent Variable Models
Ying Li (University of Hong Kong), Michael Minyi Zhang (University of Hong Kong)
OptimizationRepresentation LearningImageTabular
🎯 What it does: This paper proposes a novel Gaussian Process Latent Variable Model (GPLVM) - advised RFLVM, which prevents model collapse by learning projection noise and using a Spectral Mixture kernel combined with differentiable Random Fourier Features (RFF) to achieve fully differentiable variational inference.
Pricing with Contextual Elasticity and Heteroscedastic Valuation
Jianyu Xu (University of California), Yu-Xiang Wang (University of California)
OptimizationReinforcement LearningTabularTime Series
🎯 What it does: This paper proposes a context pricing model with feature-related price elasticity and presents an online algorithm based on price perturbation (PwP) that achieves low-scheduling learning for this model.
Principled Gradient-Based MCMC for Conditional Sampling of Text
Li Du (Johns Hopkins University), Ryan Cotterell (ETH Zurich)
GenerationOptimizationTransformerLarge Language ModelText
🎯 What it does: This paper proposes two new gradient-based MCMC samplers (p-NCG and GwL) that can correctly sample from discrete text energy models without changing the target energy distribution.
Principled Penalty-based Methods for Bilevel Reinforcement Learning and RLHF
Han Shen (Rensselaer Polytechnic Institute), Tianyi Chen (Yale University)
Reinforcement Learning from Human FeedbackReinforcement LearningSequential
🎯 What it does: A penalty-based bi-level reinforcement learning algorithm is proposed, along with two feasible penalty functions: value penalty and Bellman penalty.
Principled Preferential Bayesian Optimization
Wenjie Xu (EPFL), Colin Jones
OptimizationTabular
🎯 What it does: An optimistic preference Bayesian optimization algorithm based on likelihood ratio confidence sets (POP-BO) is proposed, along with an upper bound on cumulative loss and convergence rate from an information theory perspective.
Prior Mismatch and Adaptation in PnP-ADMM with a Nonconvex Convergence Analysis
Shirin Shoushtari (Washington University in St. Louis), Ulugbek S. Kamilov (Washington University in St. Louis)
RestorationDomain AdaptationSupervised Fine-TuningImage
🎯 What it does: This study investigates the convergence and performance of PnP-ADMM under the condition of prior distribution mismatch (different source and target domains) and provides an error upper bound.
PriorBoost: An Adaptive Algorithm for Learning from Aggregate Responses
Adel Javanmard (University of Southern California), Vahab Mirrokni (Google Research)
🎯 What it does: This paper studies how to construct bags in aggregate response learning and proposes an adaptive PriorBoost algorithm to iteratively improve bag partitioning and enhance model quality using prior information.
PRISE: LLM-Style Sequence Compression for Learning Temporal Action Abstractions in Control
Ruijie Zheng (University of Maryland), Andrey Kolobov (Microsoft Research)
CompressionRobotic IntelligenceTransformerLarge Language ModelReinforcement LearningSequential
🎯 What it does: This paper proposes a method called PRISE that utilizes Byte Pair Encoding (BPE) and vector quantization to compress continuous action sequences into variable-length action abstractions (skills), and achieves multi-task learning and few-shot transfer for robot manipulation through behavior cloning.
Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models
Siddharth Karamcheti (Stanford University), Dorsa Sadigh (Stanford University)
Computational EfficiencyRepresentation LearningTransformerLarge Language ModelVision Language ModelImageTextMultimodality
🎯 What it does: Conducted systematic experiments on the design space of Visual Language Models (VLM), built a unified evaluation suite, and proposed the PRISM model series.
Privacy Attacks in Decentralized Learning
Abdellah El Mrini (École Polytechnique Fédérale de Lausanne), Aurélien Bellet (Inria)
Federated LearningSafty and PrivacyAdversarial AttackGraphTabular
🎯 What it does: This paper studies how 'honest-curious' attackers in decentralized learning can reconstruct private data of non-neighboring nodes by analyzing the information flow in Gossip averaging and decentralized gradient descent (D-GD).
Privacy Backdoors: Stealing Data with Corrupted Pretrained Models
Shanglun Feng (ETH Zurich), Florian Tramèr (ETH Zurich)
Safty and PrivacyTransformerImageText
🎯 What it does: By tampering with the weights of a pre-trained model to implant a privacy backdoor, it is possible to write a single training sample into the model weights during subsequent fine-tuning, achieving perfect membership inference and data reconstruction in both white-box and black-box environments.
Privacy Preserving Adaptive Experiment Design
Jiachun Li (Massachusetts Institute of Technology), David Simchi-Levi (Massachusetts Institute of Technology)
OptimizationSafty and PrivacyTabular
🎯 What it does: This paper designs an adaptive experimental scheme under the multi-armed bandit framework with privacy constraints, aiming to minimize both social welfare loss (cumulative loss) and the estimation error of the conditional average treatment effect (CATE).
Privacy Profiles for Private Selection
Antti Koskela (Nokia Bell Labs), Yu-Xiang Wang (UC San Diego)
Safty and PrivacyHyperparameter SearchTabularSequential
🎯 What it does: This paper proposes a direct privacy analysis of private selection mechanisms (such as Report Noisy Max and private hyperparameter tuning) using privacy profiles for precise (ε,δ)-DP privacy analysis, and provides a general upper bound formula for numerical privacy accounting that can be directly used.
Privacy-Preserving Data Release Leveraging Optimal Transport and Particle Gradient Descent
Konstantin Donhauser (ETH Zurich), Fanny Yang (ETH Zurich)
Data SynthesisOptimizationSafty and PrivacyTabular
🎯 What it does: A differential privacy table data synthesis method based on Particle Gradient Descent (PrivPGD) is proposed, which can directly optimize in the particle space to generate high-quality synthetic data.
Privacy-Preserving Embedding via Look-up Table Evaluation with Fully Homomorphic Encryption
Jae-yun Kim (Seoul National University), Jung Hee Cheon (Seoul National University)
Safty and PrivacyComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: This paper studies the encrypted evaluation of the word embedding layer using CKKS homomorphic encryption, proposing a high-precision Encrypted Indicator Function (EIF) and a CodedHELUT algorithm based on compressed coding to achieve encrypted lookups for large word vector tables.
Privacy-Preserving Instructions for Aligning Large Language Models
Da Yu (Sun Yat-sen University), Zheng Xu (Google Research)
Safty and PrivacyReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Proposes a method to generate synthetic instructions using differential privacy to replace real instructions, enabling alignment training of LLMs while maintaining privacy security.
Private and Federated Stochastic Convex Optimization: Efficient Strategies for Centralized Systems
Roie Reshef (Technion), Kfir Yehuda Levy (Technion)
OptimizationFederated LearningSafty and PrivacyTabular
🎯 What it does: A differential privacy federated learning algorithm for trusted and untrusted servers is proposed in centralized systems, achieving optimal convergence rate with linear complexity based on µ2-SGD.
Private Gradient Descent for Linear Regression: Tighter Error Bounds and Instance-Specific Uncertainty Estimation
Gavin R Brown, Abhradeep Guha Thakurta
OptimizationSafty and PrivacyTabular
🎯 What it does: This paper provides an improved analysis of the application of the standard differential privacy gradient descent algorithm in least squares linear regression, particularly describing the iterative distribution at each time step under squared error loss.
Private Heterogeneous Federated Learning Without a Trusted Server Revisited: Error-Optimal and Communication-Efficient Algorithms for Convex Losses
Changyu Gao (University of Wisconsin Madison), Stephen Wright
OptimizationFederated LearningSafty and PrivacyImage
🎯 What it does: This paper proposes the ISRL-DP algorithm for cross-island federated learning of heterogeneous data in an untrusted server environment, and proves its optimal risk, communication, and gradient complexity under convex loss.
Private Truly-Everlasting Robust-Prediction
Uri Stemmer (Tel Aviv University)
OptimizationSafty and Privacy
🎯 What it does: A framework called 'Private Eternal Robust Prediction' (PERP) is proposed, along with an efficient implementation for axis-aligned rectangles and decision trees.
Private Vector Mean Estimation in the Shuffle Model: Optimal Rates Require Many Messages
Hilal Asi (Apple), Samson Zhou (Texas A&M University)
OptimizationSafty and PrivacyTabularFinance Related
🎯 What it does: This paper studies the problem of private vector mean estimation under the shuffling privacy model and proposes a new multi-message protocol that can achieve optimal error with the number of messages sent by each user being O(min(nε, d^2)).
Privately Learning Smooth Distributions on the Hypercube by Projections
Clément Lalanne (Toulouse School of Economics), Sébastien Gadat (Toulouse School of Economics)
OptimizationSafty and Privacy
🎯 What it does: This paper proposes an adaptive density estimation method under centralized zero-concentration differential privacy (zCDP), utilizing projection estimation to estimate Sobolev smooth densities on high-dimensional hypercubes, without requiring a prior smoothness parameter β.
Proactive Detection of Voice Cloning with Localized Watermarking
Robin San Roman (Meta), Tuan Tran (Meta)
GenerationAnomaly DetectionConvolutional Neural NetworkRecurrent Neural NetworkGenerative Adversarial NetworkAudio
🎯 What it does: Proposes the AudioSeal watermarking technology, which enables active detection, sample-level localization, and attribution of AI-generated speech.
Proactive DP: A Multiple Target Optimization Framework for DP-SGD
Marten van Dijk (Centrum Wiskunde en Informatica), Phuong Ha Nguyen (eBay Inc)
OptimizationSafty and PrivacyGaussian SplattingTabular
🎯 What it does: A Proactive DP framework is proposed, which can pre-select the hyperparameters of DP-SGD to maximize the expected model accuracy based on a given privacy budget (ϵ, δ);
Probabilistic Conceptual Explainers: Trustworthy Conceptual Explanations for Vision Foundation Models
Hengyi Wang (Rutgers University), Hao Wang (Rutgers University)
Explainability and InterpretabilityTransformerContrastive LearningImage
🎯 What it does: PACE is proposed, a variational Bayesian framework designed to generate trustworthy, stable, sparse, and multi-level concept-level explanations for Vision Transformers (ViT).
Probabilistic Constrained Reinforcement Learning with Formal Interpretability
YANRAN WANG, David Boyle (Imperial College London)
OptimizationExplainability and InterpretabilityReinforcement Learning
🎯 What it does: A new adaptive Wasserstein variational optimization framework, AWaVO, is proposed to achieve probabilistic interpretability in constrained reinforcement learning. It implements interpretable policy updates and inference through adaptive sliced Wasserstein distance and distributed representation optimization.
Probabilistic Forecasting with Stochastic Interpolants and Föllmer Processes
Yifan Chen (New York University), Eric Vanden-Eijnden (New York University)
GenerationData SynthesisDiffusion modelVideoMultimodalityTime SeriesStochastic Differential Equation
🎯 What it does: A generative model based on stochastic interpolation and the Föllmer process is proposed to map the current state of the system to the conditional distribution of future states, achieving unbiased probability predictions.
Probabilistic Generating Circuits - Demystified
Sanyam Agarwal (Saarland University), Markus Bläser (Saarland University)
🎯 What it does: This paper reveals that probabilistic generating circuits (PGC) are essentially equivalent to probabilistic circuits (PC) that allow negative weights through theoretical analysis and constructive proof. It proves that PGC can be converted to non-monotonic PC in polynomial time under binary variables; it also demonstrates that marginalization of PGC with more than two values is #P-hard, and proposes a feasible marginalization using non-monotonic PC based on set multilinear polynomials, providing relevant combinatorial operations and the relationship with Determinantal Point Processes (DPP).
Probabilistic Inference in Language Models via Twisted Sequential Monte Carlo
Stephen Zhao (University of Toronto), Roger Baker Grosse
GenerationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningContrastive LearningText
🎯 What it does: A probability inference framework based on Twisted Sequential Monte Carlo (Twisted SMC) is proposed to sample and evaluate target distributions with terminal potential (reward/classifier) in large language models; a new contrastive learning method (Contrastive Twist Learning, CTL) is introduced to learn the twisted function; the sampling quality is evaluated using upper/lower bounds of bidirectional SMC.
Probabilistic Modeling of Interpersonal Coordination Processes
Paulo Soares (University of Arizona), Kobus Barnard (University of Arizona)
Multimodality
🎯 What it does: A probabilistic model based on latent variables and dynamic Bayesian networks is proposed to characterize the causal coordination process between individuals across multiple modalities and different time scales.
Probabilistic Routing for Graph-Based Approximate Nearest Neighbor Search
Kejing Lu (Nagoya University), Yoshiharu Ishikawa (Nagoya University)
RetrievalOptimizationComputational EfficiencyGraph Neural NetworkGraph
🎯 What it does: This paper proposes a probabilistic routing-based graph search algorithm (PEOs), which performs exact distance calculations only for neighbors that have a high probability of improving the results in nearest neighbor search, significantly reducing the number of distance calculations and improving query speed.
Probabilistic Subgoal Representations for Hierarchical Reinforcement Learning
Vivienne Huiling Wang (Aalto University), Joni Pajarinen (Tampere University)
Robotic IntelligenceReinforcement LearningSequential
🎯 What it does: This paper proposes the use of Gaussian processes to learn probabilistic subgoal representations to improve subgoal abstraction and planning in hierarchical reinforcement learning.
Probabilistic Time Series Modeling with Decomposable Denoising Diffusion Model
Tijin Yan (Beijing Institute of Technology), Yuanqing Xia (Zhongyuan University of Technology)
GenerationData SynthesisAnomaly DetectionDiffusion modelTime SeriesStochastic Differential Equation
🎯 What it does: A decomposable denoising diffusion model (D3M) is proposed, achieving joint generation and flow modeling through explicit linear SDE solving;
Probability Distribution of Hypervolume Improvement in Bi-objective Bayesian Optimization
Hao Wang (Leiden University), Michael Affenzeller (University of Applied Sciences)
OptimizationTabular
🎯 What it does: This paper derives the exact probability distribution of Hypervolume Improvement (HVI) in bi-objective Bayesian optimization and proposes a new acquisition function ε-PoHVI based on this distribution.
Prodigy: An Expeditiously Adaptive Parameter-Free Learner
Konstantin Mishchenko (Samsung AI Center), Aaron Defazio (Meta)
Recommendation SystemOptimizationRecurrent Neural NetworkTransformerImageTextMagnetic Resonance Imaging
🎯 What it does: A parameter-free adaptive learning rate algorithm named Prodigy is proposed to estimate the distance D to the optimal solution, thereby achieving adaptive learning rates during the optimization process.
Profile Reconstruction from Private Sketches
Hao WU (University of Copenhagen), Rasmus Pagh (University of Copenhagen)
OptimizationSafty and Privacy
🎯 What it does: This paper proposes a method for quickly estimating the frequency distribution (Profile) of multiple sets using an updatable private histogram with discrete Laplace noise perturbation in distributed, space-constrained environments.
Progressive Inference: Explaining Decoder-Only Sequence Classification Models Using Intermediate Predictions
Sanjay Kariyappa (JPMorganChase AI Research), Manuela Veloso
ClassificationExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Proposes a Progressive Inference framework that utilizes intermediate predictions from a decoder-only Transformer to compute input feature attribution.
Projecting Molecules into Synthesizable Chemical Spaces
Shitong Luo (Helixon Research), Jianzhu Ma (Tsinghua University)
GenerationData SynthesisDrug DiscoveryTransformerGraph
🎯 What it does: This study investigates the framework for projecting generated molecules into the synthetic chemical space and generating synthesizable analogs and their synthetic routes.
Projection-Free Online Convex Optimization with Time-Varying Constraints
Dan Garber (Technion Israel Institute of Technology), Ben Kretzu (Technion Israel Institute of Technology)
Optimization
🎯 What it does: This paper proposes two classes of projection-free online convex optimization algorithms to achieve low regret and low constraint violation under time-varying soft constraints.
Projection-Free Variance Reduction Methods for Stochastic Constrained Multi-Level Compositional Optimization
Wei Jiang (Nanjing University), Lijun Zhang (Nanjing University)
Optimization
🎯 What it does: This paper studies a non-projected variational reduction algorithm for solving constrained multi-layer stochastic composite optimization problems, providing corresponding sample complexity and linear minimization complexity.