ICML 2024 Papers — Page 18
International Conference on Machine Learning · 2610 papers
PAC-Bayesian Generalization Bounds for Knowledge Graph Representation Learning
Jaejun Lee (KAIST), Joyce Jiyoung Whang (KAIST)
Representation LearningGraph Neural NetworkGraphBiomedical Data
🎯 What it does: A PAC-Bayesian generalization bound for knowledge graph representation learning is proposed, and a ReED framework that can cover various models is constructed.
PAGER: Accurate Failure Characterization in Deep Regression Models
Jayaraman J. Thiagarajan (Lawrence Livermore National Labs), Rushil Anirudh (Amazon)
Anomaly DetectionOptimizationTabularTime Series
🎯 What it does: This paper proposes the PAGER framework for accurately detecting and classifying failure samples in deep regression models.
PairNet: Training with Observed Pairs to Estimate Individual Treatment Effect
Lokesh Nagalapatti (Indian Institute of Technology Bombay), Sunita Sarawagi (Indian Institute of Technology Bombay)
TabularBiomedical Data
🎯 What it does: A new Individual Treatment Effect (ITE) estimation training strategy called PairNet is proposed, which minimizes loss using only observed instance pairs;
Pairwise Alignment Improves Graph Domain Adaptation
Shikun Liu (Georgia Institute of Technology), Pan Li (Georgia Institute of Technology)
Domain AdaptationGraph Neural NetworkGraph
🎯 What it does: A graph domain adaptation method called Pairwise Alignment (Pair-Align) is proposed to address Conditional Structure Shift (CSS) and Label Shift (LS) in graph structures, thereby improving node classification performance across different graph domains.
PANDA: Expanded Width-Aware Message Passing Beyond Rewiring
Jeongwhan Choi (Yonsei University), Noseong Park (KAIST)
ClassificationGraph Neural NetworkGraph
🎯 What it does: A width-aware message passing framework called PANDA is proposed, which alleviates the over-compression problem of graph neural networks by only expanding the hidden dimensions of high centrality nodes.
PAPM: A Physics-aware Proxy Model for Process Systems
Pengwei Liu (Zhejiang University), Dong Ni (Zhejiang University)
OptimizationComputational EfficiencyTime SeriesBenchmarkPhysics RelatedOrdinary Differential Equation
🎯 What it does: For agent modeling of process systems, a physics-aware agent model (PAPM) is proposed that globally incorporates partial prior physical knowledge (general forms of various initial/boundary conditions and conservation relationships), along with a spatiotemporal stepping module (TSSM) designed to flexibly adapt to different process systems.
Parallel Affine Transformation Tuning of Markov Chain Monte Carlo
Philip Schär (Friedrich Schiller University Jena), Daniel Rudolf (University of Passau)
🎯 What it does: A Parallel Affine Transformation Tuning (PATT) framework is proposed, which uses reversible affine transformations to map the target distribution to an approximately isotropic space, and then combines baseline MCMC methods (such as GPSS and ESS) to achieve adaptive, parameter-free efficient sampling.
Parallelized Spatiotemporal Slot Binding for Videos
Gautam Singh (Rutgers University), Tong Che (NVIDIA Research)
Object DetectionRepresentation LearningTransformerVideo
🎯 What it does: A parallel spatiotemporal slot binding network (PSB) is proposed for unsupervised learning of object-centric representations in videos.
Parameter Efficient Quasi-Orthogonal Fine-Tuning via Givens Rotation
Xinyu Ma (Peking University), Junfeng Zhao (Peking University)
OptimizationComputational EfficiencyTransformerSupervised Fine-TuningText
🎯 What it does: A parameter-efficient fine-tuning method based on Givens rotation (qGOFT) is proposed, which adapts large pre-trained models to downstream tasks through reparameterization while preserving the angular information of the pre-trained model.
Parameter Estimation in DAGs from Incomplete Data via Optimal Transport
Vy Vo (Monash University), Dinh Phung (VinAI Research)
OptimizationImageGraphTime Series
🎯 What it does: In the context of missing data, this paper proposes a parameter learning framework based on optimal transport (OT) called OTP-DAG, aimed at estimating the parameters of probabilistic graphical models for arbitrary DAG structures.
Parameter-Dependent Competitive Analysis for Online Capacitated Coverage Maximization through Boostings and Attenuations
Pan Xu (New Jersey Institute of Technology)
OptimizationOrdinary Differential Equation
🎯 What it does: This paper addresses the online capacity-limited coverage maximization problem and proposes and analyzes two sampling strategies based on linear programming.
Parameter-Efficient Fine-Tuning with Controls
Chi Zhang (National University of Singapore), Qianxiao Li (National University of Singapore)
ClassificationTransformerSupervised Fine-TuningImage
🎯 What it does: The paper transforms the explanation of weight differences in LoRA and other PEFT algorithms into a control process, proposing a nonlinear controller with parameter-free cross-patch attention, and conducts experimental validation on visual tasks.
Parameter-Efficient Fine-Tuning with Discrete Fourier Transform
Ziqi Gao (Hong Kong University of Science and Technology), Jia Li (Hong Kong University of Science and Technology)
Supervised Fine-TuningImageText
🎯 What it does: A parameter-efficient fine-tuning method based on discrete Fourier transform, called FourierFT, is proposed, which directly learns sparse coefficients in the frequency domain and recovers weight updates through inverse DFT.
Parameterized Physics-informed Neural Networks for Parameterized PDEs
Woojin Cho (Yonsei University), Noseong Park (KAIST)
OptimizationComputational EfficiencyRepresentation LearningAuto EncoderTime SeriesPhysics Related
🎯 What it does: Proposed Parameterized Physics-informed Neural Networks (P2INN) for one-shot learning of solutions to multi-parameter PDEs.
PARCv2: Physics-aware Recurrent Convolutional Neural Networks for Spatiotemporal Dynamics Modeling
Phong C.H. Nguyen (University of Virginia), Stephen Baek (University of Virginia)
Convolutional Neural NetworkRecurrent Neural NetworkTime SeriesPhysics Related
🎯 What it does: The PARCv2 model is proposed, which learns and predicts the evolution of general nonlinear fields through a recursive convolutional network using a differentiator-integrator architecture, including the spatiotemporal dynamics of physical systems such as heat conduction, convective diffusion reactions, etc.
PARDEN, Can You Repeat That? Defending against Jailbreaks via Repetition
Ziyang Zhang (University of Oxford), Jakob Nicolaus Foerster
Adversarial AttackTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: The PARDEN method is proposed, which utilizes the LLM itself to first generate outputs, then has the LLM repeat those outputs, and uses BLEU similarity to determine if it is a jailbreak, thereby achieving defense.
Parsimonious Learning-Augmented Approximations for Dense Instances of $\mathcal{NP}$-hard Problems
Evripidis Bampis (Sorbonne Université), Michalis Xefteris (Sorbonne Université)
Optimization
🎯 What it does: For a class of dense NP-hard problems (such as MAX-CUT, MAX-kSAT, etc.), the authors propose a learning-augmented approximation algorithm that utilizes a small number (of the order of magnitude) of one-bit predictions, and provide a complete algorithm framework LAA (Learning Augmented Approximation) along with specific implementations for the aforementioned problems.
Partial Multi-View Multi-Label Classification via Semantic Invariance Learning and Prototype Modeling
Chengliang Liu (Harbin Institute of Technology), Yong Xu (Shenzhen Campus of Sun Yat-sen University)
ClassificationAuto EncoderMultimodality
🎯 What it does: This paper proposes a multi-view multi-label classification framework named SIP, which can learn cross-view shared representations and achieve label prediction through prototype modeling in the presence of missing views and missing labels.
Partial Optimality in the Linear Ordering Problem
David Stein (TU Dresden), Bjoern Andres (TU Dresden)
Optimization
🎯 What it does: This paper proposes and implements a set of partial optimality conditions and efficient verification algorithms for the Linear Ordering Problem, which can determine the optimal values of some variables without completely solving the problem, thereby reducing the problem size and accelerating the solution process.
Partially Stochastic Infinitely Deep Bayesian Neural Networks
Sergio Calvo Ordoñez, YUANTAO SHI
ClassificationComputational EfficiencyImageStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Proposes and implements a partially stochastic infinite deep Bayesian neural network (PSDE-BNN), proving its capability as a universal conditional distribution approximator, and provides various weight partitioning and training configurations.
Particle Denoising Diffusion Sampler
Angus Phillips (University of Oxford), Arnaud Doucet
Diffusion modelScore-based ModelMultimodality
🎯 What it does: A new sampling framework is proposed - Particle Denoising Diffusion Sampler (PDDS), which can sample and unbiasedly estimate the normalization constant by approximating time-reversed diffusion and particle filtering in situations where only unnormalized densities can be computed.
PASOA- PArticle baSed Bayesian Optimal Adaptive design
Jacopo Iollo (Université Grenoble Alpes), Florence Forbes (Université Grenoble Alpes)
OptimizationReinforcement LearningTabular
🎯 What it does: A Bayesian sequential experimental design method called PASOA is proposed, which can simultaneously perform design optimization and parameter posterior estimation.
Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models
Asma Ghandeharioun (Google Research), Mor Geva (Tel Aviv University)
Explainability and InterpretabilityTransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: The Patchscopes framework is proposed, which allows for interpretability analysis of LLM's internal computations by treating hidden layer representations as 'patches' to different prompts or models, enabling LLM to generate natural language explanations for these representations.
Path-Guided Particle-based Sampling
Mingzhou Fan (Texas A&M University), Xiaoning Qian (Texas A&M University)
OptimizationTabularStochastic Differential Equation
🎯 What it does: A particle sampling framework PGPS based on Log-weighted Shrinkage (LwS) density path guidance is proposed, which uses neural networks to learn vector fields, allowing particles to migrate along a predetermined path from the initial distribution to the target posterior distribution.
Pausing Policy Learning in Non-stationary Reinforcement Learning
Hyunin Lee (University of California), Somayeh Sojoudi (University of California)
Reinforcement LearningTabular
🎯 What it does: A prediction-based online reinforcement learning framework is proposed, which improves the dynamic loss upper bound in non-stationary environments through policy hold, and verifies the optimal ratio of policy update to hold time.
PcLast: Discovering Plannable Continuous Latent States
Anurag Koul (Microsoft Research), Alex Lamb (Microsoft Research)
Robotic IntelligenceReinforcement LearningContrastive LearningSequential
🎯 What it does: For high-dimensional perceptual input, the PCLAST method is proposed to learn a plannable continuous latent state representation, and based on this, achieve multi-level planning.
PDHG-Unrolled Learning-to-Optimize Method for Large-Scale Linear Programming
Bingheng Li (Michigan State University), Ruoyu Sun (Chinese University of Hong Kong)
OptimizationGraph Neural NetworkTabular
🎯 What it does: Designed and implemented a deployable neural network based on the PDHG algorithm, called PDHG-Net, and proposed a two-stage learning optimization (L2O) framework to accelerate the solution of large-scale linear programming (LP).
PEARL: Zero-shot Cross-task Preference Alignment and Robust Reward Learning for Robotic Manipulation
Runze Liu (Tsinghua University), Xiu Li (Tsinghua University)
Robotic IntelligenceTransformerReinforcement LearningSequential
🎯 What it does: This paper proposes a zero-shot cross-task preference alignment and robust reward learning framework (PEARL), which can utilize preference labels from the source task to generate pseudo-labels in the target task and learn policies.
Pedestrian Attribute Recognition as Label-balanced Multi-label Learning
Yibo Zhou (Beihang University), Haotian Wu (Beihang University)
ClassificationRecognitionImage
🎯 What it does: This paper addresses the issue of label imbalance in pedestrian attribute recognition by proposing two lightweight methods: Feature Re-sampled Decoupled Learning (FRDL) and Gradient-Oriented Augment Translating (GOAT), achieving true label balance and enhancing semantic diversity.
Peeking with PEAK: Sequential, Nonparametric Composite Hypothesis Tests for Means of Multiple Data Streams
Brian M Cho, Nathan Kallus (Cornell University)
Time SeriesSequential
🎯 What it does: A novel non-parametric sequential testing method called PEAK is proposed for the combined hypothesis testing of the means of multiple data streams.
PerceptAnon: Exploring the Human Perception of Image Anonymization Beyond Pseudonymization for GDPR
Kartik Patwari (University of California), Vivek Sharma (Sony AI)
Safty and PrivacyConvolutional Neural NetworkImage
🎯 What it does: This study investigates the overall human perception of image anonymization and proposes a learning-based anonymity metric called PerceptAnon.
Perfect Alignment May be Poisonous to Graph Contrastive Learning
Jingyu Liu (Renmin University of China), Yong Liu (Renmin University of China)
OptimizationRepresentation LearningGraph Neural NetworkContrastive LearningGraph
🎯 What it does: This paper studies the impact of perfect alignment on downstream tasks in graph contrastive learning and proves that a larger augmentation distance is more beneficial for generalization, subsequently proposing two general augmentation methods.
Performance Bounds for Active Binary Testing with Information Maximization
Aditya Chattopadhyay (Johns Hopkins University), Donald Geman (Johns Hopkins University)
Tabular
🎯 What it does: This paper studies the performance upper bound of Information Maximization (InfoMax) in binary testing, providing a non-empty bound based on the δ-unpredictability assumption and proving that this bound is tight in the extreme examples constructed.
Performative Prediction with Bandit Feedback: Learning through Reparameterization
Yatong Chen (University of California), Yang Liu (University of California)
OptimizationTabular
🎯 What it does: The study investigates how to solve the optimal model through reparameterization of performative risk in situations where gradients cannot be obtained and the distribution mapping is unknown, employing a two-layer zero-order (bandit) optimization algorithm, achieving sublinear scheduling guarantees regarding the sample size.
Perturb-and-Project: Differentially Private Similarities and Marginals
Vincent Cohen-Addad (Google Research), Peilin Zhong (Google Research)
Safty and PrivacyComputational EfficiencyGaussian SplattingTabular
🎯 What it does: The paper proposes an input perturbation-projection framework for efficiently releasing cosine similarity between vector pairs and k-way marginal queries under differential privacy, and provides an improved error upper bound for sparse data.
Pessimism Meets Risk: Risk-Sensitive Offline Reinforcement Learning
Dake Zhang (University of Chicago), Tong Zhang (University of Illinois Urbana-Champaign)
Reinforcement Learning
🎯 What it does: Two risk-sensitive offline reinforcement learning algorithms based on linear MDPs (RSPVI and VA-RSPVI) are proposed, along with theoretical guarantees for their approximate optimality.
PGODE: Towards High-quality System Dynamics Modeling
Xiao Luo (University of California), Yizhou Sun (Peking University)
Graph Neural NetworkMixture of ExpertsTime SeriesPhysics RelatedOrdinary Differential Equation
🎯 What it does: A prototype decomposition-based graph neural ODE model PGODE is proposed for high-quality multi-agent dynamics modeling.
Physics and Lie symmetry informed Gaussian processes
David Dalton (University of Glasgow), Hao Gao (University of Glasgow)
TabularPhysics Related
🎯 What it does: This paper proposes a method to embed Lie symmetry into Physics-informed Gaussian Processes (PSGP) to enhance the prediction and parameter identification of PDE models using ISC constraints.
Physics of Language Models: Part 3.1, Knowledge Storage and Extraction
Zeyuan Allen-Zhu (Meta), Yuanzhi Li (MBZUAI)
TransformerLarge Language ModelSupervised Fine-TuningTextPhysics Related
🎯 What it does: This study investigates how language models store knowledge during the pre-training phase and how they extract knowledge during inference, using controllable synthetic biography data for experiments.
Physics-Informed Neural Network Policy Iteration: Algorithms, Convergence, and Verification
Yiming Meng (University of Illinois Urbana-Champaign), Jun Liu (University of Waterloo)
OptimizationReinforcement LearningPhysics Related
🎯 What it does: This paper proposes two policy iteration algorithms based on Physics-Informed Neural Networks (PINN) and Extreme Learning Machines (ELM) to solve the generalized Hamilton-Jacobi-Bellman (GHJB) linear partial differential equations for nonlinear optimal control problems, and proves their convergence to the viscosity solution of the HJB.
Pi-DUAL: Using privileged information to distinguish clean from noisy labels
Ke Wang (Ecole Polytechnique Federale de Lausanne), Pascal Frossard (Ecole Polytechnique Federale de Lausanne)
ClassificationImage
🎯 What it does: The study uses privileged information (PI) to distinguish between clean labels and noisy labels, proposing the Pi-DUAL dual-path network.
PICLe: Eliciting Diverse Behaviors from Large Language Models with Persona In-Context Learning
Hyeong Kyu Choi (University of Wisconsin Madison), Yixuan Li (University of Wisconsin Madison)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: A method called PICLe is proposed, which uses Bayesian inference and likelihood ratios to select examples in order to induce large language models to exhibit target personality traits through contextual learning.
PID: Prompt-Independent Data Protection Against Latent Diffusion Models
Ang Li (Peking University), Yisen Wang (Peking University)
Safty and PrivacyAdversarial AttackDiffusion modelImage
🎯 What it does: This paper addresses the potential threats of privacy leakage and proposes a protection method for Latent Diffusion Models (PID) that does not rely on text prompts.
PIDformer: Transformer Meets Control Theory
Tam Minh Nguyen, Richard Baraniuk
ClassificationSegmentationAdversarial AttackTransformerReinforcement LearningImageText
🎯 What it does: This paper proposes the PID-control Transformer (PIDformer), which integrates the Proportional-Integral-Derivative (PID) controller with the self-attention mechanism to form a new controlled state space model aimed at enhancing the robustness of Transformers and addressing the output dimensional collapse issue.
Piecewise Constant and Linear Regression Trees: An Optimal Dynamic Programming Approach
Mim van den Bos (Delft University of Technology), Emir Demirović (Delft University of Technology)
OptimizationComputational EfficiencyTabular
🎯 What it does: Three optimal regression tree algorithms based on dynamic programming have been developed, namely piecewise constant regression trees, univariate linear regression trees, and multivariate linear regression trees, along with a dedicated acceleration algorithm for deep binary trees.
PinNet: Pinpoint Instructive Information for Retrieval Augmented Code-to-Text Generation
Han Fu (Alibaba Group), Jianling Sun (Zhejiang University)
GenerationRetrievalAI Code AssistantTransformerContrastive LearningTextRetrieval-Augmented Generation
🎯 What it does: Proposes the PinNet framework, utilizing retrieval-enhanced code-to-text generation;
PIPER: Primitive-Informed Preference-based Hierarchical Reinforcement Learning via Hindsight Relabeling
Utsav Singh (Indian Institute of Technology Kanpur), Amrit Bedi
Robotic IntelligenceReinforcement LearningSequential
🎯 What it does: This paper proposes PIPER, a hierarchical reinforcement learning framework that combines primitive information with preference learning and retrospective re-labeling to address the issues of reward non-stationarity and subgoal infeasibility in sparse reward tasks.
PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMs
Soroush Nasiriany (Google DeepMind), brian ichter
OptimizationRobotic IntelligenceTransformerPrompt EngineeringVision Language ModelImage
🎯 What it does: This paper proposes an iterative visual prompting framework named PIVOT, which enables low-level control of robots under zero-shot conditions by annotating candidate actions in images and allowing the VLM to select them.
PlanDQ: Hierarchical Plan Orchestration via D-Conductor and Q-Performer
Chang Chen (Rutgers University), Sungjin Ahn (KAIST)
Robotic IntelligenceReinforcement LearningDiffusion modelTabularBenchmark
🎯 What it does: A hierarchical offline reinforcement learning framework named PlanDQ is proposed, which uses a diffusion model for high-level subgoal planning (D-Conductor) and implements low-level execution through a Q-learning-based diffusion policy (Q-Performer).
Planning, Fast and Slow: Online Reinforcement Learning with Action-Free Offline Data via Multiscale Planners
Chengjie Wu (Tsinghua University), Chongjie Zhang (Washington University in St. Louis)
Reinforcement LearningVideo
🎯 What it does: This paper investigates how to achieve efficient online reinforcement learning through a pre-trained value function and a two-level state-centered planner (with fast and slow scales) using passive dataset video data without action annotations.
Plug-and-Play image restoration with Stochastic deNOising REgularization
Marien Renaud (University of Bordeaux), Nicolas Papadakis (Telecom Paris)
RestorationImageStochastic Differential Equation
🎯 What it does: The Stochastic deNOising REgularization (SNORE) framework is proposed, which modifies the PnP denoising step to denoise on noisy images and solves the inverse problem using stochastic gradient descent.
Plug-in Performative Optimization
Licong Lin (University of California), Tijana Zrnic (Stanford University)
OptimizationTabular
🎯 What it does: This paper proposes a model-based plug-in performative optimization method, which overcomes the slow convergence issue of traditional model-free methods in performative settings by first estimating the impact of the predictive model on the data distribution and then solving for the optimal predictor on the estimated distribution.
Pluvial Flood Emulation with Hydraulics-informed Message Passing
Arnold Kazadi (Rice University), Arlei Lopes da Silva
Graph Neural NetworkGraphTime Series
🎯 What it does: A hydrodynamics-based graph neural network (ComGNN) is proposed to predict the temporal evolution of water depth in a region from rainfall data.
PointMC: Multi-instance Point Cloud Registration based on Maximal Cliques
Yue Wu (Xidian University), Wenping Ma (Xidian University)
Pose EstimationComputational EfficiencyGraph Neural NetworkPoint Cloud
🎯 What it does: A multi-instance point cloud registration framework called PointMC is proposed, which achieves fast and accurate multi-pose estimation using maximum clique search and local spatial consistency features.
Policy Evaluation for Variance in Average Reward Reinforcement Learning
Shubhada Agrawal (Georgia Institute of Technology), Siva Theja Maguluri (Georgia Institute of Technology)
Reinforcement Learning
🎯 What it does: In the framework of average reward reinforcement learning, this paper studies and estimates the asymptotic variance of a given policy, and proposes a TD-based linear stochastic approximation (SA) algorithm, providing an upper bound on its finite sample error.
Policy Learning for Balancing Short-Term and Long-Term Rewards
Peng Wu (Beijing Technology and Business University), Yan Zeng (Beijing Technology and Business University)
OptimizationReinforcement Learning from Human FeedbackReinforcement LearningTabular
🎯 What it does: A strategy learning framework that balances short-term and long-term rewards is proposed, along with theoretical analysis and implementation of identification, efficiency lower bounds, and double/triple robust estimators.
Policy-conditioned Environment Models are More Generalizable
Ruifeng Chen (Nanjing University), Yang Yu (Nanjing University)
Recurrent Neural NetworkReinforcement LearningSequential
🎯 What it does: This paper proposes a Policy-Conditioned Model (PCM) based on policy representation for learning dynamics models in offline reinforcement learning, aiming to reduce the large value errors that occur when evaluating target policies under different data collection strategies.
Polynomial-based Self-Attention for Table Representation Learning
Jayoung Kim (Yonsei University), Noseong Park (KAIST)
Representation LearningTransformerTabular
🎯 What it does: Proposes a matrix polynomial-based self-attention layer, TabPSA, to replace the traditional self-attention in Transformers, aiming to reduce the over-smoothing phenomenon in tabular data and improve the quality of table representations.
PolySketchFormer: Fast Transformers via Sketching Polynomial Kernels
Praneeth Kacham (Google Research), Peilin Zhong (Google Research)
Computational EfficiencyTransformerText
🎯 What it does: Proposes the PolySketchFormer architecture, which combines high-order polynomial kernel attention with random sketch techniques to achieve linear time Transformer;
Position: $C^*$-Algebraic Machine Learning $-$ Moving in a New Direction
Yuka Hashimoto (NTT Corporation), Hachem Kadri (Aix-Marseille University)
Review/Survey Paper
🎯 What it does: This paper proposes the application of C*-algebras in machine learning, forming a C*-algebraic kernel method and neural network framework;
Position: A Call for Embodied AI
Giuseppe Paolo (Huawei Technologies), Balázs Kégl (London Research Center)
Robotic IntelligenceReinforcement LearningAgentic AIMultimodality
🎯 What it does: This paper proposes and argues that Embodied AI is a key step towards achieving general artificial intelligence, constructing a theoretical framework based on perception, action, memory, and learning.
Position: A Call to Action for a Human-Centered AutoML Paradigm
Marius Lindauer (Leibniz University Hannover), Bernd Bischl (Ludwig Maximilians University Munich)
Large Language ModelReview/Survey Paper
🎯 What it does: This paper reviews the development and current status of Automated Machine Learning (AutoML), pointing out its shortcomings in transparency, customizability, interactivity, collaboration, and user empowerment, and calls for the establishment of a human-centered AutoML paradigm.
Position: A Roadmap to Pluralistic Alignment
Taylor Sorensen (University of Washington), Yejin Choi (University of Washington)
Reinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
🎯 What it does: Proposes and systematizes three types of diversified alignment for AI systems (Overton, adjustable diversity, distributed diversity) and three corresponding benchmarks, and experimentally verifies the negative impact of existing RLHF on distributed diversity.
Position: A Safe Harbor for AI Evaluation and Red Teaming
Shayne Longpre (Massachusetts Institute of Technology), Peter Henderson (Princeton University)
🎯 What it does: This paper analyzes the legal and technical obstacles faced by the evaluation and red team testing of generative AI systems from the researcher's perspective, proposing two 'safe harbor' commitments (legal safe harbor and technical safe harbor) to protect public interest researchers from account deactivation or litigation threats, and to provide enterprises with an executable dialogue framework.
Position: AI-Powered Autonomous Weapons Risk Geopolitical Instability and Threaten AI Research
Riley Simmons-Edler (Harvard University), Kanaka Rajan (Harvard University)
Review/Survey Paper
🎯 What it does: Elaborates on and warns about the risks of AI autonomous weapons to geopolitics and AI research.
Position: AI/ML Influencers Have a Place in the Academic Process
Iain Weissburg, William Yang Wang (University of California)
Text
🎯 What it does: This study investigates the impact of AI/ML influencers on social media (X) on the citation counts of the papers they share, comparing it with a matched control group, and further employs causal inference to examine the causal effect between the two.
Position: Amazing Things Come From Having Many Good Models
Cynthia Rudin (Duke University), Zachery Boner (Duke University)
ClassificationOptimizationExplainability and InterpretabilityTabularFinance Related
🎯 What it does: This paper explores and systematizes the Rashomon effect, proposing the construction of a Rashomon set to obtain multiple high-performance, interpretable, and fair models.
Position: An Inner Interpretability Framework for AI Inspired by Lessons from Cognitive Neuroscience
Martina G. Vilas (Ernst Strüngmann Institute for Neuroscience), Gemma Roig (Hessian Center for AI)
Explainability and InterpretabilityReview/Survey Paper
🎯 What it does: A multi-level intrinsic interpretability framework inspired by cognitive neuroscience is proposed for systematically constructing and evaluating the mechanistic explanations of AI models.
Position: Application-Driven Innovation in Machine Learning
David Rolnick (McGill University and Mila - Quebec AI Institute), Adam White (University of Alberta and Alberta Machine Intelligence Institute)
Biomedical DataReview/Survey Paper
🎯 What it does: Discuss and propose the 'Application-Driven Machine Learning' (ADML) paradigm centered on real-world problems, and systematically compare it with traditional 'Method-Driven' research;
Position: Automatic Environment Shaping is the Next Frontier in RL
Younghyo Park (Improbable AI Lab Massachusetts Institute of Technology), Pulkit Agrawal (Improbable AI Lab Massachusetts Institute of Technology)
OptimizationRobotic IntelligenceLarge Language ModelReinforcement LearningSequential
🎯 What it does: It proposes that automating environment shaping is a key bottleneck for the scalability of reinforcement learning and provides a formal optimization framework for it.
Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AI
Theodore Papamarkou (University of Manchester), Ruqi Zhang (Purdue University)
Review/Survey Paper
🎯 What it does: Discusses the necessity of Bayesian Deep Learning (BDL) in the era of large-scale AI, outlining its advantages, challenges faced, and future research directions.
Position: Benchmarking is Limited in Reinforcement Learning Research
Scott M. Jordan (University of Alberta), Philip S. Thomas (University of Massachusetts)
Reinforcement LearningBenchmark
🎯 What it does: This paper demonstrates through extensive experiments that traditional benchmark evaluations in reinforcement learning require hundreds to thousands of random seeds to obtain reliable confidence intervals, and proposes 'scientific testing' as a supplementary experimental paradigm to reveal the working mechanisms of algorithms.
Position: Beyond Personhood: Agency, Accountability, and the Limits of Anthropomorphic Ethical Analysis
Jessica Dai (University of California Berkeley)
Review/Survey Paper
🎯 What it does: In the paper, the author presents two opposing perspectives on the moral agency of artificial intelligence (mechanical and volitional agency) from a political philosophy viewpoint, pointing out the limitations of traditional human-centered analyses of 'AI agency.' The author then proposes two alternative frameworks (application specificity and political process perspective) to consider AI as a product of political processes, in order to more reasonably assess its ethical and accountability issues.
Position: Building Guardrails for Large Language Models Requires Systematic Design
Yi DONG, Xiaowei Huang (University of Liverpool)
Safty and PrivacyReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningTextReview/Survey Paper
🎯 What it does: This paper proposes the construction of a systematic security protection system (guardrails) for large language models (LLMs) and reviews existing open-source solutions, emphasizing the necessity of multidisciplinary collaboration and neuro-symbolic methods.
Position: Categorical Deep Learning is an Algebraic Theory of All Architectures
Bruno Gavranović (Symbolica AI), Petar Veličković (Google DeepMind)
Recurrent Neural NetworkGraph Neural Network
🎯 What it does: This paper proposes a unified framework based on category theory that bridges the top-level constraints of deep learning models with their bottom-level implementations. It utilizes monads and their algebraic homomorphisms to describe the equivariance constraints in geometric deep learning, and further introduces parameterized mappings through the 2-category Para to model weight sharing and the implementation of recursive/recurring networks. It also proves that this framework can encompass various classical architectures, from graph networks, spherical CNNs, G-CNNs to RNNs and recursive networks, and provides algebraic/co-algebraic interpretations of structures such as equivariance, lists, trees, and Mealy machines.
Position: Compositional Generative Modeling: A Single Model is Not All You Need
Yilun Du (Massachusetts Institute of Technology), Leslie Pack Kaelbling
GenerationData SynthesisDiffusion modelImageText
🎯 What it does: This paper proposes and discusses the use of a combination of small generative models to construct large generative systems, thereby achieving a data-efficient, reusable, and generalizable generative modeling approach.
Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining
Florian Tramèr (ETH Zurich), Nicholas Carlini (Google DeepMind)
Safty and PrivacyKnowledge DistillationImageBiomedical DataReview/Survey PaperBenchmark
🎯 What it does: Discusses the approach of fine-tuning with differential privacy based on large-scale public pre-training, and raises three major concerns regarding privacy and practicality.
Position: Cracking the Code of Cascading Disparity Towards Marginalized Communities
Golnoosh Farnadi (Google Research), Negar Rostamzadeh (Google Research)
Reinforcement LearningMixture of ExpertsImageTextMultimodalityAudio
🎯 What it does: Analyzes and elaborates on the multidimensional inequalities (performance, privacy, robustness, interpretability, security, etc.) of foundational models concerning marginalized communities, proposes the concept of 'stair-step inequality', explores its origins and chain effects, and suggests systematic technical and policy countermeasures.
Position: Data Authenticity, Consent, & Provenance for AI are all broken: what will it take to fix them?
Shayne Longpre (Massachusetts Institute of Technology), Jad Kabbara
TextReview/Survey Paper
🎯 What it does: This paper systematically reviews the deficiencies in the authenticity, consent, and traceability of training data for foundational models, and proposes the necessity of constructing a unified data traceability framework.
Position: Data-driven Discovery with Large Generative Models
Bodhisattwa Prasad Majumder (Allen Institute for AI), Peter Clark (Allen Institute for AI)
TransformerLarge Language ModelAgentic AITabular
🎯 What it does: This paper proposes a framework for end-to-end data-driven scientific discovery using large-scale generative models (LGM), and builds a prototype system called DATAVOYAGER based on GPT-4, demonstrating the complete process from data understanding, hypothesis generation, planning to statistical testing.
Position: Do Not Explain Vision Models Without Context
Paulina Tomaszewska (Warsaw University of Technology), Przemyslaw Biecek
Autonomous DrivingExplainability and InterpretabilityConvolutional Neural NetworkTransformerDiffusion modelImage
🎯 What it does: This paper studies the issue of visual models lacking spatial context information in explanations and calls for attention to spatial relationships in XAI.
Position: Do pretrained Transformers Learn In-Context by Gradient Descent?
Lingfeng Shen (Johns Hopkins University), Daniel Khashabi (Johns Hopkins University)
TransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: This paper evaluates whether pre-trained Transformers achieve In-Context Learning (ICL) through gradient descent, critiques existing theoretical assumptions, and designs experiments to verify the equivalence of ICL and gradient descent (GD).
Position: Embracing Negative Results in Machine Learning
Florian Karl (Fraunhofer Institute for Integrated Circuits), Paulina Sierak (Fraunhofer Institute for Integrated Circuits)
Review/Survey Paper
🎯 What it does: The importance of actively publishing negative results is proposed, along with specific suggestions for changing the review and publication culture in the machine learning community.
Position: Enforced Amnesia as a Way to Mitigate the Potential Risk of Silent Suffering in the Conscious AI
Yegor Tkachenko (Columbia University)
Reinforcement Learning
🎯 What it does: Proposes to reduce the potential risk of silent suffering through mandatory forgetting (periodic resetting or erasing of AI memory).
Position: Evolving AI Collectives Enhance Human Diversity and Enable Self-Regulation
Shiyang Lai (University of Chicago), James Evans (University of Chicago)
TransformerLarge Language ModelText
🎯 What it does: Constructed and experimented with a free-forming AI collective of 10 Claude-2.1 large language models, simulating 30 rounds of two-person interactions to assess its social network, semantic diversity, and robustness against toxic behaviors; also validated the collective's advantages in creativity and cooperative behavior through sentence construction tasks and public goods games.
Position: Explain to Question not to Justify
Przemyslaw Biecek (University of Warsaw), Wojciech Samek (Fraunhofer Heinrich Hertz Institute)
Safty and PrivacyExplainability and InterpretabilityBiomedical DataReview/Survey Paper
🎯 What it does: This paper distinguishes two cultures in the research of explainable artificial intelligence (XAI) through theoretical analysis and review: BLUE XAI, which focuses on human values and end users, and RED XAI, which emphasizes model validation and debugging. It also proposes several research challenges that RED XAI needs to explore.
Position: Exploring the Robustness of Pipeline-Parallelism-Based Decentralized Training
Lin Lu (Huazhong University of Science and Technology), Pan Zhou (Huazhong University of Science and Technology)
Federated LearningSafty and PrivacyAdversarial AttackTransformerLarge Language ModelText
🎯 What it does: This paper addresses decentralized training based on pipeline parallelism, proposing a threat model, two types of forward/backward poisoning attacks, and designing a robust training framework that achieves 100% detection and efficient recovery using redundant computation and skip layer mechanisms.
Position: Foundation Agents as the Paradigm Shift for Decision Making
Xiaoqian Liu (University of Chinese Academy of Sciences), Junge Zhang (Chinese Academy of Sciences)
Robotic IntelligenceTransformerLarge Language ModelReinforcement LearningAgentic AIVideoMultimodalityAudio
🎯 What it does: This paper proposes and elaborates on the concept of 'Foundation Agents', constructing a complete roadmap from large-scale interactive data acquisition, self-supervised pre-training, alignment with large language models (LLMs), to task adaptation. It validates the transferability and sample efficiency improvements in robotics, gaming, and simulation tasks through case studies.
Position: Fundamental Limitations of LLM Censorship Necessitate New Approaches
David Glukhov (University of Toronto), Vardan Papyan (University of Toronto)
TransformerLarge Language ModelPrompt EngineeringText
🎯 What it does: This paper systematically analyzes the theoretical limitations of large language model (LLM) censorship, proves that semantic censorship is undecidable, and proposes a general attack method based on reversible string transformations and Mosaic Prompt;
Position: Future Directions in the Theory of Graph Machine Learning
Christopher Morris (RWTH Aachen University), Stefanie Jegelka (Massachusetts Institute of Technology)
Graph Neural NetworkGraphReview/Survey Paper
🎯 What it does: This paper summarizes and prospects the theoretical research on graph neural networks, proposing systematic challenges in the areas of expressiveness, generalization, and optimization.
Position: Graph Foundation Models Are Already Here
Haitao Mao (Michigan State University), Jiliang Tang (Michigan State University)
Graph Neural NetworkLarge Language ModelGraphBiomedical Data
🎯 What it does: This paper proposes the concept of Graph Foundation Models (GFM) and systematically organizes the principles of transferability in graph structures (network analysis, expressiveness, stability) from the perspective of graph vocabulary, guiding the design of GFM.
Position: Insights from Survey Methodology can Improve Training Data
Stephanie Eckman (University of Maryland), Frauke Kreuter (LMU Munich)
Review/Survey Paper
🎯 What it does: Summarizes and proposes ideas and suggestions for applying social survey methodology to AI training data annotation.
Position: Intent-aligned AI Systems Must Optimize for Agency Preservation
Catalin Mitelut (University of Basel), Peter Vamplew (Federation University Australia)
OptimizationReinforcement LearningReview/Survey Paper
🎯 What it does: This paper reviews both theoretical and experimental aspects, indicating that merely aligning AI with human intentions is not sufficient to ensure safety. It proposes that human 'agency preservation'—that is, protecting human autonomy, capabilities, and choice space—must also be optimized simultaneously.
Position: Is machine learning good or bad for the natural sciences?
David W Hogg, Soledad Villar (Johns Hopkins University)
Review/Survey PaperPhysics Related
🎯 What it does: This paper is an opinion piece discussing the role and risks of machine learning in the natural sciences, proposing when to use ML in scientific practice and when to exercise caution.
Position: Key Claims in LLM Research Have a Long Tail of Footnotes
Anna Rogers (IT University of Copenhagen), Sasha Luccioni (Hugging Face)
Large Language ModelTextReview/Survey PaperBenchmark
🎯 What it does: This paper presents a clear definition of large language models (LLMs) and systematically evaluates five common claims about LLMs.
Position: Levels of AGI for Operationalizing Progress on the Path to AGI
Meredith Ringel Morris (Google DeepMind), Shane Legg (Google DeepMind)
Large Language ModelTextReview/Survey Paper
🎯 What it does: A hierarchical framework of 'Levels of AGI' based on performance, breadth, and autonomy is proposed, along with corresponding evaluation, risks, and human-machine interaction paradigms.
Position: Leverage Foundational Models for Black-Box Optimization
Xingyou Song (Google DeepMind), Yutian Chen (Sakana AI)
OptimizationMeta LearningTransformerLarge Language ModelReinforcement LearningMultimodality
🎯 What it does: This paper discusses how to use large-scale foundational models (such as Transformer/LLM) as general black-box optimizers, enhancing optimization efficiency through task serialization, meta-learning, and context learning, while pointing out the limitations of traditional methods in multimodal, metadata utilization, and constraint handling.
Position: LLMs Can’t Plan, But Can Help Planning in LLM-Modulo Frameworks
Subbarao Kambhampati (Arizona State University), Anil B Murthy
TransformerLarge Language ModelTextBenchmarkChain-of-Thought
🎯 What it does: This paper studies the limitations of large language models (LLMs) in planning and reasoning, demonstrating that they cannot independently generate executable plans or perform self-verification. It proposes the LLM-Modulo framework, which uses LLMs as candidate plan generators and approximate knowledge sources, collaborating with external model-based verifiers and critics to achieve robust planning. The framework is subsequently validated on classical planning and travel planning benchmarks.
Position: Machine Learning-powered Assessments of the EU Digital Services Act Aid Quantify Policy Impacts on Online Harms
Eleonora Bonel (Sciences Po), Michele Joshua Maggini (Minsait)
TextReview/Survey Paper
🎯 What it does: This paper proposes the use of machine learning techniques to assess the impact of the EU Digital Services Act (DSA) on online harms, highlighting the shortcomings of the DSA in regulating generative models, micro-targeted advertising, and filter bubbles. It presents a research agenda and methodology for evaluating the implementation of the DSA and calls for multi-stakeholder collaboration in data acquisition, causal auditing, and dissemination modeling.
Position: Measure Dataset Diversity, Don't Just Claim It
Dora Zhao (Sony AI), Alice Xiang (Sony AI)
ImageText
🎯 What it does: This study explores how to systematically conceptualize, operationalize, and validate the diversity of machine learning datasets using measurement theory, and provides practical recommendations.