ICLR 2026 Papers — Page 36
International Conference on Learning Representations · 5356 papers
PointRePar : SpatioTemporal Point Relation Parsing for Robust Category-Unified 3D Tracking
Juntao Liu (Harbin Institute of Technology), Wenjie Pei (Harbin Institute of Technology)
Object TrackingAutonomous DrivingTransformerPoint Cloud
🎯 What it does: Proposes PointRePar, a 3D single-object tracking framework that can jointly train in multi-class scenarios, simultaneously learning spatial shape and temporal motion features.
Poisson Midpoint Method for Log Concave Sampling: Beyond the Strong Error Lower Bounds
Rishikesh Srinivasan (Google DeepMind), Dheeraj Mysore Nagaraj (Google DeepMind)
OptimizationComputational EfficiencyStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Studied the sampling problem of strong log-convex distributions in R^d, proposed and analyzed the convergence of the Poisson Midpoint method (PLMC) in both overdamped and underdamped Langevin dynamics, and provided an upper bound on the gradient evaluation complexity under Wasserstein-2 distance.
PoLi-RL: A Point-to-List Reinforcement Learning Framework for Conditional Semantic Textual Similarity
Zixin Song (Tsinghua University), Chunping Li (Tsinghua University)
RetrievalTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: This study proposes PoLi-RL—a two-phase reinforcement learning (RL) framework for conditional semantic text similarity (C-STS)—which gradually transitions from point-wise rewards to a hybrid strategy combining point-wise, pair-wise, and list-wise rewards during training, leveraging LLM cross-encoders.
PoliCon: Evaluating LLMs on Achieving Diverse Political Consensus Objectives
Zhaowei Zhang (Peking University), Yaodong Yang (Peking University)
TransformerLarge Language ModelTextBenchmark
🎯 What it does: Constructed a benchmark named PoliCon to evaluate the ability of large language models in achieving diverse political consensus objectives.
Policy Contrastive Decoding for Robotic Foundation Models
Shihan Wu (University of Electronic Science and Technology of China), Lianli Gao (University of Electronic Science and Technology of China)
Object DetectionSegmentationRobotic IntelligenceDiffusion modelContrastive LearningImage
🎯 What it does: This paper proposes a training-agnostic plugin called Policy Contrastive Decoding (PCD), which improves the generalization performance of robotic policies by correcting spurious correlations through comparing action probability distributions generated from original images and object-masked images during the inference phase.
Policy Likelihood-based Query Sampling and Critic-Exploited Reset for Efficient Preference-based Reinforcement Learning
Jongkook Heo (Korea University), Seoung Bum Kim (Korea University)
Reinforcement Learning from Human FeedbackReinforcement Learning
🎯 What it does: Proposes PoLiCER, combining policy-likelihood-based query sampling and critic-based reset to enhance the efficiency and performance of preference-based reinforcement learning.
Policy Newton Algorithm in Reproducing Kernel Hilbert Space
Yixian Zhang (Tsinghua University), Wenbo Ding (Tsinghua University)
OptimizationReinforcement LearningFinance Related
🎯 What it does: This paper proposes the Policy Newton algorithm for second-order optimization in the reproducing kernel Hilbert space (RKHS), applied to non-parametric policies in reinforcement learning;
PolicyFlow: Policy Optimization with Continuous Normalizing Flow in Reinforcement Learning
Shunpeng Yang (Hong Kong University of Science and Technology), Hua Chen
Reinforcement LearningFlow-based Model
🎯 What it does: Proposed the PolicyFlow algorithm, integrating continuous normalized flows (CNF) into the on-policy reinforcement learning framework of PPO, addressing the limitations of traditional Gaussian policies in expressing complex multimodal action distributions.
Poly-attention: a general scheme for higher-order self-attention
Sayak Chakrabarti (Columbia University), Josh Alman (Columbia University)
Computational EfficiencyRepresentation LearningTransformerText
🎯 What it does: Proposed a generic polynomial attention mechanism (poly-attention) that can encompass self-attention, tensor attention, and Strassen attention, and achieves quadratic time complexity through tree-based polynomials while solving arbitrary-order function compositions.
Polychromic Objectives for Reinforcement Learning
Jubayer Ibn Hamid (Stanford University), Dorsa Sadigh (Stanford University)
Reinforcement Learning
🎯 What it does: Studied the entropy collapse problem during RL fine-tuning, proposed and implemented a polychromic objective based on ensemble reinforcement learning, integrated it into PPO to form the Polychromic PPO algorithm, which encourages generating diverse trajectories and improving task success rates.
PolyGraph Discrepancy: a classifier-based metric for graph generation
Markus Krimmel (Max Planck Institute of Biochemistry), Dexiong Chen (Max Planck Institute of Biochemistry)
GenerationTransformerGraph
🎯 What it does: Propose PolyGraph Discrepancy (PGD), which uses a binary classifier to estimate the Jensen-Shannon distance, providing a unified evaluation of graph generation models.
Polynomial Convergence of Riemannian Diffusion Models
Xingyu Xu (Carnegie Mellon University), Yuejie Chi (Carnegie Mellon University)
GenerationData SynthesisDiffusion modelStochastic Differential Equation
🎯 What it does: Proposes a discrete-time convergence analysis of diffusion models in a Riemannian geometry context, providing a polynomial convergence error upper bound on the total variation distance under L2 accurate score estimates.
Polynomial, trigonometric, and tropical activations
Ismail Khalfaoui-Hassani (Forschungszentrum Jülich), Stefan Kesselheim (Forschungszentrum Jülich)
OptimizationComputational EfficiencyRepresentation LearningConvolutional Neural NetworkTransformerImageText
🎯 What it does: Propose learnable activation functions based on orthogonal function bases (Hermite, Fourier, tropical polynomials) and corresponding variance-preserving initialization, and verify their trainability on large-scale vision and language tasks.
PolySHAP: Extending KernelSHAP with Interaction-Informed Polynomial Regression
Fabian Fumagalli (Bielefeld University), Christopher Musco (New York University)
Explainability and InterpretabilityImageTextTabular
🎯 What it does: Propose the PolySHAP method, using polynomial regression (including higher-order interaction terms) to approximate the game function, thereby more accurately estimating Shapley values;
PolySkill: Learning Generalizable Skills Through Polymorphic Abstraction For Continual Learning
Simon Yu (Northeastern University), Peng Qi (Uniphore)
Meta LearningTransformerLarge Language ModelAgentic AIText
🎯 What it does: Propose the PolySkill framework, enabling Web agents to learn generalizable and composable skills through polymorphic abstraction;
PonderLM: Pretraining Language Models to Ponder in Continuous Space
Boyi Zeng (Shanghai Jiao Tong University), Zhouhan Lin (Shanghai Jiao Tong University)
Representation LearningTransformerLarge Language ModelTextBenchmark
🎯 What it does: In each token generation step of language models, a self-‘pondering’ (deep thinking) mechanism is formed by obtaining continuous embeddings through multiple forward passes and weighting with the prediction distribution.
Pose Prior Learner: Unsupervised Categorical Prior Learning for Pose Estimation
Ziyu Wang (Nanyang Technological University), Mengmi Zhang (Nanyang Technological University)
Pose EstimationConvolutional Neural NetworkAuto EncoderImageVideo
🎯 What it does: Propose an unsupervised category prior learning method called Pose Prior Learner (PPL), which learns and extracts general pose priors through hierarchical memory, then utilizes the prior for pose estimation with iterative reasoning to address occlusions.
Pose-RFT: Aligning MLLMs for 3D Pose Generation via Hybrid Action Reinforcement Fine-Tuning
Bao Li (Chinese Academy of Sciences), Zhen Lei (Chinese Academy of Sciences)
GenerationPose EstimationTransformerLarge Language ModelReinforcement LearningVision Language ModelVision-Language-Action ModelImageTextMultimodality
🎯 What it does: Propose the Pose-RFT framework, shifting 3D human pose generation from supervised fine-tuning (SFT) to reward-driven reinforcement learning (RFT), achieving high-quality pose generation in a hybrid action space combining discrete language and continuous poses through the HyGRPO algorithm.
PoseX: AI Defeats Physics-based Methods on Protein Ligand Cross-Docking
Yize Jiang (Microcyto), Junhong Liu (Microcyto)
Drug DiscoveryProtein Structure PredictionDiffusion modelBiomedical DataBenchmark
🎯 What it does: Constructed the PoseX benchmark, collected 718 self-docking and 1312 cross-docking datasets, conducted unified evaluations of 23 physics-based, AI docking, and AI co-folding methods, and proposed an energy minimization relaxation module.
PoSh: Using Scene Graphs to Guide LLMs-as-a-Judge for Detailed Image Descriptions
Amith Ananthram (Columbia University), Kathleen McKeown (Ucla)
Explainability and InterpretabilityTransformerLarge Language ModelImageTextGraphBenchmark
🎯 What it does: Propose a fine-grained image description evaluation metric called POSH based on scene graphs, and construct a fine-grained evaluation benchmark named DOCENT containing artworks.
Positional Encoding Field
Yunpeng Bai (University of Texas at Austin), Qixing Huang (University of Texas at Austin)
GenerationDepth EstimationTransformerDiffusion modelRectified FlowNeural Radiance FieldImage
🎯 What it does: Propose Positional Encoding Field (PE-Field), extending 2D positional encoding to 3D depth-aware hierarchical encoding, and leverage this encoding on Diffusion Transformers (DiT) to achieve single-image novel view synthesis and spatial editing.
Post-hoc Probabilistic Vision-Language Models
Anton Baumann (Technical University of Munich), Martin Trapp (KTH Royal Institute of Technology)
ClassificationRetrievalExplainability and InterpretabilityVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: This paper proposes a post-hoc Bayesian method, BayesVLM, to estimate uncertainty for pre-trained vision-language models (e.g., CLIP, SigLIP) and propagate it to cosine similarity without additional training or architectural modifications;
Post-training Large Language Models for Diverse High-Quality Responses
Yilei Chen (Boston University), Aldo Pacchiano (Boston University Broad Institute of MIT and Harvard)
GenerationRepresentation LearningReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Designed and implemented a joint optimization algorithm called DQO based on Determinantal Point Process (DPP) to generate semantically diverse and high-quality responses during the later training of large language models.
Post-Training Quantization for Video Matting
Tianrui Zhu (Nanjing University), Kai Zhang (Nanjing University)
SegmentationComputational EfficiencyConvolutional Neural NetworkRecurrent Neural NetworkOptical FlowImageVideo
🎯 What it does: Designed and implemented a post-training quantization framework called PTQ4VM for video matting tasks, addressing the issues of accuracy loss and temporal inconsistency under low bit-width conditions.
PostAlign: Multimodal Grounding as a Corrective Lens for MLLMs
Yixuan Wu (University of Oxford), Jindong Gu (University of Oxford)
Explainability and InterpretabilityLarge Language ModelSupervised Fine-TuningVision Language ModelMultimodalityBenchmark
🎯 What it does: Propose a post-multimodal alignment framework MMGrounded-PostAlign, which corrects hallucinations and fine-grained understanding gaps in large vision-language models through dual visual and textual grounding.
PosterCraft: Rethinking High-Quality Aesthetic Poster Generation in a Unified Framework
Sixiang Chen (Hong Kong University of Science and Technology (Guangzhou)), Lei Zhu (Hong Kong University of Science and Technology (Guangzhou))
GenerationData SynthesisSupervised Fine-TuningVision Language ModelDiffusion modelFlow-based ModelImageTextMultimodality
🎯 What it does: Proposes the PosterCraft unified framework, achieving end-to-end generation of complete aesthetic posters from text prompts, abandoning modular layouts and fixed templates;
Potentially Optimal Joint Actions Recognition for Cooperative Multi-Agent Reinforcement Learning
Chang Huang (Tongji University), Guang Chen (Tongji University)
OptimizationReinforcement LearningBenchmark
🎯 What it does: Proposes a weighted training framework POW (Potentially Optimal Joint Actions Weighting) based on a recognition module, which identifies the potentially optimal joint action set A_r through the Q_r network and assigns higher weights to it during training, enabling value decomposition networks (e.g., QMIX, VDN, QPLEX) under CTDE environments to recover optimal policies.
PPE: Positional Preservation Embedding for Token Compression in Multimodal Large Language Models
Mouxiao Huang (Huawei Technologies), Xinghao Chen (Huawei Technologies)
CompressionTransformerLarge Language ModelVision Language ModelMultimodality
🎯 What it does: Propose Positional Preservation Embedding (PPE) to preserve spatial and temporal positional information when compressing visual tokens in multimodal large language models.
PPLLaVA: Varied Video Sequence Understanding With Prompt Guidance
Shangkun Sun (Peng Cheng Laboratory), Chen Li (Ministry of Industry and Information Technology of the People's Republic of China)
CompressionComputational EfficiencyRepresentation LearningConvolutional Neural NetworkTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringVision Language ModelContrastive LearningVideoTextMultimodality
🎯 What it does: Propose Prompt-guided Pooling LLaVA, achieving prompt-guided video visual feature compression and extraction, significantly reducing the number of visual tokens and enhancing long video understanding efficiency.
PQGAN: Product-Quantised Image Representation for High-Quality Image Synthesis
Denis Zavadski (Heidelberg University), Carsten Rother (Heidelberg University)
GenerationDiffusion modelAuto EncoderGenerative Adversarial NetworkImage
🎯 What it does: Proposed and implemented PQGAN, an image autoencoder introducing product quantization within the VQGAN framework, for generating high-quality, reconstructable discrete latent representations;
Practical estimation of the optimal classification error with soft labels and calibration
Ryota Ushio (University of Tokyo), Masashi Sugiyama (RIKEN AIP)
ClassificationImageText
🎯 What it does: This paper proposes a method to estimate the optimal classification error rate under a binary classification setup, extending previous work that utilized soft labels to estimate Bayesian error.
Pragma-VL: Towards a Pragmatic Arbitration of Safety and Helpfulness in MLLMs
Ming Wen (Fudan University), Yuedong Xu (Fudan University)
Safty and PrivacySupervised Fine-TuningReinforcement LearningMixture of ExpertsVision Language ModelContrastive LearningMultimodalityBenchmark
🎯 What it does: Proposed and implemented the Pragma-VL framework, addressing the trade-off between safety and usefulness in multimodal large language models (MLLMs) through risk-aware pre-training and context-adaptive reward models to achieve dynamic arbitration.
Pre-training Limited Memory Language Models with Internal and External Knowledge
Linxi Zhao (Cornell University), Jennifer J. Sun (Cornell University)
Computational EfficiencyRepresentation LearningTransformerLarge Language ModelTextGraphRetrieval-Augmented Generation
🎯 What it does: Propose a new pre-trained language model, LMLM, which stores entity-level facts in an external database and during pre-training inserts lookup calls with loss masking over retrieved values, enabling the model to learn how to query rather than memorize facts.
Pre-training LLM without Learning Rate Decay Enhances Supervised Fine-Tuning
Kazuki Yano (Tohoku University), Jun Suzuki (Tohoku University)
OptimizationHyperparameter SearchTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Investigate the role of learning rate scheduling in large language model pre-training, focusing on the performance of supervised fine-tuning (SFT) after pre-training, and systematically evaluate the Warmup-Stable-Only (WSO) scheduler without decay.
Pre-training under infinite compute
Konwoo Kim, Tatsunori Hashimoto (Stanford University)
Computational EfficiencyKnowledge DistillationData-Centric LearningTransformerLarge Language ModelText
🎯 What it does: Under the assumption of fixed data and infinite computation, a regularization method is proposed to prevent overfitting by significantly increasing weight decay, further combined with ensemble learning and distillation techniques to achieve higher data efficiency.
Precise and Interpretable Editing of Code Knowledge in Large Language Models
Min Xue (Heidelberg University), Artur Andrzejak (Heidelberg University)
Explainability and InterpretabilityAI Code AssistantTransformerLarge Language ModelTextBenchmark
🎯 What it does: Propose TransCoder-based Precise Editing (TCPE) for code knowledge editing and design a functional equivalence evaluation benchmark KECode.
PreciseCache: Precise Feature Caching for Efficient and High-fidelity Video Generation
Jiangshan Wang (MMLab, CUHK), Xiangyu Yue (MMLab, CUHK)
GenerationComputational EfficiencyTransformerDiffusion modelRectified FlowVideo
🎯 What it does: Propose a training-agnostic acceleration framework called PreciseCache, which achieves accelerated video generation inference by precisely identifying and skipping redundant computations.
Predicting Kernel Regression Learning Curves from Only Raw Data Statistics
Dhruva Karkada (UC Berkeley), James B Simon
Representation LearningImage
🎯 What it does: This paper proposes and verifies the Hermite Eigenstructure Ansatz (HEA), which can predict the learning curves of rotation-invariant kernel ridge regression (KRR) using only the data covariance matrix and the Hermite decomposition of the objective function.
Predicting LLM Output Length via Entropy-Guided Representations
Huanyi Xie (King Abdullah University of Science and Technology), Di Wang (King Abdullah University of Science and Technology)
Computational EfficiencyRepresentation LearningTransformerLarge Language ModelText
🎯 What it does: Propose a lightweight framework that utilizes LLM internal activations for length prediction, including the static prediction module EGTP and the dynamic prediction module PLP.
Predicting LLM Reasoning Performance with Small Proxy Model
Woosung Koh (Trillion Labs), Jay Shin (Trillion Labs)
Computational EfficiencyKnowledge DistillationTransformerLarge Language ModelTextBenchmark
🎯 What it does: Propose the RBRIDGE method, using a small model (≤1B parameters) to predict the performance of large models (>7B) on reasoning tasks;
Predicting Training Re-evaluation Curves Enables Effective Data Curriculums for LLMs
Shane Bergsma (Cerebras Systems), Joel Hestness (Cerebras Systems)
OptimizationData-Centric LearningLarge Language ModelText
🎯 What it does: Proposed the Training Re-evaluation Curve (TREC) diagnostic tool to quantify the model's retention of data at different time points during training, and demonstrated that placing high-quality data at TREC minima can significantly improve model performance.
Prediction with Expert Advice under Local Differential Privacy
Ben Jacobsen (University of Wisconsin Madison), Kassem Fawaz (University of Wisconsin Madison)
Safty and PrivacyTabularBiomedical DataElectronic Health Records
🎯 What it does: This study improves the expert advice prediction problem under local differential privacy (LDP) constraints, proposing two algorithms: RW-AdaBatch and RW-Meta;
Predictive CVaR Q-learning
Ju-Hyun Kim (Korea Research Institute of Standards and Science), Seungki Min (Seoul National University)
Reinforcement Learning
🎯 What it does: Proposed a predictive CVaR Q-learning algorithm specifically designed to optimize the Conditional Value-at-Risk (CVaR) objective in reinforcement learning.
Predictive Differential Training Guided by Training Dynamics
Fanqi Wang (University of Tennessee, Knoxville), Igor Mezic (University of California, Santa Barbara)
OptimizationComputational EfficiencyConvolutional Neural NetworkTransformerImage
🎯 What it does: Proposed and implemented a pluggable Predictive Differential Training (PDT) framework that selectively accelerates weight updates during training using a Koopman prediction model.
PredNext: Explicit Cross-View Temporal Prediction for Unsupervised Learning in Spiking Neural Networks
Yiting Dong (Peking University), Tiejun Huang (Peking University)
RecognitionRetrievalRepresentation LearningConvolutional Neural NetworkSpiking Neural NetworkContrastive LearningVideo
🎯 What it does: Studied unsupervised learning in deep spiking neural networks (SNN), proposing the PredNext module, which significantly enhances temporal consistency and representation ability through cross-perspective future feature prediction.
PrefDisco: Benchmarking Proactive Personalized Reasoning
Shuyue Stella Li (University of Washington), Yulia Tsvetkov (University of Washington)
Large Language ModelTextBenchmark
🎯 What it does: Designed and implemented the PREFDISCO evaluation framework, transforming static reasoning benchmarks into interactive personalized assessments, and defined the PREFALIGN metric to measure the alignment between responses and user preferences.
Preference Leakage: A Contamination Problem in LLM-as-a-judge
Dawei Li (Arizona State University), huan liu
Explainability and InterpretabilityData-Centric LearningTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningPrompt EngineeringTextBenchmark
🎯 What it does: This paper reveals the preference leakage problem in LLM-as-a-judge systems, where the association between the data-generating LLM and the evaluation LLM leads to biased judgments favoring the student model.
Preference-based Policy Optimization from Sparse-reward Offline Dataset
Wenjie Qiu (Rutgers University), He Zhu (Rutgers University)
OptimizationReinforcement LearningContrastive LearningBenchmark
🎯 What it does: Developed the PREFORL algorithm, using contrastive preference learning to optimize policies on sparse reward offline data.
PreferThinker: Reasoning-based Personalized Image Preference Assessment
Shengqi Xu (Fudan University), Wangmeng Zuo (Harbin Institute of Technology)
Recommendation SystemExplainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningMultimodalityChain-of-Thought
🎯 What it does: Proposed a reasoning-based personalized image preference evaluation system, PreferThinker, which first predicts user preference configurations and then performs interpretable multi-dimensional assessment.
PrefixMemory-Tuning: Modernizing Prefix-Tuning by Decoupling the Prefix from Attention
Haonan Wang (National University of Singapore), Tianyang Hu (Chinese University of Hong Kong)
ClassificationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Propose PrefixMemory-Tuning, a parameter-efficient fine-tuning method that moves the prefix module out of the attention head and expresses prefix information using a learnable matrix and feature mapping.
Premise Selection for a Lean Hammer
Thomas Zhu (Carnegie Mellon University), Sean Welleck (Carnegie Mellon University)
AI Code AssistantTransformerContrastive LearningText
🎯 What it does: Proposed LEARNPREMISE, a premise selector based on neural networks, and combined it with tools such as Lean-auto, Aesop, and Duper to build the first end-to-end Lean hammer called LEANHAMMER;
Presenting a Paper is an Art: Self-Improvement Aesthetic Agents for Academic Presentations
Chengzhi Liu (University of California, Santa Barbara), Xin Eric Wang (University of California, Santa Barbara)
GenerationReinforcement LearningAgentic AIVision Language ModelImageVideoTextMultimodalityBenchmark
🎯 What it does: Proposes the EvoPresent framework, which automates the generation of academic presentations and enhances content and visual quality through self-improvement.
Preserve and Personalize: Personalized Text-to-Image Diffusion Models without Distributional Drift
Gihoon Kim (Seoul National University), Taesup Kim (Seoul National University)
GenerationTransformerSupervised Fine-TuningDiffusion modelImageText
🎯 What it does: A method is proposed for few-shot personalization training in text-to-image diffusion models, which constrains parameter updates through Lipschitz regularization to maintain the original generation distribution and avoid overfitting.
Preserve and Sculpt: Manifold-Aligned Fine-tuning of Vision-Language Models for Few-Shot Learning
Dexia Chen (Sun Yat-sen University), Ruixuan Wang (Sun Yat-sen University)
ClassificationRepresentation LearningMeta LearningTransformerSupervised Fine-TuningVision Language ModelContrastive LearningMultimodality
🎯 What it does: Propose a fine-tuning framework called MPS-Tuning based on semantic manifold alignment and sculpting, which can improve the performance of few-shot vision-language tasks while preserving the knowledge of pre-trained models.
Preserving Forgery Artifacts: AI-Generated Video Detection at Native Scale
Zhengcen Li (Harbin Institute of Technology), Jingyong Su (Harbin Institute of Technology)
Data SynthesisAnomaly DetectionTransformerSupervised Fine-TuningVision Language ModelVideo
🎯 What it does: Studied AI-generated video detection, proposing native spatiotemporal scale processing for videos and constructing a large-scale multi-generator dataset.
Pretrain Value, Not Reward: Decoupled Value Policy Optimization
Chenghua Huang (Fudan University), Saravan Rajmohan (Microsoft)
Reinforcement Learning from Human FeedbackReinforcement LearningContrastive LearningTextBenchmark
🎯 What it does: Propose the DVPO (Decoupled Value Policy Optimization) framework, which pretrains a global value model (GVM) and freezes it during the RLHF process, directly using GVM to guide policy optimization, thereby eliminating the online value function training in traditional actor-critic methods.
Pretrain–Test Task Alignment Governs Generalization in In-Context Learning
Mary Letey (Harvard University), Cengiz Pehlevan (Harvard University)
Representation LearningMeta LearningTransformer
🎯 What it does: This paper constructs a linear attention model that can be analytically solved, deriving a high-dimensional analytical formula for the generalization error of ICL (in-context learning) when there is a covariance mismatch between pre-training and test tasks, and uses this formula to verify that the alignment between pre-training and test tasks is a key factor determining ICL performance;
Pretraining Scaling Laws for Generative Evaluations of Language Models
Rylan Schaeffer (Stanford University), Sanmi Koyejo (Stanford University)
Computational EfficiencyHyperparameter SearchTransformerLarge Language ModelText
🎯 What it does: Investigated and established three pre-training scaling laws to predict the performance of language models on generative evaluation (pass‑atk), revealing that the number of attempts k serves as a critical hyperparameter influencing scaling behavior and predictability;
Pretraining with hierarchical memories: separating long-tail and common knowledge
Hadi Pouransari (Apple), Oncel Tuzel (Apple)
Knowledge DistillationRepresentation LearningTransformerLarge Language ModelTextRetrieval-Augmented Generation
🎯 What it does: Propose a pre-training architecture based on hierarchical memory, storing long-tail knowledge in external memory while the anchor model handles common sense and reasoning
Pretraining with Re-parametrized Self-Attention: Unlocking Generalizationin SNN-Based Neural Decoding Across Time, Brains, and Tasks
Yuqi Yang, Shaomin Zhang (Zhejiang University)
Computational EfficiencySpiking Neural NetworkSupervised Fine-TuningBiomedical Data
🎯 What it does: Proposed the Re-parameterized Self-Attention Spiking Neural Network (RAT SNN) and a cross-condition pre-training framework to achieve high accuracy, strong generalization, and low power consumption in neural decoding.
Preventing Model Collapse Under Overparametrization: Optimal Mixing Ratios for Interpolation Learning and Ridge Regression
Anvit Garg (Harvard University), Pragya Sur (Harvard University)
OptimizationTabular
🎯 What it does: Theoretical analysis of the model collapse problem in overparameterized linear regression, providing closed-form expressions for the long-term generalization error of the minimum L2 norm interpolator and ridge regression when continuously mixing real and synthetic labels, and deriving the optimal mixing ratio.
Price of Quality: Sufficient Conditions for Sparse Recovery using Mixed-Quality Data
Youssef Chaabouni (Massachusetts Institute of Technology), David Gamarnik (Massachusetts Institute of Technology)
Optimization
🎯 What it does: This paper studies the problem of sparse signal recovery under mixed quality data (high-quality and low-quality noise), providing two types of recovery thresholds from information theory and algorithms, and introduces the concept of 'Price of Quality' to quantify the value of high-quality samples relative to low-quality samples.
Prima.cpp: Fast 30-70B LLM Inference on Heterogeneous and Low-Resource Home Clusters
Zonghang Li (MBZUAI), Xue Liu (MBZUAI)
Computational EfficiencyTransformerLarge Language ModelText
🎯 What it does: This paper proposes prima.cpp, a distributed LLM inference system designed for home clusters, capable of running Llama and other large models with 30–70B parameters under extreme conditions such as mixed CPU/GPU, insufficient memory/VRAM, slow disks, and Wi-Fi.
Primal-Dual Policy Optimization for Linear CMDPs with Adversarial Losses
Kihyun Yu (KAIST), Dabeen Lee (Seoul National University)
OptimizationReinforcement Learning
🎯 What it does: Proposed a primal-dual policy optimization algorithm for constrained Markov decision processes (CMDP) under adversarial loss and stochastic cost, achieving upper bounds on sublinear regret and constraint violation rates.
Primary-Fine Decoupling for Action Generation in Robotic Imitation
Xiaohan Lei, Houqiang Li (University of Science and Technology of China)
Robotic IntelligenceTransformerFlow-based ModelAuto EncoderPoint Cloud
🎯 What it does: Propose the PF-DAG two-stage robot imitation learning framework, first using VQ-VAE to discretize actions into chunks as main modes, then generating continuous fine-grained actions with MeanFlow;
Principled Fast and Meta Knowledge Learners for Continual Reinforcement Learning
Ke Sun (University of Alberta), Linglong Kong (University of Alberta)
Meta LearningReinforcement LearningBenchmark
🎯 What it does: Propose a dual-learner framework (fast learner and meta-learner), where the fast learner enables knowledge transfer and the meta-learner enables knowledge integration to address knowledge transfer and catastrophic forgetting in continuous reinforcement learning.
Principled RL for Diffusion LLMs Emerges from a Sequence-Level Perspective
Jingyang Ou (Renmin University of China), Chongxuan Li (Renmin University of China)
GenerationTransformerLarge Language ModelReinforcement LearningDiffusion modelText
🎯 What it does: A full-sequence level reinforcement learning framework (ESPO) is proposed for diffusion-based large language models (dLLMs), which constructs a sequence-level advantage function and KL regularizer to perform post-training optimization by using ELBO as a computable proxy for sequence likelihood.
Prior-aware and Context-guided Group Sampling for Active Probabilistic Subsampling
Beomgu Kang (Korea University), Hyunseok Seo (Korea University)
OptimizationComputational EfficiencyRepresentation LearningConvolutional Neural NetworkRecurrent Neural NetworkImageBiomedical DataMagnetic Resonance Imaging
🎯 What it does: Proposed a grouped active sampling method named PGA-DPS that combines prior information and context guidance to achieve efficient subsampling across different tasks.
Prior-based Noisy Text Data Filtering: Fast and Strong Alternative For Perplexity
Yeongbin Seo (Yonsei University), Jinyoung Yeo (Yonsei University)
Computational EfficiencyData-Centric LearningText
🎯 What it does: This paper proposes a text data filtering method based on lexical priors, using word frequency statistics instead of traditional PPL filtering;
Prior-free Tabular Test-time Adaptation
Rundong He (Hong Kong Polytechnic University), Jieming Shi (Hong Kong Polytechnic University)
Domain AdaptationTabularBenchmark
🎯 What it does: Propose a table-based test-time adaptation method PFT3A without source prior and source data
PriorGuide: Test-Time Prior Adaptation for Simulation-Based Inference
Yang Yang (University of Helsinki), Luigi Acerbi (Aalto University)
Domain AdaptationDiffusion modelScore-based ModelTabularBenchmarkStochastic Differential Equation
🎯 What it does: Proposed a framework called PriorGuide, which enables diffusion posterior inference to adapt to new prior distributions during testing without retraining, based on a pre-trained diffusion model.
Priors in time: Missing inductive biases for language model interpretability
Ekdeep Singh Lubana (Goodfire AI), Aaron Mueller (Harvard University)
Explainability and InterpretabilityTransformerLarge Language ModelAuto EncoderText
🎯 What it does: Proposed Temporal SAE, improving traditional sparse autoencoders to capture the temporal dependency structure in language model activations, enabling the separation of predictable slow-varying information from novel fast-varying information.
PRISM-Physics: Causal DAG-Based Process Evaluation for Physics Reasoning
Wanjia Zhao (Stanford University), James Zou (Stanford University)
Explainability and InterpretabilityGraphBenchmarkPhysics Related
🎯 What it does: Decompose physics problem solutions into formula nodes, construct directed acyclic graphs (DAGs), and perform process-level evaluation using rule-based formula equivalence matching, leading to the PRISM-PHYSICS framework.
PRISM: Enhancing PRotein Inverse Folding through Fine- Grained Retrieval on Structure-Sequence Multimodal Representations
Sazan Mahbub (Carnegie Mellon University), Eric P. Xing (Carnegie Mellon University)
GenerationProtein Structure PredictionTransformerBiomedical DataRetrieval-Augmented Generation
🎯 What it does: Propose the PRISM framework, utilizing fine-grained structure-sequence retrieval to enhance protein inverse folding generation;
PRISM: Festina Lente Proactivity—Risk-Sensitive, Uncertainty-Aware Deliberation for Proactive Agents
Yuxuan Fu, Xihe Qiu (Shanghai University Of Engineering Science)
Knowledge DistillationReinforcement Learning from Human FeedbackLarge Language ModelAgentic AITextBenchmark
🎯 What it does: This paper proposes the PRISM framework, decomposing the active agent's 'whether to intervene' and 'when to intervene' into demand probability p_need and acceptance probability p_accept, achieving precise and efficient proactivity by utilizing cost-sensitive gating and triggering slow inference only when approaching thresholds.
PRISM: Partial-label Relational Inference with Spatial and Spectral Cues
Yiyang Gu (Peking University), Ming Zhang (Peking University)
ClassificationGraph Neural NetworkGraph
🎯 What it does: This paper proposes the PRISM framework to address the partially labeled learning problem on graph data, combining spatial substructure alignment with spectral multi-band attention-based relational reasoning and label propagation.
PRISM: Progressive Robust Learning for Open-World Continual Category Discovery
Wei Feng (Monash University), Zongyuan Ge (Monash University)
ClassificationDomain AdaptationRepresentation LearningImage
🎯 What it does: Propose the PRISM framework, which automatically identifies and clusters known and unknown classes in the open-world continuous class discovery (OW-CCD) scenario through frequency domain separation, sparse matching, and domain-invariant knowledge transfer.
PrismAudio: Decomposed Chain-of-Thought and Multi-dimensional Rewards for Video-to-Audio Generation
Huadai Liu (Hong Kong University of Science and Technology), Wei Xue (Chinese University of Hong Kong)
GenerationReinforcement Learning from Human FeedbackTransformerReinforcement LearningDiffusion modelVideoMultimodalityChain-of-ThoughtStochastic Differential EquationOrdinary Differential EquationAudio
🎯 What it does: This work proposes the PrismAudio framework, which combines multi-dimensional Chain-of-Thought reasoning with reinforcement learning to achieve interpretable and controllable video-to-audio generation.
PRISMM-Bench: A Benchmark of Peer-Review Grounded Multimodal Inconsistencies
Lukas Selch (Johannes Kepler University Linz), Wei Lin (Johannes Kepler University Linz)
Large Language ModelImageTextMultimodalityTabularBenchmarkChain-of-Thought
🎯 What it does: Constructed a multimodal inconsistency benchmark PRISMM-Bench based on real peer-reviewed annotations, covering authentic errors across multiple modalities including text, charts, tables, and formulas.
PRISON: Unmasking the Criminal Potential of Large Language Models
Xinyi Wu (Fudan University), Min Yang (Fudan University)
Safty and PrivacyTransformerLarge Language ModelTextChain-of-Thought
🎯 What it does: Proposed the PRISON framework, which evaluates the criminal potential and detection capability of large language models (LLMs) in real criminal scenarios from three perspectives.
Privacy Beyond Pixels: Latent Anonymization for Privacy-Preserving Video Understanding
Joseph Fioresi (University of Central Florida), Mubarak Shah (University of Central Florida)
Anomaly DetectionSafty and PrivacyAdversarial AttackTransformerContrastive LearningVideo
🎯 What it does: Proposed a framework named SPLAVU for de-anonymization in the latent space of video foundation models, achieving privacy protection on a frozen encoder with a lightweight Anonymizing Adapter Module (AAM) while maintaining multi-task performance.
Privacy-Protected Causal Survival Analysis Under Distribution Shift
Yi Liu, Larry Han (Fred Hutchinson Cancer Center)
Federated LearningSafty and PrivacyTabularBiomedical Data
🎯 What it does: This paper proposes a privacy-preserving federated learning framework for estimating causal survival curves at target sites under multi-source distribution shift environments;
Private Rate-Constrained Optimization with Applications to Fair Learning
Mohammad Yaghini (University of Toronto), Nicolas Papernot (University of Toronto)
OptimizationSafty and PrivacyConvolutional Neural NetworkImageTabular
🎯 What it does: Developed an algorithm called RaCO-DP that can optimize machine learning models with rate constraints while satisfying differential privacy (DP);
PRO-MOF: Policy Optimization with Universal Atomistic Models for Controllable MOF Generation
Zicheng Liu (Beihang University), Di Huang (Beihang University)
GenerationOptimizationTransformerReinforcement LearningScore-based ModelFlow-based ModelGraphPhysics RelatedStochastic Differential Equation
🎯 What it does: Propose the PRO-MOF framework that combines hierarchical reinforcement learning with flow matching models and universal atomic models to achieve controllable MOF generation and inverse design.
Probabilistic Kernel Function for Fast Angle Testing
Kejing Lu (Yamanashi University), Yoshiharu Ishikawa (Nagoya University)
RetrievalComputational EfficiencyImageText
🎯 What it does: Propose two probability kernel functions K1S and K2S based on reference angles for angle comparison and angle threshold determination in high-dimensional Euclidean space, and design KS1 projection method and KS2 routing test based on this.
Probability Distributions Computed by Autoregressive Transformers
Andy Yang (University of Notre Dame), David Chiang (University of Notre Dame)
Transformer
🎯 What it does: This paper theoretically explores the expressive power of Transformers as language models (i.e., autoregressive probability models), comparing differences between Boolean and real-valued weights, classifiers and autoregressive models, and corresponding formal systems such as LTL, counting logic, and finite automata;
Probing in the Dark: State Entropy Maximization for POMDPs
Yonatan Ashlag (Technion Israel Institute of Technology), Kfir Yehuda Levy
Reinforcement LearningBenchmark
🎯 What it does: Proposes a method for reward-free pre-training in partially observable Markov decision processes (POMDP) by maximizing the entropy of predicted latent states, and implements the LatEnt algorithm and PROBE benchmark;
Probing Rotary Position Embeddings through Frequency Entropy
Yui Oka (Human Informatics Labs., NTT, Inc.), Kuniko Saito (Human Informatics Labs., NTT, Inc.)
Explainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelText
🎯 What it does: This paper proposes a metric called Frequency Entropy (FE), defining SpectrumFE and SequenceFE separately, to quantify the utilization of each frequency dimension in Transformer's Rotary Position Embedding (RoPE). Based on this, frequency structure analysis and reversible suppression experiments are conducted on the Llama-4 model (including iRoPE).
Probing to Refine: Reinforcement Distillation of LLM Reasoners via Explanatory Inversion
Zhen Tan (Arizona State University), huan liu
Explainability and InterpretabilityKnowledge DistillationTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningText
🎯 What it does: Propose the ExGRPO framework, which combines Explanatory Inversion (EI) to generate diverse explanatory probing questions, and incorporates dialogue structure utility rewards in reinforcement learning to achieve knowledge distillation and reasoning capability enhancement for small-scale LLMs.
Procedural Mistake Detection via Action Effect Modeling
Wenliang Guo (Michigan State University), Yu Kong (Michigan State University)
Anomaly DetectionExplainability and InterpretabilityRobotic IntelligenceTransformerLarge Language ModelPrompt EngineeringVision Language ModelVision-Language-Action ModelContrastive LearningVideoText
🎯 What it does: This paper proposes the Action Effect Modeling (AEM) framework, which combines action execution with the spatial states and relationships of its outcomes to improve procedural error detection.
Process-Level Trajectory Evaluation for Environment Configuration in Software Engineering Agents
Jiayi Kuang (Sun Yat-sen University), Philip S. Yu (University Of Illinois Chicago)
Data SynthesisAI Code AssistantTransformerLarge Language ModelAgentic AITextBenchmark
🎯 What it does: Propose EnConda-Bench, a process-oriented environment configuration evaluation framework, and implement automated data generation with Docker validation
Process-Verified Reinforcement Learning for Theorem Proving via Lean
Minsu Kim (KAIST AI), Se-Young Yun (KAIST AI)
AI Code AssistantTransformerLarge Language ModelReinforcement LearningTextBenchmarkChain-of-Thought
🎯 What it does: In the Lean proof assistant, symbolic process feedback is used as a fine-grained reward during training, combined with global result rewards, and the GRPO reinforcement learning framework is employed to train large language models for theorem proving tasks.
Product of Experts for Visual Generation
Yunzhi Zhang (Stanford University), Jiajun Wu (Stanford University)
GenerationData SynthesisMixture of ExpertsVision Language ModelFlow-based ModelImageVideoTextMultimodality
🎯 What it does: Built a vision generation framework based on expert product distribution, which can integrate multi-source models (generative, discriminative, physics simulation) during inference to achieve controllable image/video synthesis.
ProfBench: Multi-Domain Rubrics requiring Professional Knowledge to Answer and Judge
Zhilin Wang (NVIDIA), Yi Dong (NVIDIA)
TransformerLarge Language ModelPrompt EngineeringTextBenchmarkFinance RelatedPhysics RelatedChain-of-Thought
🎯 What it does: Create ProfBench benchmark, covering 7,347 professional domain tasks with corresponding evaluation criteria, and assess LLMs' capabilities in generation and judgment.
Programming by Backprop: An Instruction is Worth 100 Examples When Finetuning LLMs
Jonathan Cook (University of Oxford), Laura Ruis (MIT)
Computational EfficiencyAI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningPrompt EngineeringText
🎯 What it does: Propose a new training method called Programming by Backprop (PBB), enabling large language models to compile executable program logic into model parameters during training using only instructional commands (rather than specific demonstrations), allowing direct execution during inference without context instructions;
Programming with Pixels: Can Computer-Use Agents do Software Engineering?
Pranjal Aggarwal (Carnegie Mellon University), Sean Welleck (Carnegie Mellon University)
AI Code AssistantTransformerLarge Language ModelReinforcement LearningVision-Language-Action ModelMultimodalityBenchmark
🎯 What it does: Proposed the Programming with Pixels (PwP) environment and the PwP-Bench benchmark to evaluate the performance of Computer Usage Agents (CUA) in software engineering tasks.
Progressive Gaussian Transformer with Anisotropy-aware Sampling for Open Vocabulary Occupancy Prediction
Chi Yan (Hong Kong University of Science and Technology), Dan Xu (Hong Kong University of Science and Technology)
Autonomous DrivingTransformerVision Language ModelGaussian SplattingTextPoint Cloud
🎯 What it does: Propose a progressively densifying Gaussian Transformer framework (PG-Occ) for open-vocabulary 3D occupancy prediction;
Progressive Online Video Understanding with Evidence-Aligned Timing and Transparent Decisions
Kecheng Zhang (University of Science and Technology of China), Xiaojun Chang (University of Science and Technology of China)
Explainability and InterpretabilityComputational EfficiencyTransformerLarge Language ModelVision Language ModelVideoTextBenchmarkChain-of-Thought
🎯 What it does: Proposes the Thinking-QwenVL framework, focusing on online video understanding, enabling the model to answer questions in video streams at the earliest time point with sufficient evidence and provide transparent progress and reasoning during the answering process.
Projected Coupled Diffusion for Test-Time Constrained Joint Generation
Hao Luan (National University of Singapore), Chun Kai Ling (National University of Singapore)
GenerationRobotic IntelligenceDiffusion modelScore-based ModelImageStochastic Differential Equation
🎯 What it does: Proposed Projected Coupled Diffusion (PCD), a framework that generates samples satisfying hard constraints by jointly utilizing multiple pre-trained diffusion models during testing
Prompt and Parameter Co-Optimization for Large Language Models
Xiaohe Bo (Renmin University of China), Zhenhua Dong (Renmin University of China)
OptimizationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Propose the MetaTuner framework to achieve jointly collaborative optimization of prompt tuning and model fine-tuning, thereby enhancing the task performance of large language models (LLMs).