ICLR 2026 Papers — Page 40
International Conference on Learning Representations · 5356 papers
Rethinking the Gold Standard: Why Discrete Curvature Fails to Fully Capture Over-squashing in GNNs?
Jialong Chen (Sun Yat-sen University), Chuan Chen (Sun Yat-sen University)
Computational EfficiencyRepresentation LearningGraph Neural NetworkGraph
🎯 What it does: Investigate the relationship between discrete curvature and information compression (over-squashing) in graph neural networks, proving that negative curvature is not a necessary condition, proposing the MOSR metric to quantify curvature omission rate, and designing a new weighted enhanced Forman-3 curvature WAF3 along with its MinHash approximation algorithm;
Rethinking Uncertainty Estimation in LLMs: A Principled Single-Sequence Measure
Lukas Aichberger (Johannes Kepler University Linz), Sepp Hochreiter (Johannes Kepler University Linz)
Computational EfficiencyTransformerLarge Language ModelScore-based ModelText
🎯 What it does: Propose a single-sequence uncertainty estimation method based on zero-one scoring, and use greedy decoding to approximate maximum sequence log-likelihood (G-NLL), achieving low-cost uncertainty assessment without sampling.
Rethinking Unsupervised Cross-modal Flow Estimation: Learning from Decoupled Optimization and Consistency Constraint
Runmin Zhang (Zhejiang University), Hui-liang Shen
Data SynthesisDepth EstimationOptimizationConvolutional Neural NetworkOptical FlowImageMultimodality
🎯 What it does: Propose the DCFlow framework to achieve unsupervised cross-modal optical flow estimation, integrating separated optimization and cross-modal consistency constraints;
ReTool: Reinforcement Learning for Strategic Tool Use in LLMs
Jiazhan Feng (ByteDance Seed), Wanjun Zhong (ByteDance Seed)
AI Code AssistantTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningTextBenchmark
🎯 What it does: Propose the ReTool framework, which enables LLMs to adaptively invoke code interpreters during reasoning through cold-start supervised fine-tuning and interactive PPO reinforcement learning based on code interpreters, significantly improving mathematical reasoning accuracy.
ReTrace: Reinforcement Learning-Guided Reconstruction Attacks on Machine Unlearning
Mengyao Ma (University Of Queensland), Guangdong Bai (City University Of Hong Kong)
Adversarial AttackTransformerReinforcement LearningGenerative Adversarial NetworkImageText
🎯 What it does: Proposes RETRACE, which leverages residual model traces left after machine unlearning to construct an RL-guided generator, achieving instance-level and distribution-level reconstruction of forgotten data.
Retrieval-of-Thought: Efficient Reasoning via Reusing Thoughts
Ammar Ahmed (University of Minnesota), Ali Anwar (University of Minnesota)
Computational EfficiencyGraph Neural NetworkTransformerLarge Language ModelPrompt EngineeringTextGraphRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Proposes the Retrieval-of-Thought (RoT) framework, which constructs a thinking graph with sequential and semantic edges, dynamically retrieves and combines previous reasoning steps during inference to generate reusable thinking templates, thereby improving the efficiency of large language model reasoning.
Retro*: Optimizing LLMs for Reasoning-Intensive Document Retrieval
Junwei Lan (University of Science and Technology of China), Defu Lian (University of Science and Technology of China)
RetrievalExplainability and InterpretabilityLarge Language ModelSupervised Fine-TuningReinforcement LearningTextBenchmark
🎯 What it does: Propose Retro*, a rubric-based LLM mechanism for reasoning-intensive document retrieval;
Retrospective Sparse Attention for Efficient Long-Context Generation
Seonghwan Choi (Seoul National University), Jae-Joon Kim (Seoul National University)
GenerationComputational EfficiencyTransformerText
🎯 What it does: Proposes RetroAttention, a KV cache compression technique for long-text generation that compresses KV caches by backward correction of attention outputs.
Reusing Pre-Training Data at Test Time is a Compute Multiplier
Alex Fang (Apple), Tom Gunter (Apple)
RetrievalComputational EfficiencyTransformerLarge Language ModelTextBenchmarkRetrieval-Augmented Generation
🎯 What it does: After pre-training of LLMs, retrieval-augmented generation is utilized during testing to reuse pre-training data, further enhancing performance through self-consistency and diversified retrieval.
ReVeal: Self-Evolving Code Agents via Reliable Self-Verification
Yiyang Jin (Tongji University), Jing Bai (Microsoft Research Asia)
AI Code AssistantTransformerLarge Language ModelReinforcement LearningAgentic AIText
🎯 What it does: Proposed a multi-round reinforcement learning framework called ReVeal, enabling large language models to self-verify and continuously improve while generating code, ultimately achieving self-evolution of code.
Revela: Dense Retriever Learning via Language Modeling
Fengyu Cai (Technical University of Darmstadt), Heinz Koeppl (Technical University of Darmstadt)
RetrievalTransformerLarge Language ModelContrastive LearningText
🎯 What it does: Trained a dense retriever by jointly optimizing retrieval and generation through self-supervised language modeling (next-word prediction).
Revenue Maximization Under Sequential Price Competition Via The Estimation Of $s$-Concave Demand Functions
Daniele Bracale (University of Michigan), Cong Shi (University of Miami)
OptimizationTime SeriesFinance Related
🎯 What it does: This paper studies price competition among multiple sellers over T periods and proposes a dynamic pricing strategy that uses semi-parametric least squares estimation to optimize sellers' revenue.
Reverse Distillation: Consistently Scaling Protein Language Model Representations
Darius Catrina (Duke University), Rohit Singh (Flatiron Institute)
Knowledge DistillationRepresentation LearningTransformerLarge Language ModelAuto EncoderBiomedical Data
🎯 What it does: Decompose the representations of large protein language models into orthogonal subspaces with small models as a base through reverse distillation, constructing Matryoshka-style embeddings to achieve consistent scaling improvements.
Reverse-Engineered Reasoning for Open-Ended Generation
Haozhe Wang (ByteDance Seed), Fangzhen Lin (Hong Kong University of Science and Technology)
GenerationTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringText
🎯 What it does: Through reverse engineering, infer and generate deep reasoning trajectories from known high-quality generated results, thereby training models to achieve open-ended generation reasoning capabilities without reinforcement learning (RL) or a teacher.
Reversible Primitive–Composition Alignment for Continual Vision–Language Learning
Canran Xiao (Sun Yat-sen University), Yuhan Wu (Peking University)
RetrievalTransformerSupervised Fine-TuningVision Language ModelContrastive LearningMultimodality
🎯 What it does: In continuous vision-language learning without task identities and under memory constraints, a structure-priority alignment head called COMPO-REALIGN is proposed to simultaneously maintain zero-shot performance and dependency on compositional structures.
Revisit Visual Prompt Tuning: The Expressiveness of Prompt Experts
Minh Le (Trivita AI), Nhat Ho (University of Texas at Austin)
ClassificationSegmentationTransformerPrompt EngineeringMixture of ExpertsContrastive LearningImage
🎯 What it does: Proposed Visual Adaptive Prompt Tuning (VAPT), significantly enhancing visual model adaptation performance by introducing dynamic prompt experts based on VPT.
Revisiting [CLS] and Patch Token Interaction in Vision Transformers
Alexis Marouani (Meta), Huy V. Vo (Meta)
ClassificationSegmentationDepth EstimationTransformerImage
🎯 What it does: This paper separates the computation processes of [CLS] and patch tokens in Vision Transformers, specializing different layers to reduce friction between them
Revisiting Active Sequential Prediction-Powered Mean Estimation
Maria-Eleni Sfyraki (University of California San Diego), Jun-Kun Wang (University of California San Diego)
OptimizationTextTabular
🎯 What it does: Proposed a label query strategy based on online FTRL and provided non-asymptotic confidence interval analysis for active sequential mean estimation.
Revisiting Confidence Calibration for Misclassification Detection in VLMs
Jincheng Huang (University of Electronic Science and Technology of China), Xiaofeng Zhu (Hainan University)
Anomaly DetectionOptimizationExplainability and InterpretabilityTransformerContrastive LearningMultimodality
🎯 What it does: In visual-language models (VLM), a method based on confidence re-calibration is proposed to specifically enhance misclassification detection (MisD) performance.
Revisiting Group Relative Policy Optimization: Insights into On-Policy and Off-Policy Training
Youssef Mroueh (IBM Research), Jesus Rios (IBM Research)
OptimizationTransformerLarge Language ModelReinforcement LearningTextBenchmark
🎯 What it does: During the post-training phase of large language models, we redesigned and generalized Group Relative Policy Optimization (GRPO), proposing an offline (off-policy) GRPO and providing theoretical guarantees;
Revisiting Long-context Modeling from Context Denoising Perspective
Zecheng Tang (Soochow University), Min Zhang (Soochow University)
Explainability and InterpretabilityTransformerLarge Language ModelSupervised Fine-TuningTextBenchmark
🎯 What it does: Propose Context Denoising Training (CDT), which detects and suppresses irrelevant noise in the input, enabling long-context models to better focus on key information.
Revisiting Matrix Sketching in Linear Bandits: Achieving Sublinear Regret via Dyadic Block Sketching
Dongxie Wen (Renmin University of China), Zhewei Wei (Renmin University of China)
CompressionOptimizationComputational EfficiencyImage
🎯 What it does: This paper proposes an adaptive multi-scale matrix compression method called Dyadic Block Sketching, which is embedded into the linear Bandit algorithm to form DBSLinUCB. It achieves sublinear regret without prior knowledge of the stream matrix characteristics.
Revisiting Multimodal Positional Encoding in Vision–Language Models
Jie Huang (Qwen Team, Alibaba Group), Shuai Bai (Qwen Team, Alibaba Group)
TransformerSupervised Fine-TuningVision Language ModelImageVideoTextMultimodality
🎯 What it does: This work conducts a systematic analysis of multimodal RoPE position encoding, proposes two lightweight plug-and-play schemes (Multi-Head RoPE and MRoPE-Interleave), and introduces a spatial reset mechanism to enhance visual information processing;
Revisiting Nonstationary Kernel Design for Multi-Output Gaussian Processes
Qiaochu Xu (University of Hong Kong), Pablo M. Olmos (University Carlos III de Madrid)
TabularTime Series
🎯 What it does: Proposed and implemented a multi-output low-rank non-stationary kernel (MO-LRN) to enhance non-stationary modeling in multi-output Gaussian processes (MOGP).
Revisiting Parameter Server in LLM Post-Training
Xinyi Wan (Sea AI Lab), Jialin Li (National University Of Singapore)
Computational EfficiencyLarge Language ModelSupervised Fine-TuningReinforcement LearningText
🎯 What it does: Revisit the parameter server model, integrate it into Fully Sharded Data Parallel (FSDP), and propose On-Demand Communication (ODC) to reduce synchronization bottlenecks and improve the throughput of LLM post-training.
Revisiting Sharpness-Aware Minimization: A More Faithful and Effective Implementation
Jianlong Chen (Shanghai University of Finance and Economics), Zhiming Zhou (Shanghai University of Finance and Economics)
OptimizationTransformerImageText
🎯 What it does: Reinterpret and improve Sharpness-Aware Minimization (SAM), proposing a more accurate and adaptive direction estimation method called XSAM.
Revisiting the Past: Data Unlearning with Model State History
Keivan Rezaei (University of Maryland), Soheil Feizi (University of Maryland)
Safty and PrivacyData-Centric LearningTransformerLarge Language ModelTextBenchmark
🎯 What it does: Propose a machine unlearning method called MSA that utilizes historical model checkpoints to eliminate the influence of specified data on large language models without retraining.
Revisiting the Scaling Properties of Downstream Metrics in Large Language Model Training
Jakub Krajewski (University of Warsaw), Jason Ramapuram (Apple)
Computational EfficiencyTransformerLarge Language ModelText
🎯 What it does: This paper proposes a power-law scaling model that directly predicts the accuracy of downstream tasks for large language models from the training budget (FLOPs);
Revisiting Tree-Sliced Wasserstein Distance Through the Lens of the Fermat–Weber Problem
Viet-Hoang Tran (National University of Singapore), Tan Minh Nguyen (National University of Singapore)
OptimizationRepresentation LearningImageText
🎯 What it does: This paper proposes the Fermat-Weber Tree-Sliced Wasserstein (FW-TSW) method, improving the sampling strategy of Tree-Sliced Wasserstein by utilizing the geometric median and Weiszfeld algorithm to determine tree root points and directions, thereby better capturing the spatial structure of distributions.
Revisiting Weight Regularization for Low-Rank Continual Learning
Yaoyue Zheng (State Key Laboratory of Human-Machine Hybrid Augmented Intelligence), Zhiqiang Tian (Xi'an Jiaotong University)
ImageText
🎯 What it does: This paper investigates the application of weight regularization in low-rank parameterized continual learning and proposes a novel EWC-LoRA method;
Revisting Node Affinity Prediction In Temporal Graphs
Or Feldman (Ben-Gurion University of the Negev), Chaim Baskin (Ben-Gurion University of the Negev)
Graph Neural NetworkGraphTime SeriesBenchmark
🎯 What it does: Proposed a node affinity prediction framework for temporal graphs called NAVIS, and experimentally validated its superiority over existing TGNNs and simple heuristic methods.
Revisual-R1: Advancing Multimodal Reasoning From Optimized Cold Start to Staged Reinforcement Learning
Shuang Chen, Yu Cheng (Chinese University of Hong Kong)
OptimizationTransformerLarge Language ModelReinforcement LearningVision Language ModelTextMultimodality
🎯 What it does: Proposed ReVisual-R1, which integrates text-optimized cold start, multi-modal reinforcement learning, and text reinforcement learning in a three-stage training process, enhancing the inference performance of 3B/7B multi-modal large language models.
Revolutionizing Reinforcement Learning Framework for Diffusion Large Language Models
Yinjie Wang (Princeton University), Mengdi Wang (Princeton University)
AI Code AssistantTransformerLarge Language ModelReinforcement LearningDiffusion modelTextBenchmark
🎯 What it does: This paper proposes TraceRL—a trajectory-aware reinforcement learning framework that combines diffusion value models to enhance the performance of full-attention and block-attention diffusion language models on reasoning, mathematics, and code tasks, and generates the TraDo series of state-of-the-art models.
Reward Is Enough: LLMs Are In-Context Reinforcement Learners
Kefan Song (University of Virginia), Yanjun Qi (University of Virginia)
TransformerLarge Language ModelReinforcement LearningPrompt EngineeringText
🎯 What it does: Propose a minimized multi-round prompting framework, ICRL prompting, enabling large language models to self-improve through reinforcement learning using only numerical rewards during reasoning.
Reward Model Routing in Alignment
Xinle Wu (National University of Singapore), Yao Lu (National University of Singapore)
Reinforcement Learning from Human FeedbackSupervised Fine-TuningReinforcement LearningTextBenchmark
🎯 What it does: Propose BayesianRouter, a hybrid routing framework combining offline RM strength learning with online Bayesian Thompson sampling, used for dynamically selecting reward models in RLHF.
Reward Models Inherit Value Biases from Pretraining
Brian Christian, Tsvetomira Dumbalska (University Of Oxford)
Representation LearningReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningText
🎯 What it does: Investigated the value biases inherited by reward models (RM) during the pre-training phase, finding significant differences in human value preferences among RMs based on different base models;
RewardBench 2: Advancing Reward Model Evaluation
Saumya Malik (Allen Institute for Artificial Intelligence), Nathan Lambert (Allen Institute for Artificial Intelligence)
Reinforcement Learning from Human FeedbackLarge Language ModelSupervised Fine-TuningReinforcement LearningTextBenchmark
🎯 What it does: Proposed and made publicly available REWARDBENCH 2, a multi-skill, four-way evaluation benchmark for reward models (RM), covering six domains (Factual, Precise Instruction, Mathematics, Safety, Focus, Ties), and compared with existing benchmarks (e.g., RewardBench).
Rewarding Doubt: A Reinforcement Learning Approach to Calibrated Confidence Expression of Large Language Models
David Bani-Harouni (Technical University of Munich), Matthias Keicher (Technical University of Munich)
Explainability and InterpretabilityReinforcement Learning from Human FeedbackLarge Language ModelReinforcement LearningText
🎯 What it does: Propose a reward mechanism based on reinforcement learning, enabling large language models to provide calibrated confidence scores when answering questions;
RewardMap: Tackling Sparse Rewards in Fine-grained Visual Reasoning via Multi-Stage Reinforcement Learning
Sicheng Feng (Westlake University), Huan Wang (Westlake University)
RecognitionTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningVision Language ModelMultimodalityBenchmark
🎯 What it does: Construct the REASONMAP-PLUS dataset and design the REWARDMAP multi-stage reinforcement learning framework to enhance the perception and reasoning capabilities of multimodal large language models in fine-grained visual reasoning tasks (e.g., traffic route maps).
ReWatch-R1: Boosting Complex Video Reasoning in Large Vision-Language Models through Agentic Data Synthesis
Congzhi Zhang (Alibaba Group Holding Limited), Bo Zheng (Alibaba Group Holding Limited)
Data SynthesisSupervised Fine-TuningReinforcement LearningAgentic AIVision Language ModelVideoTextChain-of-Thought
🎯 What it does: This paper constructs a large-scale video reasoning dataset called ReWatch and performs SFT+RLVR post-training on large vision-language models based on this dataset, launching ReWatch-R1, which significantly improves complex video reasoning performance.
Rewriting Pre-Training Data Boosts LLM Performance in Math and Code
Kazuki Fujii, Naoaki Okazaki
Data SynthesisAI Code AssistantTransformerLarge Language ModelText
🎯 What it does: Proposed two publicly available pre-training datasets, SwallowCode and SwallowMath, which rewrite original code and mathematical texts through a four-phase transformation and retention process, enhancing LLM performance in program synthesis and mathematical reasoning
Rex-Thinker: Grounded Object Referring via Chain-of-Thought Reasoning
Qing Jiang (South China University of Technology), Lei Zhang (International Digital Economy Academy)
Object DetectionExplainability and InterpretabilityLarge Language ModelSupervised Fine-TuningReinforcement LearningVision Language ModelMultimodalityChain-of-Thought
🎯 What it does: Propose a multimodal large model framework named REX-THINKER based on Chain-of-Thought (CoT) for object reference tasks, capable of providing interpretable and verifiable predictions and refusing to output when no matching object is present.
RF-DETR: Neural Architecture Search for Real-Time Detection Transformers
Isaac Robinson (Roboflow), Neehar Peri (Carnegie Mellon University)
Object DetectionNeural Architecture SearchTransformerImage
🎯 What it does: Propose RF-DETR, a lightweight detection Transformer based on weight-sharing NAS, which can rapidly generate multiple accuracy-latency trade-off models through single training across various hardware platforms and object datasets.
RF-MatID: Dataset and Benchmark for Radio Frequency Material Identification
Xinyan Chen (Nanyang Technological University), Jianfei Yang (Nanyang Technological University)
ClassificationRecognitionConvolutional Neural NetworkRecurrent Neural NetworkTransformerTime SeriesBenchmark
🎯 What it does: Constructed and publicly released the first large-scale, wideband, and geometrically diverse RF material identification dataset, RF-MatID, and established multi-band protocols and benchmark evaluations across angles/distances on it.
RFEval: Benchmarking Reasoning Faithfulness under Counterfactual Reasoning Intervention in Large Reasoning Models
Yunseok Han (Seoul National University), Jaeyoung Do (Seoul National University)
Explainability and InterpretabilityLarge Language ModelTextBenchmark
🎯 What it does: This paper proposes RFEval, a framework for evaluating the reasoning trustworthiness of large-scale reasoning models, and constructs a benchmark dataset with 7,186 instances under this framework;
RFS: Reinforcement learning with Residual flow steering for dexterous manipulation
Entong Su (University of Washington), Abhishek Gupta (University of Washington)
Robotic IntelligenceReinforcement LearningDiffusion modelFlow-based ModelRectified FlowPoint Cloud
🎯 What it does: Developed a reinforcement learning framework called Residual Flow Steering, which combines flow matching and residual correction for data-efficient fine-tuning on pre-trained generative imitation learning policies, achieving robotic grasping and manipulation tasks from simulation to real-world environments.
RIDER: 3D RNA Inverse Design with Reinforcement Learning-Guided Diffusion
Tianmeng Hu (University of Exeter), Ke Li (University of Exeter)
GenerationOptimizationGraph Neural NetworkReinforcement LearningDiffusion modelBiomedical Data
🎯 What it does: Investigated RNA 3D inverse design, proposing a method combining conditional diffusion models with reinforcement learning to directly optimize structural similarity.
Riemannian Federated Learning via Averaging Gradient Streams
Zhenwei Huang (Xiamen University), Bamdev Mishra (Microsoft India)
Federated LearningImageText
🎯 What it does: This paper proposes a new algorithm called RFedAGS for federated learning on Riemannian manifolds, which can be trained under conditions of partial participation and data heterogeneity.
Riemannian High-Order Pooling for Brain Foundation Models
Chen Hu (Jiangnan University), Nicu Sebe (University of Trento)
ClassificationRepresentation LearningTransformerTime SeriesBiomedical Data
🎯 What it does: Propose a pluggable Riemannian High-Order Pooling (RHOP) module to enhance the classification head of large-scale EEG foundational models by leveraging distribution embedding and higher-order statistical information from Riemannian geometry;
Riemannian Optimization on Relaxed Indicator Matrix Manifold
Jh Yuan, Xuelong Li (Northwestern Polytechnical University)
OptimizationImage
🎯 What it does: Propose a new indicator matrix relaxation form, prove that it constitutes a Riemannian manifold (RIM manifold), and demonstrate its applicability in large-scale machine learning optimization.
Riemannian Variational Flow Matching for Material and Protein Design
Olga Zaghen (AMLab), Erik J Bekkers (AMLab)
GenerationDrug DiscoveryFlow-based ModelBiomedical Data
🎯 What it does: Proposed a Riemannian Gaussian Variational Flow Matching (RG-VFM), which geometrically extends Variational Flow Matching on Riemannian manifolds, and validated its effectiveness in material (MOF) and protein scaffold generation tasks.
Riemannian Zeroth-Order Gradient Estimation with Structure-Preserving Metrics for Geodesically Incomplete Manifolds
Shaocong Ma (University of Maryland), Heng Huang (University of Maryland)
OptimizationMeshPhysics Related
🎯 What it does: The study constructs a structure-preserving metric in Riemannian zeroth-order optimization when the underlying metric may not be geodesically complete, providing intrinsic two-point gradient estimates, uniform sampling methods, and convergence analysis of SGD under this metric.
Riesz Neural Operator for Solving Partial Differential Equations
shouyiliu, Yuntian Chen (Shanghai Jiao Tong University)
Time SeriesBenchmarkPhysics Related
🎯 What it does: Proposed a new framework, Riesz Neural Operator (RNO), which incorporates the Riesz transform into neural operators, improving the solution of PDEs through direction-aware local spectral derivatives
RIG: Synergizing Reasoning and Imagination in End-to-End Generalist Policy
Zhonghan Zhao (Zhejiang University), Kai Chen (Shanghai AI Laboratory)
Robotic IntelligenceTransformerLarge Language ModelReinforcement LearningVision-Language-Action ModelWorld ModelImageVideoTextMultimodalityChain-of-Thought
🎯 What it does: Propose an end-to-end general-purpose strategy named RIG, which can simultaneously generate reasoning text, low-level actions, and future visual images within a single Transformer.
Rigidity-Aware Geometric Pretraining for Protein Design and Conformational Ensembles
Zhanghan Ni (University Of Illinois Urbana Champaign), Shengchao Liu
GenerationProtein Structure PredictionTransformerFlow-based ModelContrastive LearningBiomedical Data
🎯 What it does: Proposed a two-stage rigid self-supervised pre-training framework called RigidSSL, aimed at learning geometric representations of proteins and enhancing generation task performance.
Ringleader ASGD: The First Asynchronous SGD with Optimal Time Complexity under Data Heterogeneity
Arto Maranjyan (King Abdullah University of Science and Technology), Peter Richtárik (King Abdullah University of Science and Technology)
Optimization
🎯 What it does: Proposes Ringleader ASGD, an asynchronous SGD algorithm achieving optimal time complexity in distributed learning with data heterogeneity and heterogeneous computation speeds.
Risk Phase Transitions in Spiked Regression: Alignment Driven Benign and Catastrophic Overfitting
Jiping Li (University of California Los Angeles), Rishi Sonthalia (Boston College)
Image
🎯 What it does: Studied the generalization error of minimum norm interpolation in linear regression under spiked covariance data models, providing an exact risk decomposition and systematically categorizing three regimes: benign, mild, and catastrophic overfitting.
Risk-Sensitive Agent Compositions
Guruprerana Shabadi (University of Pennsylvania), Rajeev Alur (University of Pennsylvania)
OptimizationSafty and PrivacyReinforcement LearningGraph
🎯 What it does: This paper models agent workflows as directed acyclic graphs (agent graphs) and proposes an efficient algorithm called BucketedVaR to find agent combinations that minimize value at risk (VaR) or conditional value at risk (CVaR) under constraints of maximizing safety, privacy, and fairness in the graph.
Risk-Sensitive Reinforcement Learning for Alleviating Exploration Dilemmas in Large Language Models
Yuhua Jiang (Tsinghua University), Lin Yan (ByteDance Seed)
Large Language ModelSupervised Fine-TuningReinforcement LearningTextBenchmark
🎯 What it does: This paper proposes a risk-sensitive reinforcement learning framework (RS-GRPO) to address the exploration dilemma during fine-tuning of large language models, improving multi-solution performance (pass@k) while maintaining single-solution performance (pass@1).
RiskPO: Risk-based Policy Optimization with Verifiable Reward for LLM Post-Training
Tao Ren (Peking University), Yijie Peng (Peking University)
OptimizationLarge Language ModelReinforcement LearningTextMultimodality
🎯 What it does: Propose the RiskPO framework for post-training of LLMs, employing a risk-sensitive objective MVaR (Conditional Value at Risk) for distributed optimization of verifiable rewards.
RIVER: A Real-Time Interaction Benchmark for Video LLMs
Yansong Shi (University of Science and Technology of China), Limin Wang (Nanjing University)
TransformerLarge Language ModelSupervised Fine-TuningVision Language ModelVideoBenchmark
🎯 What it does: Constructed a real-time video interaction benchmark named RIVER Bench to evaluate the capabilities of video large language models (LLMs) in retro memory, real-time perception, and proactive response, and proposed a framework to enhance online inference along with specialized online training data.
RL for Reasoning by Adaptively Revealing Rationales
Mohammad Hossein Amani (EPFL Apple), Robert West (EPFL Apple)
Supervised Fine-TuningReinforcement LearningText
🎯 What it does: AdaBack proposes an adaptive per-sample semi-supervised learning framework that achieves new reasoning capabilities in the sparse reward problem between RL and SFT.
RL Grokking Recipe: How Does RL Unlock and Transfer New Algorithms in LLMs?
Yiyou Sun, Dawn Song (University Of California Berkeley)
TransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Designed and constructed a controlled programming problem set named DELTA, and trained LLMs using reinforcement learning (RL) to investigate the feasibility of learning new reasoning strategies and their transferability.
RL makes MLLMs see better than SFT
Junha Song (KAIST), Byeongho Heo (NAVER AI Lab)
ClassificationSegmentationRepresentation LearningData-Centric LearningReinforcement Learning from Human FeedbackSupervised Fine-TuningReinforcement LearningMultimodality
🎯 What it does: This paper systematically compares the effects of supervised fine-tuning (SFT) and reinforcement learning (RL, using DPO) on multi-modal language models (MLLMs) and their visual encoders, and proposes a training scheme called Preference-Instructed Vision OpTimization (PIVOT) based on the advantages of RL. This scheme optimizes the visual encoder using RL, significantly improving visual representations and downstream task performance.
RL of Thoughts: Navigating LLM Reasoning with Inference-time Reinforcement Learning
Qianyue Hao (Tsinghua University), Yong Li (Tsinghua University)
Large Language ModelReinforcement LearningTextChain-of-Thought
🎯 What it does: Propose the RL-of-Thoughts (RLoT) framework, which uses reinforcement learning to train a lightweight navigator that dynamically generates task-specific logical structures during inference to enhance the reasoning capabilities of large language models (LLMs).
RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMs
Kohsei Matsutani (University of Tokyo), Yutaka Matsuo (University of Tokyo)
Explainability and InterpretabilityLarge Language ModelSupervised Fine-TuningReinforcement LearningTextGraphBenchmark
🎯 What it does: Explore the differences between RL and SFT in LLM reasoning paths, constructing an analysis framework at the trajectory level and step level.
RL's Razor: Why Online Reinforcement Learning Forgets Less
Idan Shenfeld (MIT), Pulkit Agrawal (MIT)
Robotic IntelligenceTransformerSupervised Fine-TuningReinforcement LearningText
🎯 What it does: The study compares catastrophic forgetting phenomena when learning new tasks between reinforcement learning (RL) and supervised fine-tuning (SFT), finding that RL better retains existing knowledge even when new task performance is the same.
RLAC: Reinforcement Learning with Adversarial Critic for Free-Form Generation Tasks
Mian Wu (Shanghai Jiao Tong University), Aviral Kumar (Carnegie Mellon University)
GenerationTransformerLarge Language ModelReinforcement LearningTextSequential
🎯 What it does: Propose and implement a post-training framework RLAC, which dynamically verifies multiple evaluation rules for open-ended generation tasks through co-training of a generator and adversarial critic.
RLAD: Training LLMs to Discover Abstractions for Solving Reasoning Problems
Yuxiao Qu (Carnegie Mellon University), Aviral Kumar (Carnegie Mellon University)
AI Code AssistantReinforcement Learning from Human FeedbackTransformerSupervised Fine-TuningReinforcement LearningTextBenchmark
🎯 What it does: Propose and train LLMs to generate and utilize 'reasoning abstractions' to enhance mathematical and program reasoning performance
RLAP-CLIP: Continual Multimodal Learning with Prototype Adaptation and Difficulty-Aware Routing
Ruikun Luo (National Engineering Research Center for Big Data Technology and System), Xiaoyu Xia (Royal Melbourne Institute of Technology)
TransformerReinforcement LearningPrompt EngineeringMixture of ExpertsVision Language ModelMultimodality
🎯 What it does: Propose RLAP-CLIP, a multimodal framework for class-incremental learning, which significantly enhances the robustness and generalization of CLIP in continual learning scenarios through reinforcement learning-based active prototype optimization, difficulty-aware Mixture-of-Experts routing, and dual-modal prompting to balance visual and textual adaptation;
RLBFF: Binary Flexible Feedback to bridge between Human Feedback & Verifiable Rewards
Zhilin Wang (NVIDIA), Oleksii Kuchaiev (NVIDIA)
Explainability and InterpretabilityReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningTextBenchmark
🎯 What it does: Propose a new reinforcement learning framework RLBFF that combines human feedback with verifiable rewards, using Binary Flexible Feedback to guide the training of the reward model.
RLP: Reinforcement as a Pretraining Objective
Ali Hatamizadeh (NVIDIA), Yejin Choi (NVIDIA)
Representation LearningTransformerLarge Language ModelReinforcement LearningTextChain-of-Thought
🎯 What it does: Propose an RLP (Reinforcement Learning Pretraining) method for improving large language models by rewarding Chain-of-Thought information gain during the pretraining phase.
RLVER: Reinforcement Learning with Verifiable Emotion Rewards for Empathetic Agents
Peisong Wang (Hong Kong University of Science and Technology), Xiaolong Li (Tencent)
TransformerReinforcement LearningTextBenchmarkChain-of-Thought
🎯 What it does: Proposed and implemented the RLVER framework, which utilizes verifiable sentiment rewards to train large language models, enabling them to possess higher-order empathy and emotional support capabilities.
RLVMR: Reinforcement Learning with Verifiable Meta-Reasoning Rewards for Robust Long-Horizon Agents
Zijing Zhang (Tencent), Xiaolong Li (Tencent)
Large Language ModelSupervised Fine-TuningReinforcement LearningAgentic AIText
🎯 What it does: Propose the RLVMR framework, which uses verifiable meta-reasoning rewards to guide LLM agents in learning more efficient and interpretable reasoning processes for long-horizon tasks.
RM-R1: Reward Modeling as Reasoning
Xiusi Chen (University of Illinois at Urbana-Champaign), Heng Ji (University of Illinois at Urbana-Champaign)
Explainability and InterpretabilityComputational EfficiencyKnowledge DistillationReinforcement Learning from Human FeedbackTransformerLarge Language ModelSupervised Fine-TuningReinforcement LearningPrompt EngineeringTextChain-of-Thought
🎯 What it does: This paper proposes a framework that treats reward modeling as an inference task—Reasoning Reward Models (RM-R1). By employing a two-phase training process (reasoning distillation + reinforcement learning with verifiable rewards), the model first generates long-chain reasoning and evaluation criteria before assigning reward scores, thereby enhancing the interpretability and performance of reward models.
RMAAT: Astrocyte-Inspired Memory Compression and Replay for Efficient Long-Context Transformers
Md Zesun Ahmed Mia (Pennsylvania State University), Abhronil Sengupta (Pennsylvania State University)
Computational EfficiencyTransformerTextBenchmark
🎯 What it does: Proposes the RMAAT (Recurrent Memory Augmented Astromorphic Transformer) architecture, abstracting the long-term and short-term plasticity mechanisms of astrocytes in processing long sequences into memory compression and linear attention, addressing the quadratic complexity bottleneck of Transformers on long sequences.
RMFlow: Refined Mean Flow by a Noise-Injection Step for Multimodal Generation
Yuhao Huang (University of Utah), Bao Wang (University of California, Los Angeles)
GenerationData SynthesisFlow-based ModelRectified FlowAuto EncoderImageMultimodalityTime Series
🎯 What it does: Proposed an RMFlow model that enhances the single-step (1-NFE) MeanFlow with refined noise injection, achieving high-fidelity multi-modal generation;
RNE: plug-and-play diffusion inference-time control and energy-based training
Jiajun He (University of Cambridge), Francisco Vargas (Xaira Therapeutics)
GenerationDiffusion modelImagePhysics Related
🎯 What it does: Proposes the Radon-Nikodym Estimator (RNE), a unified framework for estimating marginal density in diffusion models, enabling control during inference (e.g., annealing, reward bias, model combination), and providing lightweight regularization for energy-based diffusion models.
RoboCasa365: A Large-Scale Simulation Framework for Training and Benchmarking Generalist Robots
Soroush Nasiriany (University of Texas at Austin), Yuke Zhu (University of Texas at Austin)
Robotic IntelligenceLarge Language ModelVision Language ModelDiffusion modelVideoTextBenchmark
🎯 What it does: Proposed and implemented a large-scale kitchen simulation framework named RoboCasa365 for training and evaluating general-purpose robot models, and conducted systematic experiments on multi-task learning, foundation model training, and lifelong learning within this framework.
RoboInter: A Holistic Intermediate Representation Suite Towards Robotic Manipulation
Hao Li (University of Science and Technology of China), Jiangmiao Pang (Shanghai Artificial Intelligence Laboratory)
Robotic IntelligenceTransformerVision Language ModelDiffusion modelVideoMultimodalityBenchmarkChain-of-Thought
🎯 What it does: This study constructs the RoboInter Manipulation Suite, which encompasses large-scale real-world robot manipulation data, rich intermediate representations, and an evaluation benchmark, and applies it to end-to-end and modular planning-execution frameworks.
RoboMD: Uncovering Robot Vulnerabilities through Semantic Potential Fields
Som Sagar (Arizona State University), Ransalu Senanayake (Arizona State University)
Robotic IntelligenceTransformerReinforcement LearningVision Language ModelContrastive LearningMultimodality
🎯 What it does: Developed a diagnostic framework called RoboMD based on deep reinforcement learning to actively search for failure modes in robot manipulation strategies.
RoboOmni: Proactive Robot Manipulation in Omni-modal Context
Siyin Wang (Fudan University), Xipeng Qiu (Fudan University)
Robotic IntelligenceTransformerLarge Language ModelVision-Language-Action ModelMultimodalityBenchmark
🎯 What it does: This study proposes RoboOmni, an end-to-end full-modal large language model based on the Perceiver-Thinker-Talker-Executor architecture, capable of proactively inferring user intent from cross-modal context instructions (voice, environmental sounds, visual) and completing confirmation and action execution; meanwhile, it constructs the OmniAction dataset containing 140k multimodal tasks, 5k+ speakers, 2.4k event audios, and 640 background sounds, and generates OmniAction-LIBERO simulation and real environment evaluations based on this dataset.
RoboPARA: Dual-Arm Robot Planning with Parallel Allocation and Recomposition Across Tasks
Shiying Duan (Beihang University), wenjun wu
Robotic IntelligenceTransformerLarge Language ModelTextGraphBenchmarkRetrieval-Augmented Generation
🎯 What it does: Proposes RoboPARA, a dual-arm robot task parallel planning framework based on large language models, and designs a new dataset X-DAPT for evaluating dual-arm parallel planning.
RobotArena $\infty$: Scalable Robot Benchmarking via Real-to-Sim Translation
Yash Jangir (Carnegie Mellon University), Katerina Fragkiadaki (National Taiwan University)
Domain AdaptationRobotic IntelligenceReinforcement Learning from Human FeedbackVision Language ModelVideoBenchmark
🎯 What it does: Propose the RobotArena ∞ framework, which automatically converts real robot videos into simulation environments and evaluates general robot policies in these digital twins.
Robotic Manipulation by Imitating Generated Videos Without Physical Demonstrations
Shivansh Patel (University of Illinois Urbana Champaign), Yunzhu Li (Columbia University)
Pose EstimationDepth EstimationRobotic IntelligenceLarge Language ModelDiffusion modelImageVideo
🎯 What it does: The RIGVid system enables robots to learn and perform complex manipulation tasks through synthetic videos alone, without any physical demonstrations or robot-specific training.
Robust Adaptive Multi-Step Predictive Shielding
Tanmay Ambadkar (Pennsylvania State University), Abhinav Verma (Pennsylvania State University)
OptimizationRobotic IntelligenceReinforcement LearningBenchmark
🎯 What it does: Proposes the RAMPS framework, combining linear dynamics models with robust multi-step control barrier functions to achieve scalable model predictive shielding.
Robust Adversarial Attacks Against Unknown Disturbance via Inverse Gradient Sample
Zhaoyang Zhang (Harbin Institute of Technology), Yihan Yan (Harbin Institute of Technology)
Adversarial AttackImage
🎯 What it does: Propose a new robust adversarial attack framework called IGSA, which uses inverse gradient sampling to identify the most destructive direction in the perturbation space and iteratively optimizes adversarial examples to maintain high attack success rates when facing unknown perturbations.
Robust Adversarial Quantification via Conflict-Aware Evidential Deep Learning
Charmaine Barker (University of York), Simos Gerasimou (Cyprus University of Technology)
Anomaly DetectionAdversarial AttackImage
🎯 What it does: Perform post-hoc uncertainty calibration on pre-trained Evidential Deep Learning (EDL) models by generating multi-perspective evidence through label-preserving transformations, and adjust the evidence based on conflict measures to enhance detection of out-of-distribution (OOD) samples and adversarial examples.
Robust Amortized Bayesian Inference with Self-Consistency Losses on Unlabeled Data
Aayush Mishra (TU Dortmund University), Paul-Christian Bürkner (TU Dortmund University)
Flow-based ModelImageTime Series
🎯 What it does: Propose a semi-supervised neural amortized Bayesian inference method that trains on unlabelled data using a self-consistency loss, thereby enhancing inference robustness in out-of-distribution scenarios.
Robust and Interpretable Adaptation of Equivariant Materials Foundation Models via Sparsity-promoting Fine-tuning
Youngwoo Cho (KAIST), Hongkee Yoon (Kangwon National University)
Explainability and InterpretabilityGraph Neural NetworkSupervised Fine-TuningGraphPhysics Related
🎯 What it does: Proposed a sparse-promoting fine-tuning method specifically designed for E(3)-equivariant material foundation models, enabling the model to update only a minimal number of parameters while maintaining equivariance.
Robust Decision-Making with Partially Calibrated Forecasters
Shayan Kiyani (University of Pennsylvania), Aaron Roth (University of Pennsylvania)
OptimizationTabular
🎯 What it does: This paper proposes a robust decision framework based on partially calibrated (H-calibrated) predictions, solving the decision strategy that maximizes the worst-case expected utility under calibration constraints;
Robust Deep Reinforcement Learning against Adversarial Behavior Manipulation
Shojiro Yamabe, Jun Sakuma (Institute of Science Tokyo)
Adversarial AttackReinforcement LearningBenchmark
🎯 What it does: Proposes Behavior-Targeted Attacks (BIA) and the corresponding Time-Discounted Robust Training (TDRT) to enhance DRL models' ability to defend against such attacks.
Robust Equation Structure Learning with Adaptive Refinement
Yunlun Li (The Chinese University of Hong Kong), Sinno Jialin Pan (The Chinese University of Hong Kong)
OptimizationTransformerLarge Language ModelPrompt EngineeringTextBenchmarkPhysics Related
🎯 What it does: Proposes the RESTART framework, fully realizing the hypothesis-experiment-analysis cycle in scientific discovery, achieving adaptive structural learning and improvement in symbolic regression
Robust Federated Inference
Akash Dhasade (EPFL), Rafael Pinot (Sorbonne Université and Université Paris Cité)
Federated LearningSafty and PrivacyAdversarial AttackImageText
🎯 What it does: Proposed and studied the robustness issue in federated inference scenarios, providing security analysis and defense schemes for average aggregation and nonlinear aggregation (DeepSet) when up to f/2 clients are attacked.
Robust Fine-Tuning from Non-Robust Pretrained Models: Mitigating Suboptimal Transfer With Epsilon-Scheduling
Jonas Ngnawe (Université Laval), Christian Gagné (Université Laval)
ClassificationConvolutional Neural NetworkTransformerSupervised Fine-TuningImage
🎯 What it does: Investigated the suboptimal transfer phenomenon that occurs when performing robust fine-tuning from non-robust pre-trained models, and proposed Epsilon-Scheduling to alleviate it;
Robust Fine-tuning of Vision-Language-Action Robot Policies via Parameter Merging
Yajat Yadav (UC Berkeley), Sergey Levine (UC Berkeley)
Robotic IntelligenceSupervised Fine-TuningVision-Language-Action ModelMultimodality
🎯 What it does: Proposes a method called RETAIN, which achieves robust fine-tuning on new tasks by performing linear interpolation between pre-trained weights and parameters before and after fine-tuning.
Robust Generalized Schr\"{o}dinger Bridge via Sparse Variational Gaussian Processes
Minyoung Kim (Samsung AI Center)
OptimizationComputational EfficiencyImagePoint CloudStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Proposes a Schrödinger bridge robustified generalized path matching method based on sparse variational Gaussian processes, which can better estimate path distributions in noisy scenarios
Robust LLM Unlearning via Post Judgment and Multi-round Thinking
Xinrui Chen (Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences), Ou Wu (Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences)
Safty and PrivacyComputational EfficiencyTransformerLarge Language ModelPrompt EngineeringTextBenchmarkChain-of-Thought
🎯 What it does: Propose the PoRT framework to achieve zero-shot learning in LLMs and significantly enhance robustness.
Robust Multi-Objective Controlled Decoding of Large Language Models
Seongho Son (University College London), Ilija Bogunovic (University of Basel)
OptimizationReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningText
🎯 What it does: Proposed an algorithm that achieves robust multi-objective decoding (RMOD) during inference by solving a maximin game.
Robust Optimization for Mitigating Reward Hacking with Correlated Proxies
Zixuan Liu (Tulane University), Zizhan Zheng (Tulane University)
OptimizationReinforcement Learning from Human FeedbackReinforcement LearningBenchmark
🎯 What it does: This paper proposes a robust max-min strategy optimization framework to prevent reward hacking when there is an r-relatedness between the reward proxy and the true reward; meanwhile, it provides an interpretable version for linear reward structures;