CoRL 2025 Papers — Page 2
Conference on Robot Learning · 263 papers
From Tabula Rasa to Emergent Abilities: Discovering Robot Skills via Real-World Unsupervised Quality-Diversity
Luca Grillotti (Imperial College London), Antoine Cully (Imperial College London)
Robotic IntelligenceReinforcement LearningAuto EncoderWorld Model
🎯 What it does: Propose the URSA framework to achieve spontaneous exploration of diverse skills on real robots through unsupervised learning;
GC-VLN: Instruction as Graph Constraints for Training-free Vision-and-Language Navigation
Hang Yin (Tsinghua University), Jiwen Lu (Tsinghua University)
Autonomous DrivingGraph Neural NetworkTransformerLarge Language ModelVision Language ModelImageText
🎯 What it does: Proposed a training-free graph constraint method, GC-VLN, for visual-language navigation (VLN) in continuous environments. The method decomposes language instructions into a directed acyclic graph (DAG) and generates navigation paths by progressively determining coordinate positions of path points using a constraint solver.
Gen2Act: Human Video Generation in Novel Scenarios enables Generalizable Robot Manipulation
Homanga Bharadhwaj (Carnegie Mellon University), Sean Kirmani (Google DeepMind)
GenerationData SynthesisRobotic IntelligenceTransformerLarge Language ModelVision Language ModelVision-Language-Action ModelDiffusion modelImageVideoText
🎯 What it does: Propose a robot manipulation framework called Gen2Act based on zero-shot human video generation, which maps video information generated by a pre-trained video generation model and point trajectory prediction auxiliary training to robot actions, enabling task execution in unseen scenes, objects, and motion types.
Generalist Robot Manipulation beyond Action Labeled Data
Alexander Spiridonov (INSAIT, Sofia University 'St. Kliment Ohridski'), Danda Pani Paudel (ETH Zurich)
Domain AdaptationRobotic IntelligenceTransformerMixture of ExpertsVision Language ModelFlow-based ModelVideoMultimodalityPoint Cloud
🎯 What it does: This work proposes MotoVLA, a general-purpose robotic manipulation model that learns motion priors from unlabeled videos (including human and robot demonstrations) and aligns actions using a small amount of labeled action data, enabling cross-task, cross-modal zero-shot manipulation.
Generating Robot Constitutions & Benchmarks for Semantic Safety
Pierre Sermanet (Google DeepMind), Vikas Sindhwani (Google DeepMind)
Robotic IntelligenceLarge Language ModelVision Language ModelDiffusion modelImageTextMultimodalityBiomedical DataElectronic Health RecordsBenchmark
🎯 What it does: Proposes the ASIMOV benchmark and an automated robot constitutional generation method to evaluate and enhance the semantic safety of robots in real-world scenarios.
Generative Visual Foresight Meets Task-Agnostic Pose Estimation in Robotic Table-top Manipulation
Chuye Zhang (Southern University Of Science And Technology), Wei Zhang (Southern University Of Science And Technology)
GenerationPose EstimationDepth EstimationRobotic IntelligenceTransformerVision-Language-Action ModelFlow-based ModelRectified FlowContrastive LearningImageVideoTextMultimodality
🎯 What it does: This paper proposes the GVF-TAPE framework, achieving closed-loop robot manipulation without action labels in desktop scenarios through generative visual prediction and task-agnostic pose estimation.
GENNAV: Polygon Mask Generation for Generalized Referring Navigable Regions
Kei Katsumata (Keio University), Komei Sugiura (Keio University)
SegmentationDepth EstimationAutonomous DrivingConvolutional Neural NetworkTransformerSupervised Fine-TuningVision Language ModelImageTextBenchmark
🎯 What it does: Propose GENNAV, which predicts the existence of the target area and generates corresponding segmentation polygons based on natural language navigation instructions and front-facing camera images for mobile platforms.
Geometric Red-Teaming for Robotic Manipulation
Divyam Goel (Carnegie Mellon University), Zackory Erickson (Carnegie Mellon University)
OptimizationRobotic IntelligenceSupervised Fine-TuningReinforcement LearningVision Language ModelTextMesh
🎯 What it does: Proposed a red team framework named GRT, which tests the robustness of robot manipulation strategies using feasible geometric deformations (CrashShapes), and induces realistic failures through VLM-guided control point selection, Jacobian field deformation, and gradient-free optimization; simultaneously introduces blue team fine-tuning techniques to enhance adaptability to failure geometries.
GLOVER++: Unleashing the Potential of Affordance Learning from Human Behaviors for Robotic Manipulation
Teli Ma (Hong Kong University of Science and Technology), Junwei Liang (Hong Kong University of Science and Technology)
Robotic IntelligenceTransformerVision Language ModelMultimodality
🎯 What it does: This paper proposes learning manipulability occupancy from human demonstration videos and transferring it to robotic manipulation tasks.
Granular loco-manipulation: Repositioning rocks through strategic sand avalanche
Haodi Hu (University of Southern California), Feifei Qian (University of Southern California)
Robotic IntelligenceDiffusion modelImageVideo
🎯 What it does: This work proposes the DiffusiveGRAIN framework, which utilizes leg digging to induce dune slippage, enabling multi-legged robots to indirectly reposition multiple rocks on sand slopes and complete walking tasks.
GraphEQA: Using 3D Semantic Scene Graphs for Real-time Embodied Question Answering
Saumya Saxena (Carnegie Mellon University), Oliver Kroemer (Carnegie Mellon University)
Robotic IntelligenceGraph Neural NetworkTransformerVision Language ModelImageGraph
🎯 What it does: Propose the GraphEQA method, which constructs a multimodal memory using real-time 3D semantic scene graphs and task-related images to support embodied question answering based on vision-language models;
GraspMolmo: Generalizable Task-Oriented Grasping via Large-Scale Synthetic Data Generation
Abhay Deshpande (Allen Institute for AI), Ranjay Krishna (University of Texas at Austin)
Data SynthesisPose EstimationRobotic IntelligenceTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelImageTextMultimodalityPoint Cloud
🎯 What it does: Propose a general task-oriented grasping model GraspMolmo based on language instructions, which can predict stable grasping poses that align with task semantics from single-view RGB-D images.
GraspQP: Differentiable Optimization of Force Closure for Diverse and Robust Dexterous Grasping
René Zurbrügg (ETH Zürich), Marco Hutter (ETH Zürich)
OptimizationRobotic IntelligenceStochastic Differential Equation
🎯 What it does: Propose a differentiable force closure optimization framework called GraspQP for generating large-scale, diverse, and physically feasible multi-fingered grasp data.
GraspVLA: a Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Data
Shengliang Deng (Galbot), He Wang (Galbot)
Robotic IntelligenceMixture of ExpertsVision-Language-Action ModelVideoTextChain-of-Thought
🎯 What it does: This paper introduces GraspVLA, a grasp foundation model pre-trained on billions of synthetic grasp data frames.
HALO : Human Preference Aligned Offline Reward Learning for Robot Navigation
Gershom Seneviratne (University of Maryland), Dinesh Manocha (University of Maryland)
Robotic IntelligenceReinforcement Learning from Human FeedbackConvolutional Neural NetworkTransformerReinforcement LearningContrastive LearningImageSequential
🎯 What it does: This paper proposes a human-preference-based offline reward learning framework called HALO, which learns visual navigation reward functions from RGB images and trajectories, and applies them in offline policy learning and MPC planning.
Hand-Eye Autonomous Delivery: Learning Humanoid Navigation, Locomotion and Reaching
Sirui Chen (Stanford University), Karen Liu (Stanford University)
Robotic IntelligenceTransformerReinforcement LearningVision-Language-Action ModelGenerative Adversarial NetworkVideoPoint CloudSequential
🎯 What it does: Proposed the HEAD (Hand-Eye Autonomous Delivery) framework, which learns coordinated control for navigation, gait, and grasping of humanoid robots in 3D environments using human motion and visual data.
Hold My Beer: Learning Gentle Humanoid Locomotion and End-Effector Stabilization Control
Yitang Li (Tsinghua University), Guanya Shi (Carnegie Mellon University)
Robotic IntelligenceReinforcement Learning
🎯 What it does: Proposed and implemented the SoFTA (Slow-Fast Two-Agent) framework, which achieves soft and stable end-effector (EE) control during locomotion and has been deployed on actual humanoid robots.
HuB: Learning Extreme Humanoid Balance
Tong Zhang (Tsinghua University), Yang Gao (Tsinghua University)
Robotic IntelligenceReinforcement LearningVideoMesh
🎯 What it does: This paper proposes the HuB (Humanoid Balance) framework, aiming to enable humanoid robots to achieve robust execution in extreme tasks such as single-leg standing and high kicks under superstatic balance conditions;
Human-like Navigation in a World Built for Humans
Bhargav Chandaka (University of Illinois Urbana-Champaign), Shenlong Wang (University of Illinois Urbana-Champaign)
Robotic IntelligenceTransformerVision Language ModelSimultaneous Localization and MappingImageTextMultimodality
🎯 What it does: Propose ReasonNav, a modular robot navigation system that enables human-like navigation behaviors (reading signs, asking humans for directions) through Vision-Language Models (VLM), achieving efficient target room localization in unknown buildings.
Humanoid Policy ~ Human Policy
Ri-Zhao Qiu (University Of California San Diego), Xiaolong Wang (University Of California San Diego)
Data-Centric LearningRobotic IntelligenceReinforcement Learning from Human FeedbackTransformerVision-Language-Action ModelVideoTextMultimodality
🎯 What it does: This paper proposes a method to train robot manipulation policies using human first-person perspective demonstration data, and implements a learning framework for cross-body forms (humans and humanoid robots).
HyperTASR: Hypernetwork-Driven Task-Aware Scene Representations for Robust Manipulation
Li Sun (University of Hong Kong), Yanchao Yang (University of Hong Kong)
Representation LearningRobotic IntelligenceConvolutional Neural NetworkDiffusion modelAuto EncoderImageVideo
🎯 What it does: This paper proposes HyperTASR, a scene representation framework that dynamically generates task and execution-phase parameters using a hypernetwork, aimed at enhancing policy learning in robotic manipulation;
Imagine, Verify, Execute: Memory-guided Agentic Exploration with Vision-Language Models
Seungjae Lee (University of Maryland), Jia-Bin Huang (University of Maryland)
Robotic IntelligenceTransformerReinforcement LearningAgentic AIVision Language ModelImageMultimodalityRetrieval-Augmented Generation
🎯 What it does: This paper proposes the IVE framework, which utilizes vision-language models for an imagine-validate-execute loop to enable autonomous exploration for robots in reward-free environments.
Imitation Learning Based on Disentangled Representation Learning of Behavioral Characteristics
Ryoga Oishi (Saitama University), Toshiaki Tsuji (Saitama University)
Representation LearningRobotic IntelligenceRecurrent Neural NetworkTransformerAuto EncoderTextTime SeriesSequential
🎯 What it does: Propose an online action generation model that learns a separable latent space through weakly supervised labels, enabling robots to adjust movements in real-time according to human corrective instructions during task execution;
ImLPR: Image-based LiDAR Place Recognition using Vision Foundation Models
Minwoo Jung (Seoul National University), Ayoung Kim (Seoul National University)
RecognitionAutonomous DrivingComputational EfficiencyRepresentation LearningTransformerSupervised Fine-TuningContrastive LearningPoint Cloud
🎯 What it does: Propose ImLPR, a novel pipeline for LiDAR Place Recognition that leverages a pre-trained Vision Foundation Model (DINOv2);
ImMimic: Cross-Domain Imitation from Human Videos via Mapping and Interpolation
Yangcen Liu (Georgia Institute of Technology), Danfei Xu (Georgia Institute of Technology)
Robotic IntelligenceReinforcement Learning from Human FeedbackDiffusion modelVideo
🎯 What it does: Propose a cross-domain imitation framework called ImMimic, which jointly trains robot manipulation policies using a large number of human videos and a small number of robot teleoperation examples.
Improving Efficiency of Sampling-based Motion Planning via Message-Passing Monte Carlo
Makram Chahine (MIT), Daniela Rus (MIT)
OptimizationComputational EfficiencyRobotic IntelligenceGraph Neural Network
🎯 What it does: Use Message-Passing Monte Carlo based on Graph Neural Networks to generate low-discrepancy point sets, improving the sampling efficiency of sample-based motion planning.
In-Context Iterative Policy Improvement for Dynamic Manipulation
Mark Van der Merwe (University of Michigan), Devesh K. Jha (MERL)
OptimizationRobotic IntelligenceTransformerLarge Language ModelReinforcement LearningTextRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: This study proposes an In-Context Policy Improvement (ICPI) framework based on pre-trained large language models, designed to iteratively refine parameterized policies in dynamic operational tasks using a small amount of interaction data;
IRIS: An Immersive Robot Interaction System
Xinkai Jiang (Karlsruhe Institute of Technology), Rudolf Lioutikov (Karlsruhe Institute of Technology)
Robotic IntelligencePoint Cloud
🎯 What it does: Proposed and implemented IRIS, a cross-platform, cross-simulator, collaborative immersive robot interaction system that can real-time visualize simulations and real environments on XR headsets, and perform data collection and collaborative operations.
JaxRobotarium: Training and Deploying Multi-Robot Policies in 10 Minutes
Shalin Jain (Georgia Institute of Technology), Harish Ravichandar (Georgia Institute of Technology)
Domain AdaptationComputational EfficiencyRobotic IntelligenceReinforcement LearningBenchmark
🎯 What it does: Developed a full-stack robot simulation, learning, deployment, and benchmarking platform based on Jax, supporting Robotarium hardware and providing 8 standard collaborative scenarios.
Joint Model-based Model-free Diffusion for Planning with Constraints
Wonsuhk Jung (Georgia Institute of Technology), Shreyas Kousik (Georgia Institute of Technology)
OptimizationRobotic IntelligenceReinforcement LearningDiffusion modelBenchmark
🎯 What it does: This paper proposes a joint model-based and model-free diffusion framework called JM2D, which achieves mutual compatibility between trajectory generation and safety constraints.
KDPE: A Kernel Density Estimation Strategy for Diffusion Policy Trajectory Selection
Andrea Rosasco (Istituto Italiano di Tecnologia), Lorenzo Natale (Istituto Italiano di Tecnologia)
Robotic IntelligenceConvolutional Neural NetworkTransformerDiffusion modelImageSequential
🎯 What it does: To address the randomness and potential outliers in trajectories generated by Diffusion Policy, this paper proposes KDPE—a trajectory screening strategy based on kernel density estimation (KDE) that selects trajectories most consistent with the training data distribution.
KineDex: Learning Tactile-Informed Visuomotor Policies via Kinesthetic Teaching for Dexterous Manipulation
Di Zhang (Tongji University), Yang Gao (Tsinghua University)
Robotic IntelligenceDiffusion modelImageMultimodality
🎯 What it does: Proposes the KineDex framework, which utilizes hand-to-hand kinesthetic teaching to collect demonstrations with tactile information, trains a visuomotor strategy combining vision and touch, and incorporates force control during execution to achieve precise contact manipulation.
KineSoft: Learning Proprioceptive Manipulation Policies with Soft Robot Hands
Uksang Yoo (Carnegie Mellon University), Jeffrey Ichnowski (Carnegie Mellon University)
Domain AdaptationRobotic IntelligenceDiffusion modelPoint CloudMesh
🎯 What it does: Collect operational demonstrations of a soft robotic hand through kinesthetic teaching, achieve high-precision shape estimation using internal strain sensors, then train a diffusion imitation policy to generate shape trajectories, and employ a shape-conditioned controller to realize precise trajectory tracking, enabling various in-hand manipulation tasks.
KoopMotion: Learning Almost Divergence Free Koopman Flow Fields for Motion Planning
Alice Kate Li (University of Pennsylvania), M. Ani Hsieh (University of Pennsylvania)
Robotic IntelligenceFlow-based ModelSequential
🎯 What it does: Propose a flow field learning framework called KoopMotion based on the Koopman operator, which generates smooth motion planning flow fields enabling robots to move from any initial state along demonstration trajectories and ultimately converge to the target point.
LaDi-WM: A Latent Diffusion-Based World Model for Predictive Manipulation
Yuhang Huang, Kai Xu (National University Of Defense Technology)
Robotic IntelligenceTransformerVision Language ModelDiffusion modelWorld ModelImageVideo
🎯 What it does: Propose a latent diffusion world model LaDi-WM that leverages visual foundation models to predict future latent states of robot-object interactions and perform action planning based on these predictions.
Latent Adaptive Planner for Dynamic Manipulation
Donghun Noh (Ucla), Dennis Hong (Ucla)
Object DetectionRobotic IntelligenceReinforcement Learning from Human FeedbackTransformerAuto EncoderVideo
🎯 What it does: Proposes the Latent Adaptive Planner (LAP), a strategy based on trajectory-level latent variables that treats planning as latent space reasoning. It converts human demonstration videos into robot dynamics states through model-scale mapping, achieving real-time adaptation and cross-platform transfer for dynamic non-grasping manipulation tasks (e.g., box capturing).
Latent Theory of Mind: A Decentralized Diffusion Architecture for Cooperative Manipulation
Chengyang He (Stanford University), Guillaume Adrien Sartoretti (National University Of Singapore)
Robotic IntelligenceDiffusion modelImage
🎯 What it does: Propose a decentralized diffusion strategy called LatentToM for multi-arm collaborative manipulation tasks, enabling robots to achieve cooperative control without explicit communication or with limited communication.
LaVA-Man: Learning Visual Action Representations for Robot Manipulation
Chaoran Zhu (Queen Mary University of London), Changjae Oh (Queen Mary University of London)
Representation LearningRobotic IntelligenceTransformerVision-Language-Action ModelAuto EncoderTextMultimodalityBenchmark
🎯 What it does: This paper learns visual-action representations through a self-supervised target image prediction task, and applies them to robotic operations such as picking and placing guided by language.
Learn from What We HAVE: History-Aware VErifier that Reasons about Past Interactions Online
Yishu Li (Carnegie Mellon University), David Held (Carnegie Mellon University)
Robotic IntelligenceGraph Neural NetworkTransformerDiffusion modelPoint Cloud
🎯 What it does: Propose a history-aware validator (HAVE) that separates action generation from validation, utilizing past interaction history to eliminate uncertainty caused by visual ambiguity, significantly enhancing robot performance in ambiguous manipulation tasks.
Learning a Unified Policy for Position and Force Control in Legged Loco-Manipulation
Peiyuan Zhi (State Key Laboratory of General Artificial Intelligence, BIGAI), Siyuan Huang (State Key Laboratory of General Artificial Intelligence, BIGAI)
Robotic IntelligenceReinforcement LearningDiffusion modelImageTabularTime Series
🎯 What it does: This paper proposes a unified force-position control strategy that enables leg-controlled robots to achieve position tracking, force control, impedance control, and hybrid control without relying on external force sensors; additionally, it utilizes the force estimation module of this strategy to construct a force-aware imitation learning data collection pipeline, enhancing the success rate in tasks involving rich contact interactions.
Learning Deployable Locomotion Control via Differentiable Simulation
Clemens Schwarke (ETH Zurich), Marco Hutter (ETH Zurich)
Robotic IntelligenceReinforcement LearningWorld Model
🎯 What it does: Train quadruped robot locomotion control policies in a differentiable simulation environment and achieve zero-shot real-world transfer.
Learning from 10 Demos: Generalisable and Sample-Efficient Policy Learning with Oriented Affordance Frames
Krishan Rana (Queensland University of Technology), Niko Suenderhauf
Robotic IntelligenceReinforcement LearningVision-Language-Action ModelDiffusion modelImage
🎯 What it does: Propose the Oriented Affordance Frames (OAF) framework, which learns strategies generalizable from only 10 demonstrations by using object-localized, tool-aligned coordinate systems in each subtask, and automatically combines sub-strategies to complete long-horizon multi-object tasks.
Learning Impact-Rich Rotational Maneuvers via Centroidal Velocity Rewards and Sim-to-Real Techniques: A One-Leg Hopper Flip Case Study
Dongyun Kang (Korea Advanced Institute of Science and Technology), Hae-Won Park (Korea Advanced Institute of Science and Technology)
Domain AdaptationRobotic IntelligenceReinforcement Learning
🎯 What it does: Studied a method utilizing center of mass velocity reward and simulation-to-physical transfer techniques to learn and implement forward roll actions in a single-leg hopper.
Learning Long-Context Diffusion Policies via Past-Token Prediction
Marcel Torne Villasevil, Chelsea Finn
Robotic IntelligenceDiffusion modelImageSequential
🎯 What it does: Propose Past-Token Prediction (PTP) as an auxiliary task, combined with a multi-stage training process and a self-verification mechanism during testing, significantly improving the performance of robotic long-sequence diffusion policies in learning historical context.
Learning Long-Horizon Robot Manipulation Skills via Privileged Action
Xiaofeng Mao (University of Edinburgh), Michael Mistry (University of Edinburgh)
Robotic IntelligenceReinforcement Learning
🎯 What it does: This paper proposes a framework that combines privileged actions with curriculum learning, aiming to enable robots to automatically learn long-horizon, contact-rich manipulation tasks without the need for reward shaping;
Learning Smooth State-Dependent Traversability from Dense Point Clouds
Zihao Dong (Northeastern University), Michael Everett (Northeastern University)
Autonomous DrivingConvolutional Neural NetworkPoint Cloud
🎯 What it does: Propose the SPARTA method, which leverages dense point clouds to learn smooth traversability estimates for vehicles at different entry angles, outputting corresponding risk distributions; during planning, risk awareness based on angles is achieved by querying this distribution.
Leveraging Correlation Across Test Platforms for Variance-Reduced Metric Estimation
Rachel Luo (NVIDIA), Marco Pavone (NVIDIA)
Autonomous DrivingOptimizationImageVideoPoint CloudTime SeriesSequential
🎯 What it does: Propose the Sim2Val framework, which significantly reduces the variance of real-world metric estimation by leveraging cross-platform paired data and control variable methods, thereby improving sample efficiency.
LLM-Guided Probabilistic Program Induction for POMDP Model Estimation
Aidan Curtis, Leslie Pack Kaelbling (Mit Csail)
Explainability and InterpretabilityTransformerLarge Language ModelReinforcement Learning
🎯 What it does: This paper proposes a probabilistic program induction method (POMDP Coder) guided by large language models (LLMs) to learn interpretable, low-complexity partially observable Markov decision process (POMDP) models and perform online planning based on these models;
LocoFormer: Generalist Locomotion via Long-context Adaptation
Min Liu (Skild AI), Ananye Agarwal (Skild AI)
Domain AdaptationRobotic IntelligenceTransformerReinforcement Learning
🎯 What it does: Proposes LocoFormer, a general strategy capable of achieving zero-shot or few-shot adaptive locomotion control on seen or unseen multi-morphology robots (legged, wheeled, humanoid).
LocoTouch: Learning Dynamic Quadrupedal Transport with Tactile Sensing
Changyi Lin (Carnegie Mellon University), Ding Zhao (Carnegie Mellon University)
Knowledge DistillationRobotic IntelligenceRecurrent Neural NetworkReinforcement LearningTime Series
🎯 What it does: This paper proposes the LocoTouch system, enabling quadruped robots to transport long-distance unconstrained cylinders without containers using distributed tactile sensors on the back.
LodeStar: Long-horizon Dexterity via Synthetic Data Augmentation from Human Demonstrations
Weikang Wan (University of California San Diego), Hao Su (University of California San Diego)
Data SynthesisDomain AdaptationRobotic IntelligenceTransformerReinforcement LearningVision Language ModelVideoSequential
🎯 What it does: This paper proposes the LODESTAR framework, which automatically segments a few human demonstrations into semantic skills, generates diverse synthetic data through residual reinforcement learning in simulation, and concatenates these skills using a Skill Routing Transformer to achieve long-term dexterous manipulation.
Long Range Navigator (LRN): Extending robot planning horizons beyond metric maps
Matt Schmittle (University of Washington), Siddhartha Srinivasa (University of Washington)
Robotic IntelligenceTransformerSupervised Fine-TuningVideo
🎯 What it does: Propose Long Range Navigator (LRN), which learns frontier directions using camera images to extend the navigation horizon of mobile robots.
Long-VLA: Unleashing Long-Horizon Capability of Vision Language Action Model for Robot Manipulation
Yiguo Fan (Westlake University), Donglin Wang (Westlake University)
Robotic IntelligenceTransformerVision-Language-Action ModelDiffusion modelContrastive LearningMultimodalitySequential
🎯 What it does: Propose a unified end-to-end Vision-Language-Action model called Long-VLA, specifically designed to address skill chaining problems in long-horizon robotic manipulation.
Lucid-XR: An Extended-Reality Data Engine for Robotic Manipulation
Yajvan Ravan (MIT), Ge Yang (MIT)
Data SynthesisRobotic IntelligenceTransformerPrompt EngineeringVision-Language-Action ModelDiffusion modelScore-based ModelImageTextMultimodality
🎯 What it does: Proposed Lucid-XR, a full-process data engine that runs real-time physics simulations on XR devices and generates multi-view synthetic images through human demonstrations for training visual manipulation strategies that can be directly transferred to real robots.
ManiFlow: A General Robot Manipulation Policy via Consistency Flow Training
Ge Yan (University of Washington), Dieter Fox (University of Washington)
Robotic IntelligenceTransformerVision-Language-Action ModelFlow-based ModelMultimodality
🎯 What it does: Developed a robot manipulation strategy called ManiFlow, which achieves high-precision dexterous manipulation of multimodal inputs (visual, language, perception) through a small number of reasoning steps;
ManipBench: Benchmarking Vision-Language Models for Low-Level Robot Manipulation
Enyu Zhao (University of Southern California), Daniel Seita (University of Southern California)
Robotic IntelligenceTransformerLarge Language ModelPrompt EngineeringImageVideoTextMultimodalityBenchmarkChain-of-Thought
🎯 What it does: Proposes ManipBench, a benchmark for evaluating visual language models' low-level robotic manipulation reasoning capabilities, containing 12,617 multiple-choice questions covering tasks such as grasping, articulated objects, deformable objects, tools, and dynamic operations;
Mastering Multi-Drone Volleyball through Hierarchical Co-Self-Play Reinforcement Learning
Ruize Zhang (Tsinghua University), Yu Wang (Tsinghua University)
Robotic IntelligenceReinforcement Learning
🎯 What it does: Research and solve the 3v3 multi-drone volleyball task, proposing the Hierarchical Co-Self-Play (HCSP) framework that separates high-level strategic decision-making from low-level motion control, enabling strategies and skills to spontaneously evolve from scratch through a three-stage training process.
Mechanistic Interpretability for Steering Vision-Language-Action Models
Bear Häon (University of California, Berkeley), Claire Tomlin (University of California, Berkeley)
Explainability and InterpretabilityRobotic IntelligenceTransformerVision-Language-Action ModelMultimodalitySequential
🎯 What it does: This paper investigates how to utilize the internal representations of vision-language-action models (VLA) to achieve explainable real-time control, proposing an activation layer injection method that enables zero-copy adjustment of robot behavior during inference without additional training, and verifies its effectiveness on simulated and real robots.
MEReQ: Max-Ent Residual-Q Inverse RL for Sample-Efficient Alignment from Intervention
Yuxin Chen (University of California, Berkeley), Masayoshi Tomizuka (University of California, Berkeley)
Robotic IntelligenceReinforcement Learning from Human FeedbackSupervised Fine-TuningReinforcement Learning
🎯 What it does: Propose a human-intervention-based inverse reinforcement learning framework called MEREQ, which infers human preferences using a residual reward function and fine-tunes existing policies through residual Q-learning (RQL) to align robot behavior with human preferences.
Merging and Disentangling Views in Visual Reinforcement Learning for Robotic Manipulation
Abdulaziz Almuzairee (University of California San Diego), Henrik I Christensen (University of California San Diego)
Robotic IntelligenceConvolutional Neural NetworkReinforcement LearningImage
🎯 What it does: A MAD framework is proposed in robotic visual reinforcement learning that integrates multi-view features and decouples perspectives.
Meta-Optimization and Program Search using Language Models for Task and Motion Planning
Denis Shcherba (TU Berlin), Marc Toussaint (TU Berlin)
OptimizationRobotic IntelligenceMeta LearningTransformerLarge Language Model
🎯 What it does: Propose a hierarchical framework named MOPS that utilizes language models for constraint search, black-box optimization for continuous parameters, and gradient-based optimization to solve trajectories, thereby achieving task and motion planning under language conditions.
MimicFunc: Imitating Tool Manipulation from a Single Human Video via Functional Correspondence
Chao Tang, Hong Zhang
Robotic IntelligenceVision Language ModelVideo
🎯 What it does: Propose a framework called MimicFunc that enables robots to imitate tool manipulation from a single RGB-D human video, allowing robots to understand and replicate tool-use actions in a functional coordinate system and generate trajectories directly usable for training visual-motor policies.
MirrorDuo: Reflection-Consistent Visuomotor Learning from Mirrored Demonstration Pairs
Zheyu Zhuang (KTH Royal Institute of Technology), Danica Kragic (KTH Royal Institute of Technology)
Pose EstimationRobotic IntelligenceConvolutional Neural NetworkDiffusion modelImage
🎯 What it does: Propose MirrorDuo, which generates mirrored pairs of original demonstrations through mirror reflection as data augmentation or structural prior, significantly enhancing the generalization performance of visual behavior cloning in mirror workspaces.
Mobi-$\pi$: Mobilizing Your Robot Learning Policy
Jingyun Yang (Stanford University), Jeannette Bohg (Toyota Research Institute)
Robotic IntelligenceVision Language ModelGaussian SplattingImage
🎯 What it does: Propose and solve the 'policy mobilization' problem, which involves finding robot base poses on mobile robots that match the training distribution of existing visual control policies, enabling deployment of originally static-only control policies in new environments without retraining.
Morphologically Symmetric Reinforcement Learning for Ambidextrous Bimanual Manipulation
Zechu Li (TU Darmstadt), Georgia Chalvatzaki (TU Darmstadt)
Knowledge DistillationRobotic IntelligenceReinforcement Learning
🎯 What it does: Propose the SYMDEX framework, leveraging robot symmetry to train bimanual operable dual-arm/multi-arm grasping and manipulation;
Motion Blender Gaussian Splatting for Dynamic Reconstruction
Xinyu Zhang, Abdeslam Boularias (Rutgers University)
GenerationData SynthesisOptimizationRobotic IntelligenceGaussian SplattingVideo
🎯 What it does: This paper proposes Motion-Blender Gaussian Splatting (MBGS), achieving controllable dynamic scene reconstruction by combining sparse motion graphs with 3D Gaussian splatting;
Motion Priors Reimagined: Adapting Flat-Terrain Skills for Complex Quadruped Mobility
Zewei Zhang (École Polytechnique Fédérale de Lausanne), Marco Hutter (ETH Zurich)
Robotic IntelligenceReinforcement LearningPoint CloudSequential
🎯 What it does: Propose a hierarchical reinforcement learning framework that first pre-trains a low-level motion prior using animal motion capture data on flat ground, then learns residual corrections at the high level, enabling quadruped robots to achieve perceptual gaits and local navigation on complex terrains;
MoTo: A Zero-shot Plug-in Interaction-aware Navigation for General Mobile Manipulation
Zhenyu Wu (Beijing University of Posts and Telecommunications), Haibin Yan (Beijing University of Posts and Telecommunications)
OptimizationRobotic IntelligenceTransformerPrompt EngineeringVision Language ModelSimultaneous Localization and MappingImageTextMultimodalityPoint Cloud
🎯 What it does: Propose a zero-shot pluggable module called MoTo, enabling any fixed-base manipulation model to achieve mobile manipulation through interactive perception navigation, keypoint generation, and trajectory optimization, realizing a closed-loop from environment perception to action execution.
Multi-critic Learning for Whole-body End-effector Twist Tracking
Aravind Elanjimattathil Vijayan (ETH Zurich), Marco Hutter (ETH Zurich)
Robotic IntelligenceReinforcement Learning
🎯 What it does: Designed and implemented a multi-arm multi-task reinforcement learning framework for quadruped robots to achieve continuous and smooth end-effector pose and velocity tracking during locomotion.
Multi-Loco: Unifying Multi-Embodiment Legged Locomotion via Reinforcement Learning Augmented Diffusion
Shunpeng Yang (Southern University of Science and Technology), Hua Chen (Zhejiang University-University of Illinois Urbana-Champaign Institute)
Robotic IntelligenceTransformerReinforcement LearningDiffusion modelSequential
🎯 What it does: Propose a unified framework, Multi-Loco, which combines morphology-agnostic diffusion generative models with a shared residual reinforcement learning strategy to achieve motion control across various legged robots (humanoid, bipedal, wheeled bipedal, quadrupedal).
Multimodal Fused Learning for Solving the Generalized Traveling Salesman Problem in Robotic Task Planning
Jiaqi Cheng (Central South University), Guillaume Adrien Sartoretti (National University of Singapore)
OptimizationRobotic IntelligenceGraph Neural NetworkTransformerReinforcement LearningImageMultimodalityGraphBenchmark
🎯 What it does: Developed a multimodal fusion learning framework, MMFL, for solving the generalized traveling salesman problem in mobile robot task planning, combining graph structures with spatial image representations to achieve high-quality real-time path planning.
Neural Robot Dynamics
Jie Xu, Yashraj Narang (NVIDIA)
Robotic IntelligenceTransformerSupervised Fine-TuningReinforcement LearningSequential
🎯 What it does: Proposed and trained NeRD, a general neural dynamics simulator for articulated robots, capable of maintaining high precision and stability over long time steps, and supporting reasoning across multiple tasks, environments, and controllers.
NeuralSVCD for Efficient Swept Volume Collision Detection
Hojin Jung (Korea Advanced Institute of Science and Technology), Beomjoon Kim (Korea Advanced Institute of Science and Technology)
Robotic IntelligencePoint CloudMesh
🎯 What it does: Proposed a continuous collision detection framework called NeuralSVCD, which achieves efficient and accurate swept volume collision detection in robot motion planning by utilizing distributed latent representations and a neural decoder.
Non-conflicting Energy Minimization in Reinforcement Learning based Robot Control
Skand Peri (Oregon State University), Stefan Lee (Oregon State University)
Robotic IntelligenceReinforcement LearningBenchmark
🎯 What it does: Propose a hyperparameter-free gradient projection method called PEGrad, which trains RL controllers to significantly reduce energy consumption without sacrificing task performance.
O$^3$Afford: One-Shot 3D Object-to-Object Affordance Grounding for Generalizable Robotic Manipulation
Tongxuan Tian (University of Virginia), Yen-Ling Kuo (University of Virginia)
Robotic IntelligenceTransformerLarge Language ModelVision-Language-Action ModelImagePoint Cloud
🎯 What it does: Propose a 3D object-to-object affordance grounding framework O3Afford based on a single sample, which utilizes the DINOv2 semantic features from multi-view RGB-D data projected onto point clouds, combines a bidirectional attention Transformer to predict point-level interaction probabilities, and employs an LLM to automatically generate constraint functions that guide the robot in performing multi-step manipulations.
ObjectReact: Learning Object-Relative Control for Visual Navigation
Sourav Garg (University of Adelaide), Ian Reid (University of Adelaide)
Object DetectionDepth EstimationRobotic IntelligenceGraph Neural NetworkVision-Language-Action ModelImagePoint Cloud
🎯 What it does: This paper proposes a vision navigation framework called ObjectReact based on object-relative control, which constructs an object-level connectivity graph using a relative 3D scene graph, and generates a WayObject Costmap through object segmentation and path length, training a local controller without requiring RGB input.
Off Policy Lyapunov Stability in Reinforcement Learning
Sarvan Gill (University of Victoria), Daniela Constantinescu (University of Victoria)
Supervised Fine-TuningReinforcement Learning
🎯 What it does: Proposes a method for self-learning a Lyapunov function using offline (off-policy) data, integrating it into Soft Actor-Critic (SAC) and Proximal Policy Optimization (PPO) to achieve sample-efficient and stable reinforcement learning.
Omni-Perception: Omnidirectional Collision Avoidance of Legged Robots in Dynamic Environments
Zifan Wang (Hong Kong University of Science and Technology), Junwei Liang (Hong Kong University of Science and Technology)
Robotic IntelligenceRecurrent Neural NetworkReinforcement LearningPoint Cloud
🎯 What it does: Developed the Omni-Perception framework, achieving omnidirectional collision avoidance and efficient motion control for legged robots through end-to-end reinforcement learning using raw LiDAR point clouds.
One Demo is Worth a Thousand Trajectories: Action-View Augmentation for Visuomotor Policies
Chuer Pan (Stanford University), Shuran Song (Stanford University)
Data SynthesisOptimizationRobotic IntelligenceDiffusion modelGaussian SplattingSimultaneous Localization and MappingImagePoint Cloud
🎯 What it does: This paper proposes the 1001 DEMOS framework, which reconstructs scenes using demonstration videos recorded by a single monocular intraocular fisheye camera. The framework employs 3D Gaussian splatting (modified for fisheye versions) to reconstruct the scene, then generates a large number of physically feasible, collision-safe, and diverse action trajectories through trajectory optimization. Corresponding visual sequences are rendered via 3DGS, enabling the generation of thousands of perspective-action pairs of augmented data from a single demonstration.
One View, Many Worlds: Single-Image to 3D object Meets Generative Domain Randomization for One-Shot 6D Pose Estimation
Zheng Geng (Beijing Academy of Artificial Intelligence), Hao Zhao (Nanyang Technological University)
Data SynthesisPose EstimationImageMesh
🎯 What it does: Propose a complete pipeline that directly generates textured 3D models from a single reference image and performs 6D object pose estimation, solving the one-time pose estimation problem in the absence of 3D models.
OPAL: Visibility-aware LiDAR-to-OpenStreetMap Place Recognition via Adaptive Radial Fusion
Shuhao Kang (Technical University of Munich), Daniel Cremers (Technical University of Munich)
RetrievalAutonomous DrivingConvolutional Neural NetworkPoint Cloud
🎯 What it does: Proposes a scene recognition framework named OPAL for mapping single-frame LiDAR data to OpenStreetMap (OSM), achieving meter-level localization with only a single LiDAR scan.
ParticleFormer: A 3D Point Cloud World Model for Multi-Object, Multi-Material Robotic Manipulation
Suning Huang (Stanford University), Mac Schwager
Robotic IntelligenceTransformerWorld ModelImagePoint Cloud
🎯 What it does: Proposed ParticleFormer, a Transformer-based 3D point cloud world model capable of learning robot interaction dynamics with multiple materials and objects, and directly applied to Model Predictive Control (MPC).
Phantom: Training Robots Without Robots Using Only Human Videos
Marion Lepert (Stanford University), Jeannette Bohg (Stanford University)
Image HarmonizationRestorationData SynthesisPose EstimationDomain AdaptationRobotic IntelligenceDiffusion modelVideo
🎯 What it does: This paper proposes a method to directly train robot manipulation policies from human videos without requiring robot data, utilizing hand pose estimation to generate action labels and image editing to replace human arms with robot models, ultimately achieving zero-shot deployment.
PicoPose: Progressive Pixel-to-Pixel Correspondence Learning for Novel Object Pose Estimation
Lihua Liu, Kui Jia (Chinese University of Hong Kong, Shenzhen)
Pose EstimationTransformerContrastive LearningImage
🎯 What it does: Proposes the PicoPose framework, achieving zero-shot new object pose estimation using only RGB images through three-stage pixel-level correspondence learning.
Planning from Point Clouds over Continuous Actions for Multi-object Rearrangement
Kallol Saha (Carnegie Mellon University), David Held (Carnegie Mellon University)
OptimizationRobotic IntelligencePoint Cloud
🎯 What it does: Propose a hybrid learning-planning framework called SPOT based on A* search, which directly plans multi-object rearrangement tasks in point cloud space without discretizing actions or relations.
Point Policy: Unifying Observations and Actions with Key Points for Robot Manipulation
Siddhant Haldar (New York University), Lerrel Pinto (New York University)
Pose EstimationRobotic IntelligenceTransformerVideo
🎯 What it does: This paper proposes Point Policy, which learns robot control policies using offline human videos, completely without requiring robot demonstrations or online interaction.
Pointing3D: A Benchmark for 3D Object Referral via Pointing Gestures
Mert Arslanoglu (RWTH Aachen University), Bastian Leibe (RWTH Aachen University)
SegmentationPose EstimationConvolutional Neural NetworkTransformerImagePoint CloudBenchmark
🎯 What it does: Proposed a 3D object segmentation task based on pointing gestures, constructed the POINTR3D dataset, and implemented a two-stage Transformer model named Pointing3D, completing the end-to-end pointing prediction and 3D segmentation process.
Poke and Strike: Learning Task-Informed Exploration Policies
Marina Y. Aoyama (University of Edinburgh), Sethu Vijayakumar (University of Edinburgh)
Robotic IntelligenceRecurrent Neural NetworkTransformerReinforcement LearningTabularTime SeriesPhysics Related
🎯 What it does: This paper proposes a task-information-driven exploration strategy, which uses reinforcement learning to train exploration policies and online attribute estimators, enabling the agent to first identify key information through physical interaction in one-time tasks and then immediately execute the task;
Predictive Red Teaming: Breaking Policies Without Breaking Robots
Anirudha Majumdar (Google DeepMind), Vikas Sindhwani (Google DeepMind)
GenerationAnomaly DetectionRobotic IntelligenceTransformerDiffusion modelImage
🎯 What it does: Propose a predictive red teaming technique that uses generative image editing and anomaly detection to predict performance degradation of visual motion strategies under different environmental factors without requiring hardware experiments.
PrioriTouch: Adapting to User Contact Preferences for Whole-Arm Physical Human-Robot Interaction
Rishabh Madan, Tapomayukh Bhattacharjee (Cornell University)
OptimizationRobotic IntelligenceReinforcement LearningTabular
🎯 What it does: Design and validate the PrioriTouch framework to achieve user contact preference learning and control priority sorting in multi-contact physical human-robot interaction.
Pseudo-Simulation for Autonomous Driving
Wei Cao (Robert Bosch GmbH), Kashyap Chitta (University of Tübingen)
Autonomous DrivingGaussian SplattingImageVideoPoint Cloud
🎯 What it does: Proposed a novel autonomous driving evaluation paradigm called Pseudo-Simulation, combining real-world data with pre-rendered synthetic observations to assess vehicle behavior in a more efficient and reproducible manner.
QuaDreamer: Controllable Panoramic Video Generation for Quadruped Robots
Sheng Wu (Hunan University), Kailun Yang (Zhejiang University)
GenerationRobotic IntelligenceConvolutional Neural NetworkDiffusion modelVideo
🎯 What it does: Proposed a controllable panoramic video generation framework called QuaDreamer for quadruped robots, which can synthesize panoramic videos with realistic vibrations and precise object movements based on single-frame images, object trajectories, and robot vibration signals.
Rapid Mismatch Estimation via Neural Network Informed Variational Inference
Mateusz Jaszczuk (University of Pennsylvania), Nadia Figueroa (University of Pennsylvania)
Robotic IntelligenceTransformerTabularTime Series
🎯 What it does: Designed and implemented the Rapid Mismatch Estimation (RME) framework, which utilizes torque feedback from the robot's body to rapidly (approximately 400 ms) estimate the mass and center-of-mass mismatch of the end-effector online, and instantly compensates in a passive impedance controller to maintain task tracking and system passivity.
Reactive In-Air Clothing Manipulation with Confidence-Aware Dense Correspondence and Visuotactile Affordance
Neha Sunil (Massachusetts Institute of Technology), Alberto Rodriguez Garcia (Massachusetts Institute of Technology)
Robotic IntelligenceConvolutional Neural NetworkSupervised Fine-TuningContrastive LearningImageVideoMultimodality
🎯 What it does: Developed a dual-arm visual-tactile framework that directly manipulates clothing in highly occluded folded and hanging states using confidence-aware dense visual correspondence and tactile-supervised grasping affinity.
Real-Time Out-of-Distribution Failure Prevention via Multi-Modal Reasoning
Milan Ganai (Stanford University), Marco Pavone (Stanford University)
Autonomous DrivingSafty and PrivacyRobotic IntelligenceTransformerLarge Language ModelVision Language ModelMultimodality
🎯 What it does: This work proposes the FORTRESS framework, which leverages multimodal foundation models to preemptively identify potential failure modes under offline low-frequency conditions, generating fallback goals and semantic safety constraints. It rapidly plans semantically safe fallback paths in real-time triggering scenarios, thereby preventing catastrophic failures in OOD (out-of-distribution) environments for robots.
Real2Render2Real: Scaling Robot Data Without Dynamics Simulation or Robot Hardware
Justin Yu (University of California, Berkeley), Ken Goldberg (Toyota Research Institute)
Object TrackingSegmentationData SynthesisRobotic IntelligenceDiffusion modelGaussian SplattingVideoPoint Cloud
🎯 What it does: Propose a pipeline for generating large-scale robot training data from smartphone scans and single-person demonstration videos, entirely without relying on dynamics simulation or robot hardware.
ReasonPlan: Unified Scene Prediction and Decision Reasoning for Closed-loop Autonomous Driving
Xueyi Liu (Chinese Academy of Sciences), Chen Huiyong
Autonomous DrivingTransformerLarge Language ModelVision-Language-Action ModelMultimodalityChain-of-Thought
🎯 What it does: Propose ReasonPlan, a closed-loop driving framework based on multi-modal large language models (LLMs), which integrates visual information and enables interpretable decision-making through self-supervised next scene prediction (NSP) and supervised decision chain-of-thought (DeCoT); simultaneously constructs a planning-oriented decision reasoning dataset PDR with 210k samples.
ReCoDe: Reinforcement Learning-based Dynamic Constraint Design for Multi-Agent Coordination
Michael Amir (University of Cambridge), Amanda Prorok (University of Cambridge)
OptimizationGraph Neural NetworkReinforcement LearningGraph
🎯 What it does: Propose a ReCoDe framework that, in multi-agent navigation and consensus tasks, uses GNN to enable each agent to dynamically learn and generate additional quadratic constraints while maintaining the safety constraints of expert controllers, thereby enhancing collaborative efficiency.
Reflective Planning: Vision-Language Models for Multi-Stage Long-Horizon Robotic Manipulation
Yunhai Feng (Cornell University), Jianlan Luo (UC Berkeley)
Robotic IntelligenceTransformerPrompt EngineeringDiffusion modelMultimodality
🎯 What it does: Propose a method that uses reflection mechanisms and diffusion dynamics models during testing to enhance the physical reasoning and planning capabilities of visual language models (VLMs) in multi-stage, long-horizon robotic manipulation tasks.
Residual Neural Terminal Constraint for MPC-based Collision Avoidance in Dynamic Environments
Bojan Derajic (Technical University of Berlin), Wolfgang Hönig (Technical University of Berlin)
Autonomous DrivingOptimizationConvolutional Neural NetworkImagePoint CloudTime Series
🎯 What it does: Propose a terminal constraint MPC based on residual neural networks, which learns to approximate the maximum safe set in dynamic environments;