CoRL 2025 Papers — Page 3
Conference on Robot Learning · 263 papers
ReWiND: Language-Guided Rewards Teach Robot Policies without New Demonstrations
Jiahui Zhang (University of Southern California), Jesse Zhang (University of Southern California)
Robotic IntelligenceReinforcement Learning from Human FeedbackTransformerReinforcement LearningVision Language ModelVideoTextMultimodalityBenchmark
🎯 What it does: Learn a language-guided reward function using a small amount of demonstration data, and use this reward function to train and fine-tune robot policies, achieving efficient learning on unseen tasks.
RICL: Adding In-Context Adaptability to Pre-Trained Vision-Language-Action Models
Kaustubh Sridhar (University of British Columbia), Insup Lee (University of British Columbia)
RetrievalRobotic IntelligenceMeta LearningTransformerLarge Language ModelSupervised Fine-TuningPrompt EngineeringVision-Language-Action ModelMultimodalityRetrieval-Augmented Generation
🎯 What it does: This paper proposes a post-training method called RICL, which converts a pre-trained vision-language-action model (VLA) into a model with retrieval-augmented context learning (RAG+ICL) capabilities, enabling it to quickly adapt to new tasks with only 10-20 demonstration examples.
RoboArena: Distributed Real-World Evaluation of Generalist Robot Policies
Pranav Atreya (University Of California Berkeley), Sergey Levine (University Of California Berkeley)
Robotic IntelligenceLarge Language ModelVision Language Model
🎯 What it does: Proposed a distributed, decentralized robotic evaluation framework called RoboArena, which evaluates general robot strategies through crowdsourced double-blind A/B comparisons;
RoboChemist: Long-Horizon and Safety-Compliant Robotic Chemical Experimentation
Zongzheng Zhang (Beijing Academy of Artificial Intelligence), Hao Zhao (Beijing Academy of Artificial Intelligence)
Robotic IntelligencePrompt EngineeringVision Language ModelVision-Language-Action ModelImageTabular
🎯 What it does: Propose a dual-loop framework named RoboChemist, which employs a Vision Language Model (VLM) for planning, visual prompt generation, and monitoring, combined with a Vision Language Action Model (VLA) to achieve safe and compliant robot chemistry experiments.
RoboMonkey: Scaling Test-Time Sampling and Verification for Vision-Language-Action Models
Jacky Kwok (Stanford University), Marco Pavone (Stanford University)
Robotic IntelligenceReinforcement Learning from Human FeedbackLarge Language ModelSupervised Fine-TuningVision-Language-Action ModelMultimodality
🎯 What it does: Proposed the RoboMonkey framework, which significantly improves the accuracy and robustness of robot control by performing multiple samplings of Vision-Language-Action models during deployment and verifying them using a VLM-based validator.
Robot Learning from Any Images
Siheng Zhao (University of Southern California), Yue Wang (University of Southern California)
Image HarmonizationGenerationData SynthesisRobotic IntelligenceLarge Language ModelReinforcement LearningVision Language ModelVision-Language-Action ModelDiffusion modelImageText
🎯 What it does: Convert any single image into an interactive, physically simulatable robot environment and generate robot demonstrations at scale for training and deploying robot policies.
Robot Operating Home Appliances by Reading User Manuals
Jian Zhang (National University of Singapore), David Hsu (National University of Singapore)
Object DetectionRobotic IntelligenceTransformerLarge Language ModelVision Language ModelImageText
🎯 What it does: Propose ApBot, which generates structured appliance models using user manuals, enabling robots to operate various home appliances via natural language instructions in zero-shot scenarios.
Robot Trains Robot: Automatic Real-World Policy Adaptation and Learning for Humanoids
Kaizhe Hu (Stanford University), Shuran Song (Stanford University)
Domain AdaptationOptimizationRobotic IntelligenceReinforcement LearningTabularTime Series
🎯 What it does: Propose the Robot-Trains-Robot (RTR) framework, which enables long-term real-world learning without human intervention by providing safety protection, reward signals, dynamic scheduling, and automatic reset functions through a robotic arm teacher for a humanoid robot student; additionally, design a fast fine-tuning pipeline based on dynamic latent variables for rapid transfer from pre-training to real-world precision tasks.
RobotxR1: Enabling Embodied Robotic Intelligence on Large Language Models through Closed-Loop Reinforcement Learning
Liam Boyle (ETH Zurich), Luca Benini (ETH Zurich)
Autonomous DrivingRobotic IntelligenceTransformerSupervised Fine-TuningReinforcement LearningTime SeriesRetrieval-Augmented Generation
🎯 What it does: Integrate low-parameter LLMs into robot decision-making and control through closed-loop reinforcement learning to achieve embodied AI driving behavior.
Robust Dexterous Grasping of General Objects
Hui Zhang (ETH Zurich), Jie Song (HKUST)
Robotic IntelligenceRecurrent Neural NetworkReinforcement LearningImage
🎯 What it does: Train a zero-shot multi-finger grasp controller under single-view visual input to achieve robust grasping on hundreds of unknown objects (including thin, light, heavy, deformable, transparent objects, etc.).
SafeBimanual: Diffusion-based trajectory optimization for safe bimanual manipulation
Haoyuan Deng (Nanyang Technological University), Ziwei Wang (Nanyang Technological University)
OptimizationRobotic IntelligenceDiffusion modelMultimodality
🎯 What it does: This paper proposes the SafeBimanual framework, which optimizes safe trajectories for pre-trained bimanual diffusion policies during testing by guiding the sampling of the diffusion process, thereby avoiding hazardous robot behaviors and improving success rates.
SAIL: Faster-than-Demonstration Execution of Imitation Learning Policies
Nadun Ranawaka Arachchige (Georgia Institute of Technology), Danfei Xu (Georgia Institute of Technology)
Computational EfficiencyRobotic IntelligenceDiffusion model
🎯 What it does: Propose the SAIL framework, which enables robots to execute tasks at speeds exceeding the original demonstration speed based on offline imitation learning strategies, achieving higher task throughput.
Sample-Efficient Online Control Policy Learning with Real-Time Recursive Model Updates
Zixin Zhang (Northwestern University), Todd Murphey
Robotic IntelligenceWorld Model
🎯 What it does: Proposed a recursive Koopman learning (RKL) pipeline to achieve real-time recursive model updates and sample-efficient control strategy learning.
Sampling-based System Identification with Active Exploration for Legged Sim2Real Learning
Nikhil Sobanbabu (Carnegie Mellon University), Guanya Shi (Carnegie Mellon University)
Domain AdaptationOptimizationRobotic IntelligenceReinforcement LearningTime Series
🎯 What it does: Proposes a two-stage active sampling system identification framework (SPI-Active), which estimates the physical parameters of legged robots through parallel sampling and combines active exploration to maximize Fisher information, thereby improving the accuracy of simulation-to-reality transfer.
SAVOR: Skill Affordance Learning from Visuo-Haptic Perception for Robot-Assisted Bite Acquisition
Zhanxin Wu (Cornell University), Tapomayukh Bhattacharjee (University of California San Diego)
Robotic IntelligenceVision Language ModelVision-Language-Action ModelImageMultimodality
🎯 What it does: This paper proposes the SAVOR framework, which integrates tool affordance and food affordance, utilizing visual-tactile perception to achieve biting operations in robot-assisted meal assistance;
ScrewSplat: An End-to-End Method for Articulated Object Recognition
Seungyeon Kim (Seoul National University), Frank C. Park (Seoul National University)
RecognitionPose EstimationVision Language ModelGaussian SplattingImage
🎯 What it does: This paper proposes an end-to-end method called ScrewSplat, which identifies and reconstructs the geometric and motion structure of articulated objects using only RGB images, and enables text-guided manipulation.
SDS – See it, Do it, Sorted: Quadruped Skill Synthesis from Single Video Demonstration
Maria Stamatopoulou (University College London), Dimitrios Kanoulas (University College London)
Pose EstimationRobotic IntelligenceReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningPrompt EngineeringVision Language ModelVideoMultimodality
🎯 What it does: Leverages a single unlabeled video demonstration, combined with GPT-4o to automatically generate an executable reward function, training a quadruped robot (Unitree Go1) via PPO in IsaacGym, achieving the learning and field deployment of four gaits (trotting, bound, pace, hop).
Search-TTA: A Multi-Modal Test-Time Adaptation Framework for Visual Search in the Wild
Derek Ming Siang Tan (National University of Singapore), Guillaume Adrien Sartoretti (National University of Singapore)
RetrievalDomain AdaptationRobotic IntelligenceTransformerReinforcement LearningVision Language ModelContrastive LearningImageTextMultimodalityBenchmarkAudio
🎯 What it does: Propose a multimodal test-time adaptation framework called Search-TTA for utilizing satellite images to guide robots in visual search tasks in the wild.
See, Point, Fly: A Learning-Free VLM Framework for Universal Unmanned Aerial Navigation
Chih Yao Hu (National Taiwan University), Yu-Lun Liu (National Yang Ming Chiao Tung University)
Autonomous DrivingRobotic IntelligencePrompt EngineeringVision Language ModelImageTextMultimodality
🎯 What it does: Proposed a zero-training UAV visual language navigation framework (SPF), which generates 2D waypoints through a frozen visual language model and converts them into 3D actions, enabling UAV navigation in any environment and with arbitrary free-text instructions.
Self-supervised Learning Of Visual Pose Estimation Without Pose Labels By Classifying LED States
Nicholas Carlotti (Dalle Molle Institute for Artificial Intelligence), Alessandro Giusti (Dalle Molle Institute for Artificial Intelligence)
Pose EstimationRobotic IntelligenceConvolutional Neural NetworkImage
🎯 What it does: Propose a self-supervised visual pose estimation method that uses the LED states on a robot, which can be independently turned on/off, as a pre-training task. The method learns localization and orientation estimation without requiring pose labels or CAD models.
Self-supervised perception for tactile skin covered dexterous hands
Akash Sharma (FAIR at Meta), Mustafa Mukadam (Carnegie Mellon University)
Representation LearningRobotic IntelligenceTransformerAuto EncoderContrastive LearningMultimodalityTime Series
🎯 What it does: Developed Sparsh-skin, a full-hand tactile encoder based on self-supervised self-distillation, which can learn hand tactile representations from approximately 4 hours of random play data, mapping short-term tactile + hand pose inputs to potential embeddings directly usable for multiple downstream tasks.
Sequence Modeling for Time-Optimal Quadrotor Trajectory Optimization with Sampling-based Robustness Analysis
Katherine Mao (University of Pennsylvania), Vijay Kumar (University of Pennsylvania)
OptimizationRobotic IntelligenceRecurrent Neural NetworkTime SeriesSequential
🎯 What it does: Leverage a learning model to imitate an optimization-based time-optimal trajectory planner, rapidly generating trajectories executable in real-time on quadrotors.
Shortcut Learning in Generalist Robot Policies: The Role of Dataset Diversity and Fragmentation
Youguang Xing (University of Electronic Science and Technology of China), Jingkuan Song (University of Electronic Science and Technology of China)
Robotic IntelligenceVision Language ModelMultimodality
🎯 What it does: Investigated the shortcut learning problem present in large robot datasets (e.g., OXE), revealing that insufficient data diversity and subdataset fragmentation lead to reduced generalization capabilities, and proposed using data augmentation to alleviate this issue.
Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids
Toru Lin (UC Berkeley), Yuke Zhu
SegmentationDomain AdaptationHyperparameter SearchRobotic IntelligenceReinforcement LearningImage
🎯 What it does: Under a sim-to-real reinforcement learning framework, a humanoid robot equipped with dual multi-fingered hands was trained and deployed to perform visually-based high-precision manipulation tasks, including grasping and stretching, lifting boxes, and hand-over operations.
SimShear: Sim-to-Real Shear-based Tactile Servoing
Kipp Freud, Nathan F. Lepora (University of Bristol)
Image TranslationPose EstimationDomain AdaptationRobotic IntelligenceConvolutional Neural NetworkGenerative Adversarial NetworkImageTabular
🎯 What it does: By establishing a shear-aware transformation pipeline from simulation to reality, tactile control can leverage shear information to achieve more precise object tracking and co-lifting.
SIREN: Semantic, Initialization-Free Registration of Multi-Robot Gaussian Splatting Maps
Olao Shorinwa, Anirudha Majumdar (Princeton University)
OptimizationRobotic IntelligenceGaussian SplattingSimultaneous Localization and MappingImagePoint Cloud
🎯 What it does: Propose the SIREN algorithm to achieve registration of multi-robot Gaussian Splatting maps without relying on camera poses, images, or initial transformations.
SLAC: Simulation-Pretrained Latent Action Space for Whole-Body Real-World RL
Jiaheng Hu (University of Texas at Austin), Roberto Martín-Martín (University of Texas at Austin)
Robotic IntelligenceReinforcement LearningWorld Model
🎯 What it does: Propose the SLAC framework, which utilizes low-fidelity simulation to pre-train a task-agnostic latent action space, and then employs offline reinforcement learning in the real world to accomplish high-degree-of-freedom robotic tasks.
SocialNav-SUB: Benchmarking VLMs for Scene Understanding in Social Robot Navigation
Michael Joseph Munje (University of Texas at Austin), Peter Stone (University of Texas at Austin)
Robotic IntelligenceVision Language ModelImageBenchmarkChain-of-Thought
🎯 What it does: Designed and released the SocialNav-SUB benchmark to evaluate the scene understanding capabilities of Vision-Language Models in social robot navigation scenarios
Steerable Scene Generation with Post Training and Inference-Time Search
Nicholas Ezra Pfaff (Massachusetts Institute of Technology), Russ Tedrake (Massachusetts Institute of Technology)
GenerationData SynthesisRobotic IntelligenceTransformerReinforcement LearningDiffusion modelPhysics Related
🎯 What it does: Built and trained a diffusion-based SE(3) scene generation model, then achieved guided generation for downstream objectives (e.g., high crowding, physical feasibility) through post-training with reinforcement learning, text/conditional generation, and MCTS inference search, providing scenes directly usable for robot simulation.
Steering Your Diffusion Policy with Latent Space Reinforcement Learning
Andrew Wagenmaker (UC Berkeley), Sergey Levine (UC Berkeley)
Robotic IntelligenceReinforcement LearningDiffusion modelFlow-based Model
🎯 What it does: Propose a method that utilizes reinforcement learning to 'steer' (diffusion steering) in the latent noise space of diffusion strategies, enabling rapid adaptation of existing BC diffusion strategies without modifying the diffusion model's weights;
Streaming Flow Policy: Simplifying diffusion/flow-matching policies by treating action trajectories as flow trajectories
Sunshine Jiang (Massachusetts Institute of Technology), Siddharth Ancha (Massachusetts Institute of Technology)
Robotic IntelligenceDiffusion modelFlow-based ModelTime SeriesSequentialOrdinary Differential Equation
🎯 What it does: Propose a continuous regularized flow model (Streaming Flow Policy) that treats action trajectories as flow trajectories, enabling real-time streaming sampling and execution of action trajectories.
Subteaming and Adaptive Formation Control for Coordinated Multi-Robot Navigation
Zihao Deng (University Of Massachusetts Amherst), Hao Zhang (University Of Denver)
Robotic IntelligenceGraph Neural NetworkReinforcement Learning
🎯 What it does: This paper proposes the STAF method, which supports multiple robots to dynamically divide into subteams and adaptively maintain formations to complete navigation tasks in complex environments.
Tactile Beyond Pixels: Multisensory Touch Representations for Robot Manipulation
Carolina Higuera (FAIR at Meta), Mustafa Mukadam
Representation LearningRobotic IntelligenceTransformerReinforcement LearningContrastive LearningMultimodality
🎯 What it does: Proposes Sparsh-X, a self-supervised multimodal tactile representation that integrates image, audio, motion, and pressure modalities, and applies it to manipulation tasks such as plugging/unplugging and in-hand rotation on real robots.
Text2Touch: Tactile In-Hand Manipulation with LLM-Designed Reward Functions
Harrison Field (University of Bristol), Nathan F. Lepora (University of Bristol)
Domain AdaptationKnowledge DistillationRobotic IntelligenceTransformerLarge Language ModelReinforcement LearningPrompt EngineeringVision-Language-Action ModelMultimodality
🎯 What it does: Built the Text2Touch framework, leveraging large language models to automatically generate reward functions under tactile feedback, training and implementing multi-axis spinning tactile in-hand manipulation on a real Allegro hand.
The Sound of Simulation: Learning Multimodal Sim-to-Real Robot Policies with Generative Audio
Renhao Wang (University of California, Berkeley), Alexei A Efros
Domain AdaptationRobotic IntelligenceSupervised Fine-TuningVision Language ModelDiffusion modelVideoMultimodalityAudio
🎯 What it does: This paper proposes the MULTIGEN framework, which combines large-scale generative models with physical simulation to synthesize multimodal (visual + audio) data in simulation, enabling training for robot pouring tasks without real-world data and achieving zero-shot transfer to the real world.
ToddlerBot: Open-Source ML-Compatible Humanoid Platform for Loco-Manipulation
Haochen Shi (Stanford University), Karen Liu
Robotic IntelligenceReinforcement LearningDiffusion modelTime SeriesSequential
🎯 What it does: Proposed and implemented ToddlerBot—a low-cost, open-source 30-degree-of-freedom humanoid robot platform for machine learning-driven locomotion and manipulation research; achieved high-fidelity simulation through digital twin, zero-point calibration, and system identification; equipped with an intuitive full-body teleoperation interface for high-quality real-world data collection; demonstrated zero-shot simulation-to-reality transfer for multiple tasks, including walking, pushing, dual-arm manipulation, and full-body manipulation.
Tool-as-Interface: Learning Robot Policies from Observing Human Tool Use
Haonan Chen (Columbia University), Katherine Rose Driggs-Campbell (Columbia University)
Robotic IntelligenceReinforcement Learning from Human FeedbackDiffusion modelGaussian SplattingVideo
🎯 What it does: This paper proposes an extensible 'Tool-as-Interface' framework that trains robots to learn audio-visual motor strategies for various tool-use tasks using ordinary human videos captured by dual cameras.
TopoCut: Learning Multi-Step Cutting with Spectral Rewards and Discrete Diffusion Policies
Liquan Wang (Georgia Institute of Technology), Animesh Garg (Georgia Institute of Technology)
Robotic IntelligenceGraph Neural NetworkReinforcement LearningDiffusion modelScore-based ModelMesh
🎯 What it does: Proposes the TopoCut framework, combining high-precision particle simulation, damage-driven topology tracking, pose-invariant spectral reward, and discrete diffusion strategies to achieve end-to-end learning and evaluation for multi-step flexible object cutting.
Toward Real-World Cooperative and Competitive Soccer with Quadrupedal Robot Teams
Zhi Su (University of California, Berkeley), Koushil Sreenath (University of California, Berkeley)
Robotic IntelligenceRecurrent Neural NetworkReinforcement LearningSimultaneous Localization and MappingPoint Cloud
🎯 What it does: In this paper, the authors propose a hierarchical multi-agent reinforcement learning framework that realizes decentralized football matches with quadruped robots in the real world, covering low-level skills such as walking, dribbling, and shooting, as well as high-level strategy decision-making.
Towards Embodiment Scaling Laws in Robot Locomotion
Bo Ai (University of California San Diego), Hao Su (University of California San Diego)
Robotic IntelligenceTransformerSupervised Fine-TuningReinforcement Learning
🎯 What it does: This paper trains a single cross-body-type policy in robotic gait control, using approximately 1,000 procedurally generated robot body types. It verifies in simulation that increasing the number of body types enhances generalization to unseen body types, and achieves zero-shot transfer to two real robots.
Towards Generalizable Safety in Crowd Navigation via Conformal Uncertainty Handling
Jianpeng Yao (University of California, Riverside), Jiachen Li (University of California, Riverside)
Autonomous DrivingSafty and PrivacyReinforcement LearningPoint CloudSequential
🎯 What it does: Under the reinforcement learning framework, predictive uncertainty generated by Adaptive Conformal Inference (ACI) is integrated into the robot's observational information, and decision-making is guided by Constrained Reinforcement Learning (CRL), thereby enhancing safety and robustness in crowd navigation.
TrackVLA: Embodied Visual Tracking in the Wild
Shaoan Wang (Peking University), He Wang (Peking University)
Object TrackingRobotic IntelligenceTransformerLarge Language ModelVision-Language-Action ModelDiffusion modelVideoTextMultimodalityBenchmark
🎯 What it does: Propose TrackVLA, a visual-language-action framework that unifies target recognition and trajectory planning within a single model for embedded visual tracking.
Train-Once Plan-Anywhere Kinodynamic Motion Planning via Diffusion Trees
Yaniv Hassidof (Technion-Israel Institute of Technology), Kiril Solovey (Technion-Israel Institute of Technology)
Robotic IntelligenceDiffusion modelFlow-based ModelSequential
🎯 What it does: Proposed Diffusion Tree (DiTree), a motion planning framework that combines diffusion models with sampling tree search to rapidly solve collision-free trajectories under dynamic constraints.
Training Strategies for Efficient Embodied Reasoning
William Chen, Sergey Levine (UC Berkeley)
Computational EfficiencyRobotic IntelligenceTransformerVision-Language-Action ModelMultimodalityBenchmarkChain-of-Thought
🎯 What it does: Propose ECoT-Lite, a lightweight training strategy that leverages robot reasoning data to enhance visual-language-action model (VLA) performance without introducing additional inference steps during deployment, maintaining high inference speed.
TReF-6: Inferring Task-Relevant Frames from a Single Demonstration for One-Shot Skill Generalization
Yuxuan Ding (Yale University), Tesca Fitzgerald (Yale University)
OptimizationRobotic IntelligenceLarge Language ModelVision Language Model
🎯 What it does: Infer task-related 6-degree-of-freedom reference frames through trajectory optimization under the condition of a single demonstration, and utilize this framework to achieve cross-scenario generalization of DMP.
TWIST: Teleoperated Whole-Body Imitation System
Yanjie Ze (Stanford University), Karen Liu
Robotic IntelligenceReinforcement LearningTime SeriesSequential
🎯 What it does: Proposed and implemented TWIST, a humanoid robot teleoperation system capable of simulating full-body human motions through real-time full-body motion capture.
TypeTele: Releasing Dexterity in Teleoperation by Dexterous Manipulation Types
Yuhao Lin (Sun Yat-sen University), Wei-Shi Zheng (Sun Yat-sen University)
Robotic IntelligenceTransformerLarge Language ModelDiffusion modelMultimodalityTime SeriesRetrieval-Augmented Generation
🎯 What it does: Proposed the TypeTele system, which utilizes operator-selected manipulation types to control multi-fingered robot hands, overcoming the limitations of traditional hand pose remapping and enabling more complex manipulation tasks.
Uncertainty-aware Accurate Elevation Modeling for Off-road Navigation via Neural Processes
Sanghun Jung (University of Washington), James Hays (Overland AI)
Autonomous DrivingTransformerImageMultimodalityPoint Cloud
🎯 What it does: Proposes a semantic-conditioned neural process (NP) model for real-time high-precision offline terrain height modeling, along with uncertainty prediction.
Uncertainty-aware Latent Safety Filters for Avoiding Out-of-Distribution Failures
Junwon Seo (Carnegie Mellon University), Andrea Bajcsy (Carnegie Mellon University)
World ModelImageTime SeriesSequential
🎯 What it does: Proposes UNISafe, a framework for safe control in latent space that leverages world models and uncertainty-awareness, enabling proactive defense in both known and unknown (OOD) failure scenarios.
Uncertainty-Aware Scene Understanding via Efficient Sampling-Free Confidence Estimation
Hanieh Shojaei Miandashti (Leibniz University Hannover), Claus Brenner (Leibniz University Hannover)
SegmentationAutonomous DrivingPoint CloudBenchmark
🎯 What it does: This paper proposes a sampling-free confidence estimation method for explicitly modeling and quantifying aleatoric uncertainty in LiDAR point cloud semantic segmentation.
UniSkill: Imitating Human Videos via Cross-Embodiment Skill Representations
Hanjung Kim (Yonsei University), Youngwoon Lee (Yonsei University)
Depth EstimationRepresentation LearningRobotic IntelligenceDiffusion modelVideo
🎯 What it does: Learn transferable, body-structure-agnostic skill representations by training on large-scale unlabeled cross-genre videos (human vs. robot), and use these representations to enable robots to imitate humans based on video demonstrations.
UniTac2Pose: A Unified Approach Learned in Simulation for Category-level Visuotactile In-hand Pose Estimation
Mingdong Wu (Peking University), Hao Dong (Peking University)
Pose EstimationDiffusion modelScore-based ModelMultimodalityOrdinary Differential Equation
🎯 What it does: Propose a unified framework called UniTac2Pose based on an energy diffusion model for visual-tactile fusion hand-in-hand object pose estimation, tracking, and uncertainty assessment, which can achieve high-precision pose inference on both seen and unseen objects of the same CAD model category.
UnPose: Uncertainty-Guided Diffusion Priors for Zero-Shot Pose Estimation
Zhaodong Jiang (Huawei Noah's Ark Lab), Binbin Xu (Huawei Noah's Ark Lab)
GenerationPose EstimationDiffusion modelGaussian SplattingImagePoint Cloud
🎯 What it does: Propose a zero-shot, model-free 6D pose estimation framework called UnPose, which leverages the 3D prior and uncertainty estimation from pre-trained diffusion models, achieving for the first time recursive reconstruction and pose inference from single-view RGB-D images to complete 3DGS models;
Unsupervised Skill Discovery as Exploration for Learning Agile Locomotion
Seungeun Rho (Georgia Institute of Technology), Sehoon Ha (Georgia Institute of Technology)
Data SynthesisRobotic IntelligenceReinforcement Learning
🎯 What it does: Proposes a framework named SDAX that utilizes unsupervised skill discovery for high-level exploration in reinforcement learning, enabling quadruped robots to autonomously learn complex agile gaits such as crawling, climbing, jumping, and wall-jumping without explicit reward engineering, demonstration data, or curriculum learning.
Versatile Loco-Manipulation through Flexible Interlimb Coordination
Xinghao Zhu (RAI Institute), Kuan Fang (RAI Institute)
Robotic IntelligenceReinforcement LearningMultimodality
🎯 What it does: Designed an interactive control framework that dynamically allocates leg and arm functions, enabling robots to achieve flexible walking and manipulation tasks in irregular environments.
Vision in Action: Learning Active Perception from Human Demonstrations
Haoyu Xiong (Stanford University), Shuran Song (Stanford University)
Robotic IntelligenceTransformerDiffusion modelImagePoint Cloud
🎯 What it does: Developed the Vision in Action (ViA) system, which learns operational strategies for dual-arm robots through human demonstrations to achieve active perception.
Visual Imitation Enables Contextual Humanoid Control
Arthur Allshire (University Of California Berkeley), Angjoo Kanazawa (University Of California Berkeley)
Pose EstimationDepth EstimationOptimizationReinforcement LearningVideoPoint CloudMesh
🎯 What it does: Proposes the VIDEOMIMIC pipeline, which can simultaneously reconstruct human motion and scene geometry from monocular video. The reconstructed results are then used to train a single general control policy in simulation, which is transferred to the real Unitree G1 humanoid robot to achieve environment-height-map-based adaptive skills (e.g., climbing stairs, sitting up, walking).
VLM-AD: End-to-End Autonomous Driving through Vision-Language Model Supervision
Yi Xu (Cruise LLC), Xin Huang (Waymo LLC)
Autonomous DrivingKnowledge DistillationTransformerVision Language ModelVideoMultimodality
🎯 What it does: Use VLM (GPT-4o) to generate reasoning and action text annotations for end-to-end autonomous driving models as additional supervision;
VT-Refine: Learning Bimanual Assembly with Visuo-Tactile Feedback via Simulation Fine-Tuning
Binghao Huang (Columbia University), Yunzhu Li (Columbia University)
Robotic IntelligenceReinforcement LearningDiffusion modelMultimodalityPoint Cloud
🎯 What it does: This paper proposes a two-stage learning framework: first pre-train diffusion policies using a small amount of real demonstrations and visual + tactile data, then fine-tune through large-scale reinforcement learning in a digital twin simulation environment to achieve precise bimanual assembly.
Wheeled Lab: Modern Sim2Real for Low-cost, Open-source Wheeled Robotics
Tyler Han (University of Washington), Byron Boots (University of Washington)
Data SynthesisDomain AdaptationRobotic IntelligenceConvolutional Neural NetworkReinforcement LearningImagePoint Cloud
🎯 What it does: Built Wheeled Lab, a Sim2Real ecosystem integrating low-cost, open-source wheeled platforms with the Isaac Lab simulation framework, where three zero-shot projection reinforcement learning policies were trained and deployed: drifting, terrain traversal, and visual navigation.
WoMAP: World Models For Embodied Open-Vocabulary Object Localization
Tenny Yin (Princeton University), Lihan Zha (Princeton University)
Object DetectionOptimizationRobotic IntelligenceTransformerVision Language ModelGaussian SplattingWorld ModelVideoMesh
🎯 What it does: Developed a world-model-based open-vocabulary object localization framework, WoMAP, without requiring expert demonstrations.
X-Sim: Cross-Embodiment Learning via Real-to-Sim-to-Real
Prithwish Dan (Cornell University), Sanjiban Choudhury (Cornell University)
Domain AdaptationRobotic IntelligenceReinforcement LearningDiffusion modelContrastive LearningGaussian SplattingImageVideoPoint CloudMesh
🎯 What it does: Propose the X-SIM framework, which utilizes object motion in human videos as a supervisory signal. It first learns RL policies in simulation, then distills them into image-conditioned diffusion policies using multi-view synthetic data, and finally aligns real and simulated images online, achieving sim-to-real transfer with zero teleoperation data.
ZipMPC: Compressed Context-Dependent MPC Cost via Imitation Learning
Rahel Rickenbach (ETH Zurich), Johannes A. Stork (Örebro University)
Autonomous DrivingOptimizationSequential
🎯 What it does: This study learns a compressed and context-aware cost function, enabling short-horizon MPC to mimic the behavior of long-horizon MPC, thereby significantly improving control performance while maintaining real-time computation; the approach is validated on autonomous racing tasks.