CoRL 2024 Papers — Page 3
Conference on Robot Learning · 264 papers
RT-Sketch: Goal-Conditioned Imitation Learning from Hand-Drawn Sketches
Priya Sundaresan (Stanford University), Stefan Schaal (Google Intrinsic)
Image TranslationData SynthesisRobotic IntelligenceTransformerGenerative Adversarial NetworkImage
🎯 What it does: Investigate hand-drawn sketches as target representations, propose RT-Sketch for goal-conditioned imitation learning, and evaluate it on desktop object rearrangement tasks.
Safe Bayesian Optimization for the Control of High-Dimensional Embodied Systems
Yunyue Wei (Tsinghua University), Yanan Sui (Tsinghua University)
OptimizationRobotic IntelligenceTabularBiomedical Data
🎯 What it does: Proposed and implemented HDSAFEBO, which enables safe and efficient online optimization of muscle systems and neural stimulation control in high-dimensional (hundreds to thousands of dimensions) control parameter spaces using Bayesian optimization.
Scaling Cross-Embodied Learning: One Policy for Manipulation, Navigation, Locomotion and Aviation
Ria Doshi (University of California Berkeley), Sergey Levine (University of California Berkeley)
Robotic IntelligenceTransformerReinforcement LearningVision-Language-Action ModelMultimodalitySequential
🎯 What it does: Train a single Transformer policy capable of performing manipulation, navigation, locomotion, and aerial tasks across different robots (single-arm, dual-arm, ground navigation, quadrupedal, drones, etc.).
Scaling Manipulation Learning with Visual Kinematic Chain Prediction
Xinyu Zhang (Rutgers University), Abdeslam Boularias (Rutgers University)
Robotic IntelligenceTransformerVision Language ModelImagePoint CloudBenchmark
🎯 What it does: This study proposes a generic action representation based on visual motion chains, and constructs a convolution-free, multi-view attention Transformer (VKT) to predict robot motion on the image plane, enabling cross-robot, cross-workspace multi-task learning.
Scaling Robot Policy Learning via Zero-Shot Labeling with Foundation Models
Nils Blank (Karlsruhe Institute of Technology), Rudolf Lioutikov (Karlsruhe Institute of Technology)
Object DetectionSegmentationRepresentation LearningData-Centric LearningRobotic IntelligenceReinforcement Learning from Human FeedbackTransformerLarge Language ModelVision Language ModelVideoTextMultimodality
🎯 What it does: Developed a NILS system that automatically segments long-term robot demonstration videos and generates natural language task labels without manual annotation or model fine-tuning, enabling subsequent language-conditioned policy training.
Scaling Safe Multi-Agent Control for Signal Temporal Logic Specifications
Joe Eappen (Purdue University West Lafayette), Suresh Jagannathan (Purdue University West Lafayette)
OptimizationRobotic IntelligenceGraph Neural NetworkGraphBenchmarkOrdinary Differential Equation
🎯 What it does: This study proposes an scalable multi-agent control method to satisfy signal temporal logic (STL) specifications, combining graph neural networks (GNN), neural ODE planner, and a safety controller based on graph control barrier function (GCBF+), achieving end-to-end differentiable learning;
ScissorBot: Learning Generalizable Scissor Skill for Paper Cutting via Simulation, Imitation, and Sim2Real
Jiangran Lyu (Peking University), He Wang (Peking University)
Domain AdaptationRobotic IntelligenceImagePoint Cloud
🎯 What it does: Studied a learning-based robotic paper-cutting system called ScissorBot, which can precisely cut curves on suspended paper using scissors.
SELFI: Autonomous Self-Improvement with RL for Vision-Based Navigation around People
Noriaki Hirose (University of California Berkeley), Sergey Levine (University of California Berkeley)
Robotic IntelligenceReinforcement LearningImage
🎯 What it does: SELFI method combines model-based learning with online model-free reinforcement learning to achieve rapid fine-tuning of pre-trained visual social navigation policies.
SHADOW: Leveraging Segmentation Masks for Cross-Embodiment Policy Transfer
Marion Lepert (Stanford University), Jeannette Bohg (Stanford University)
SegmentationDomain AdaptationRobotic IntelligenceReinforcement LearningDiffusion modelImage
🎯 What it does: Propose the Shadow scheme, achieving cross-robot control without collecting target robot data by overlaying segmentation masks of the source and target robots during training and testing.
Shelf-Supervised Cross-Modal Pre-Training for 3D Object Detection
Mehar Khurana, Deva Ramanan (Georgia Institute of Technology)
Object DetectionAutonomous DrivingKnowledge DistillationTransformerVision Language ModelImageMultimodalityPoint Cloud
🎯 What it does: This paper proposes a cross-modal 3D detection distillation (CM3D) method based on vision-language models (VLMs), generating zero-shot 3D box pseudo labels by combining instance segmentation generated by off-the-shelf 2D VLMs (e.g., Detic, SAM) with LiDAR point cloud projection, HD maps, and shape priors. These pseudo labels are then used to pretrain LiDAR, RGB, or multimodal 3D detectors.
Sim-to-Real Transfer via 3D Feature Fields for Vision-and-Language Navigation
Zihan Wang (Institute of Computing Technology Chinese Academy of Sciences), Shuqiang Jiang (Institute of Computing Technology Chinese Academy of Sciences)
Domain AdaptationRobotic IntelligenceConvolutional Neural NetworkTransformerVision-Language-Action ModelNeural Radiance FieldMultimodality
🎯 What it does: Designed semantic traversability maps and 3D feature fields based on monocular RGB-D cameras, enabling panoramic perception for monocular robots and achieving VLN model transfer from simulation to real environments.
Simple Masked Training Strategies Yield Control Policies That Are Robust to Sensor Failure
Skand Skand, Stefan Lee (Oregon State University)
Robotic IntelligenceConvolutional Neural NetworkTransformerReinforcement LearningMultimodality
🎯 What it does: This paper proposes a robust multi-modal encoder (RME) along with a corresponding modal dropout training strategy, enabling reinforcement learning control policies to maintain stable performance when sensors fail.
SkillMimicGen: Automated Demonstration Generation for Efficient Skill Learning and Deployment
Caelan Reed Garrett (NVIDIA), Dieter Fox (NVIDIA)
Robotic IntelligenceRecurrent Neural NetworkSequential
🎯 What it does: This paper proposes the SkillMimicGen (SkillGen) system, which decomposes tasks into operational skills and free-space motion using a small number of human demonstrations, and concatenates them through motion planning to generate a large amount of demonstration data; simultaneously, the Hybrid Skill Policy (HSP) framework is proposed to learn the initiation, control, and termination of skills, enabling skill sequencing during testing.
SLR: Learning Quadruped Locomotion without Privileged Information
Shiyi Chen (Tsinghua University), Fasih Ud Din Farrukh (Tsinghua University)
Robotic IntelligenceReinforcement LearningContrastive Learning
🎯 What it does: This paper proposes a Self-Learning Latent Representation (SLR) method, which trains a quadruped robot to walk by utilizing only proprioceptive information without any privileged information.
So You Think You Can Scale Up Autonomous Robot Data Collection?
Suvir Mirchandani (Stanford University), Dorsa Sadigh (Stanford University)
Robotic IntelligenceReinforcement LearningDiffusion modelSequential
🎯 What it does: Assessed the scalability of multiple autonomous imitation learning (Autonomous IL) methods in real-world and simulated tasks, and systematically analyzed the trade-offs between environment design and human supervision costs.
SoftManiSim: A Fast Simulation Framework for Multi-Segment Continuum Manipulators Tailored for Robot Learning
Mohammadreza Kasaei (University of Edinburgh), Mohsen Khadem (University of Edinburgh)
Robotic IntelligenceReinforcement LearningOrdinary Differential Equation
🎯 What it does: This paper proposes SoftManiSim, a fast simulation framework for real-time simulation of multi-segment continuum manipulators, capable of co-operating with rigid robots;
SoloParkour: Constrained Reinforcement Learning for Visual Locomotion from Privileged Experience
Elliot Chane-Sane (Universit e de Toulouse), Nicolas Mansard (Universit e de Toulouse)
Robotic IntelligenceConvolutional Neural NetworkRecurrent Neural NetworkReinforcement LearningImage
🎯 What it does: Using constrained reinforcement learning and prior experience, the Solo-12 quadruped robot is trained to end-to-end learn visually driven flexible motion policies from depth pixels, achieving 'parkour' skills such as running, climbing slopes, jumping over obstacles, and crawling.
Solving Offline Reinforcement Learning with Decision Tree Regression
Prajwal Koirala (Iowa State University), Cody Fleming (Iowa State University)
Reinforcement LearningTabularBenchmark
🎯 What it does: This paper reformulates offline reinforcement learning tasks as regression problems, achieving offline RL using decision trees.
SonicSense: Object Perception from In-Hand Acoustic Vibration
Jiaxun Liu (Duke University), Boyuan Chen (Duke University)
ClassificationObject DetectionRetrievalConvolutional Neural NetworkPoint CloudAudio
🎯 What it does: Studied a multi-fingered robotic hand based on acoustic vibrations in the hand, achieving container inventory detection, material classification, 3D shape reconstruction, and object re-identification.
Sparse Diffusion Policy: A Sparse, Reusable, and Flexible Policy for Robot Learning
Yixiao Wang (UC Berkeley), Masayoshi Tomizuka (UC Berkeley)
Robotic IntelligenceTransformerMixture of ExpertsDiffusion modelPoint Cloud
🎯 What it does: Propose Sparse Diffusion Policy (SDP), integrating Mixture of Experts (MoE) into Transformer-based diffusion policy to achieve a sparse, reusable, and flexible robotic learning framework that supports multi-task learning, continual learning, and task transfer.
Sparsh: Self-supervised touch representations for vision-based tactile sensing
Carolina Higuera (FAIR at Meta), Mustafa Mukadam (FAIR at Meta)
Representation LearningRobotic IntelligenceTransformerSupervised Fine-TuningAuto EncoderContrastive LearningImageBenchmark
🎯 What it does: This study proposes Sparsh, a general tactile representation based on self-supervised learning for visual tactile sensors; simultaneously, the TacBench benchmark is constructed, covering six tactile-related tasks.
SPIRE: Synergistic Planning, Imitation, and Reinforcement Learning for Long-Horizon Manipulation
Zihan Zhou (NVIDIA), Ajay Mandlekar (NVIDIA)
Robotic IntelligenceReinforcement LearningText
🎯 What it does: This paper proposes the SPIRE framework, decomposing tasks into TAMP control and learning subtasks, achieving long-term spatiotemporal manipulation through a combination of behavioral cloning and reinforcement learning.
Splat-MOVER: Multi-Stage, Open-Vocabulary Robotic Manipulation via Editable Gaussian Splatting
Olaolu Shorinwa (Stanford University), Mac Schwager (Stanford University)
Robotic IntelligenceVision-Language-Action ModelGaussian SplattingImageTextPoint Cloud
🎯 What it does: Developed the Splat-MOVER stack, utilizing editable Gaussian Splatting to achieve multi-stage, open-vocabulary robotic grasping and manipulation.
Steering Your Generalists: Improving Robotic Foundation Models via Value Guidance
Mitsuhiko Nakamoto (University of California Berkeley), Sergey Levine (University of California Berkeley)
Robotic IntelligenceConvolutional Neural NetworkReinforcement LearningVision-Language-Action ModelMultimodality
🎯 What it does: Propose a method that leverages an offline RL-learned value function during deployment to reorder actions of a pre-trained general robot policy, enhancing its accuracy and robustness.
Structured Bayesian Meta-Learning for Data-Efficient Visual-Tactile Model Estimation
Shaoxiong Yao (University of Illinois at Urbana-Champaign), Kris Hauser (University of Illinois at Urbana-Champaign)
Robotic IntelligenceMeta LearningMultimodality
🎯 What it does: This paper proposes a structured Bayesian meta-learning method (SBML) for rapidly estimating visual-tactile models of deformable objects with limited tactile data.
Surgical Robot Transformer (SRT): Imitation Learning for Surgical Tasks
Ji Woong Kim (Johns Hopkins University), Axel Krieger (Johns Hopkins University)
Robotic IntelligenceTransformerDiffusion modelImageBiomedical Data
🎯 What it does: On the da Vinci Surgical Robot (dVRK), imitation learning was used to achieve three basic surgical tasks: tissue lifting, needle pickup and handoff, and tying; and executable control strategies were successfully trained under conditions of using only approximate forward kinematics data (without additional calibration).
T$^2$SQNet: A Recognition Model for Manipulating Partially Observed Transparent Tableware Objects
Young Hun Kim (Seoul National University), Frank C. Park
RecognitionObject DetectionData SynthesisConvolutional Neural NetworkTransformerImage
🎯 What it does: Propose a transparent tableware recognition and manipulation model T-SQNet and the corresponding TablewareNet dataset, generating low-dimensional instance-level 3D geometric representations for each object using extended deformable super tetrahedrons, achieving identification and grasping from partial perspective RGB images
Tag Map: A Text-Based Map for Spatial Reasoning and Navigation with Large Language Models
Mike Zhang (ETH Zurich), Marco Hutter (ETH Zurich)
Representation LearningRobotic IntelligenceLarge Language ModelImageText
🎯 What it does: Propose a text-based tag map to store a large number of semantic tags and utilize LLM for navigation planning based on spatial reasoning.
TaMMa: Target-driven Multi-subscene Mobile Manipulation
Jiawei Hou (Fudan University), Yanwei Fu (Fudan University)
RestorationSegmentationGenerationPose EstimationDepth EstimationOptimizationComputational EfficiencyRepresentation LearningRobotic IntelligenceDiffusion modelGaussian SplattingImagePoint Cloud
🎯 What it does: Proposes a framework named TaMMa, which initializes a 3D Gaussian distribution using sparse RGB-D scans and combines diffusion models to complete depth and repair scenes, enabling goal-driven precise mobility and manipulation of mobile bases and robotic arms across multiple sub-scenes.
Task Success Prediction for Open-Vocabulary Manipulation Based on Multi-Level Aligned Representations
Miyu Goko (Keio University), Komei Sugiura (Keio University)
Robotic IntelligenceTransformerVision Language ModelContrastive LearningImageTextMultimodality
🎯 What it does: Predict whether a robot successfully completes a manipulation task given open-source verbal instructions and images from front and rear perspectives.
Task-Oriented Hierarchical Object Decomposition for Visuomotor Control
Jianing Qian (University of Pennsylvania), Dinesh Jayaraman (University of Pennsylvania)
Robotic IntelligenceTransformerVision Language ModelVision-Language-Action ModelContrastive LearningImageTextMultimodality
🎯 What it does: This paper proposes a task-based hierarchical object decomposition visual representation (HODOR), which automatically assembles object and part embeddings from pre-trained models (e.g., SAM, DINO-v2) to provide efficient, scalable, and task-related features for robot vision control.
Teaching Robots with Show and Tell: Using Foundation Models to Synthesize Robot Policies from Language and Visual Demonstration
Michael Murray (University of Washington), Maya Cakmak (University of Washington)
Robotic IntelligenceTransformerLarge Language ModelVision Language ModelVision-Language-Action ModelDiffusion modelVideoTextMultimodality
🎯 What it does: Proposes SHOWTELL, a neural-symbolic framework based on language and visual demonstrations, enabling robots to perform multiple manipulation tasks with a single demonstration.
Text2Interaction: Establishing Safe and Preferable Human-Robot Interaction
Jakob Thumm (Technical University of Munich), Matthias Althoff (Technical University of Munich)
Robotic IntelligenceLarge Language ModelText
🎯 What it does: Propose the Text2Interaction framework, achieving zero-shot integration of user preferences across task, motion, and control levels based on a single language instruction. The framework generates task plans, motion preference functions (Python code), and safety controller parameters using a large language model (LLM), and employs probability maximization (feasibility × user satisfaction) for planning.
Theia: Distilling Diverse Vision Foundation Models for Robot Learning
Jinghuan Shang (AI Institute), Laura Herlant (AI Institute)
Knowledge DistillationRobotic IntelligenceConvolutional Neural NetworkTransformerImageBenchmark
🎯 What it does: Proposes Theia — a visual representation model that integrates multiple vision foundation models (such as CLIP, DINOv2, ViT, SAM, Depth-Anything) into a smaller, more computationally efficient model using knowledge distillation, and applies this model to robot vision control.
ThinkGrasp: A Vision-Language System for Strategic Part Grasping in Clutter
Yaoyao Qian (Northeastern University), Robert Platt (Northeastern University)
Robotic IntelligenceTransformerLarge Language ModelVision-Language-Action ModelImageTextPoint CloudChain-of-Thought
🎯 What it does: This paper proposes ThinkGrasp, a pluggable vision-language system that leverages GPT-4o for context reasoning, combining language-guided segmentation with a k×k grid strategy to progressively remove obstacles and grasp target objects in high-density cluttered environments.
TidyBot++: An Open-Source Holonomic Mobile Manipulator for Robot Learning
Jimmy Wu (Princeton University), Jeannette Bohg (Stanford University)
Robotic IntelligenceDiffusion model
🎯 What it does: Proposed and implemented TidyBot++ — an open-source, low-cost, modular omnidirectional mobile manipulator, along with an intuitive mobile WebXR-based remote control interface for data collection and training of mobility manipulation tasks.
TieBot: Learning to Knot a Tie from Visual Demonstration through a Real-to-Sim-to-Real Approach
Weikun Peng (National University of Singapore), Lin Shao (Shanghai Jiao Tong University)
Domain AdaptationRobotic IntelligenceReinforcement LearningDiffusion modelVideoPoint CloudMesh
🎯 What it does: Developed TieBot, a real-to-simulation-to-real learning framework based on visual demonstrations, enabling dual-arm robots to complete tie-tying tasks by estimating the tie's mesh and learning through subgoal-driven strategies.
TLDR: Unsupervised Goal-Conditioned RL via Temporal Distance-Aware Representations
Junik Bae (Yonsei University), Youngwoon Lee (Yonsei University)
Reinforcement Learning
🎯 What it does: Propose the TLDR algorithm, which utilizes Temporal Distance-Aware Representations to achieve unsupervised goal-conditioned reinforcement learning, improving exploration strategies and goal-achievement strategies, and significantly enhancing state coverage in various simulation environments.
Tokenize the World into Object-level Knowledge to Address Long-tail Events in Autonomous Driving
Thomas Tian, Marco Pavone (NVIDIA)
Autonomous DrivingTransformerLarge Language ModelTextMultimodalityChain-of-Thought
🎯 What it does: Propose TOKEN, a multimodal large language model that achieves autonomous driving planning by transforming the world into object-level knowledge, particularly for long-tail events.
TOP-Nav: Legged Navigation Integrating Terrain, Obstacle and Proprioception Estimation
Junli Ren (Tsinghua University), Guijin Wang (Tsinghua University)
Robotic IntelligenceReinforcement LearningImage
🎯 What it does: This paper proposes the TOP-Nav framework, which integrates terrain awareness, obstacle avoidance, and proprioception feedback loops to enable robot navigation in open-world environments.
Toward General Object-level Mapping from Sparse Views with 3D Diffusion Priors
Ziwei Liao (University of Toronto), Steven L. Waslander (University of Toronto)
GenerationPose EstimationDiffusion modelNeural Radiance FieldImagePoint CloudMesh
🎯 What it does: Propose a multi-class sparse view object-level mapping system called GOM based on a pre-trained 3D diffusion model, which can simultaneously recover the shape and pose of multiple instance objects from a few RGB-D views.
Towards Open-World Grasping with Large Vision-Language Models
Georgios Tziafas (University of Groningen), Hamidreza Kasaei (University of Groningen)
SegmentationRobotic IntelligenceConvolutional Neural NetworkTransformerLarge Language ModelPrompt EngineeringVision Language ModelMultimodalityChain-of-Thought
🎯 What it does: Propose OWG, integrating GPT-4v, visual marker prompts, SAM segmentation, and GR-ConvNet grasping synthesis to achieve an end-to-end pipeline from open-ended language instructions to zero-shot closed-loop grasping.
Trajectory Improvement and Reward Learning from Comparative Language Feedback
Zhaojing Yang (University of Southern California), Erdem Biyik
Robotic IntelligenceReinforcement Learning from Human FeedbackTransformerLarge Language ModelReinforcement LearningTextSequential
🎯 What it does: Leverage comparative language feedback to learn robot trajectories and reward functions, constructing a shared latent space and iteratively refining trajectories.
Transferable Tactile Transformers for Representation Learning Across Diverse Sensors and Tasks
Jialiang Zhao (MIT), Edward Adelson (MIT)
ClassificationPose EstimationRepresentation LearningTransformerSupervised Fine-TuningAuto EncoderImageMultimodality
🎯 What it does: Proposed the Transferable Tactile Transformers (T3) framework and the FoTa dataset to learn shared tactile representations across sensors and tasks;
TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correction
Yunfan Jiang (Stanford University), Li Fei-Fei (Stanford University)
Robotic IntelligenceReinforcement LearningPoint Cloud
🎯 What it does: A baseline control policy trained in a simulated environment (baseline policy) is combined with residual strategies generated from human real-time corrections on a physical robot, forming a gated residual strategy, ultimately achieving high-precision, contact-rich robot manipulation tasks with simulation-to-real (sim-to-real) transfer.
Trust the PRoC3S: Solving Long-Horizon Robotics Problems with LLMs and Constraint Satisfaction
Aidan Curtis (MIT), Leslie Pack Kaelbling (MIT)
OptimizationRobotic IntelligenceReinforcement Learning from Human FeedbackLarge Language Model
🎯 What it does: Generate parameterized operation code using large language models (LLMs), validate and iterate through simulators, ultimately obtaining long-term robotic operation plans that satisfy constraints.
Twisting Lids Off with Two Hands
Toru Lin (University of California, Berkeley), Jitendra Malik (University of California, Berkeley)
Robotic IntelligenceReinforcement LearningImage
🎯 What it does: This paper trains two multi-finger humanoid robot hands via deep reinforcement learning in simulation to complete rotation/removal of bottle caps tasks, and transfers the learned policy to real robots without any fine-tuning, achieving stable grasping and continuous rotation on bottles with different shapes, sizes, and materials.
UBSoft: A Simulation Platform for Robotic Skill Learning in Unbounded Soft Environments
Chunru Lin (University Of Massachusetts Amherst), Chuang Gan (University Of Massachusetts Amherst)
OptimizationRobotic IntelligenceReinforcement LearningPoint CloudBenchmarkPhysics Related
🎯 What it does: Developed a differentiable physics simulation platform called UBSOFT, specifically designed for training robot manipulation and locomotion skills in large-scale, unbounded environments containing multiple soft materials.
UMI-on-Legs: Making Manipulation Policies Mobile with Manipulation-Centric Whole-body Controllers
Huy Ha (Stanford University), Shuran Song (Stanford University)
Domain AdaptationRobotic IntelligenceReinforcement LearningDiffusion modelImageSequential
🎯 What it does: This paper proposes the UMI-on-Legs framework, which combines real demonstration data collected by humans using handheld grippers with a whole-body controller trained in simulation, achieving autonomous manipulation on quadruped robots.
Uncertainty-Aware Decision Transformer for Stochastic Driving Environments
Zenan Li (Tsinghua), Hang Zhao (Shanghai Qi Zhi)
Autonomous DrivingTransformerReinforcement LearningMixture of ExpertsSequential
🎯 What it does: Proposed a novel offline reinforcement learning decision Transformer called UNREST, which avoids the over-optimism problem in traditional decision Transformers by estimating environmental uncertainty and training with truncated returns, achieving dynamic uncertainty-guided planning in stochastic autonomous driving environments.
Unpacking Failure Modes of Generative Policies: Runtime Monitoring of Consistency and Progress
Christopher Agia (Stanford University), Jeannette Bohg (Stanford University)
Anomaly DetectionRobotic IntelligenceDiffusion modelVideoMultimodality
🎯 What it does: Propose Sentinel, a runtime monitoring framework for generative robot policies, divided into two categories of failure detection: STAC based on temporal consistency detects random loss-of-control behaviors; task progress monitoring using Vision-Language Models (VLM) detects failure due to lack of progress.
Velociraptor: Leveraging Visual Foundation Models for Label-Free, Risk-Aware Off-Road Navigation
Samuel Triest (Carnegie Mellon University), Sebastian Scherer (Carnegie Mellon University)
Autonomous DrivingConvolutional Neural NetworkReinforcement LearningVision Language ModelContrastive LearningSimultaneous Localization and MappingImagePoint Cloud
🎯 What it does: Propose the Velociraptor system to achieve zero manual annotation, risk-aware off-road traversability analysis. It generates cost, speed, and uncertainty maps in BEV (Bird's Eye View) space by combining visual foundation models (VFM) with geometric mapping, and employs self-supervised learning for control.
Verification of Neural Control Barrier Functions with Symbolic Derivative Bounds Propagation
Hanjiang Hu (Carnegie Mellon University), Changliu Liu (Carnegie Mellon University)
Safty and PrivacyRobotic IntelligenceTabular
🎯 What it does: This paper proposes a symbolic derivative bound propagation method to efficiently verify the forward invariance of parameterized control barrier functions (neural CBF) in ReLU networks, addressing the issue of traditional interval arithmetic being overly conservative.
View-Invariant Policy Learning via Zero-Shot Novel View Synthesis
Stephen Tian (Stanford University), Jiajun Wu (Stanford University)
Data SynthesisRobotic IntelligenceDiffusion modelImage
🎯 What it does: This paper proposes a data augmentation method called VISTA, which utilizes the zero-shot single-image novel view synthesis model (ZeroNVS) to generate multi-view images during training, thereby enabling visual imitation learning strategies to be robust against camera perspective variations.
ViPER: Visibility-based Pursuit-Evasion via Reinforcement Learning
Yizhuo Wang (National University Of Singapore), Guillaume Adrien Sartoretti
Graph Neural NetworkReinforcement LearningGraph
🎯 What it does: Propose ViPER, a multi-agent reinforcement learning method based on graph attention networks, for visualizing pursuit-evasion tasks, learning distributed collaborative strategies to quickly clear areas and detect potential escapers at arbitrary speeds in unknown/known environments.
VIRL: Self-Supervised Visual Graph Inverse Reinforcement Learning
Lei Huang (Columbia University), Zhengbo Zou (Columbia University)
Robotic IntelligenceGraph Neural NetworkTransformerReinforcement LearningAuto EncoderContrastive LearningVideoBenchmark
🎯 What it does: Propose the VIRL method, which leverages a self-supervised trained visual graph encoder to learn a reward function from unlabeled videos, enabling cross-model and task extrapolation learning;
Visual Manipulation with Legs
Xialin He (University of Illinois Urbana Champaign), Xiaolong Wang (Uc San Diego)
Robotic IntelligenceConvolutional Neural NetworkReinforcement LearningPoint Cloud
🎯 What it does: This paper proposes a vision-based leg manipulation system, enabling quadruped robots to perform non-grasping operations (e.g., pushing, flipping, remote movement) through leg-object interactions.
Visual Whole-Body Control for Legged Loco-Manipulation
Minghuan Liu (University Of California San Diego), Xiaolong Wang (University Of California San Diego)
Domain AdaptationRepresentation LearningRobotic IntelligenceConvolutional Neural NetworkReinforcement LearningImagePoint Cloud
🎯 What it does: This paper proposes a vision-based whole-body control framework (VBC), enabling a quadruped robot + manipulator arm platform to autonomously complete object-picking tasks with different heights, positions, and poses.
VLM-Grounder: A VLM Agent for Zero-Shot 3D Visual Grounding
Runsen Xu (Chinese University of Hong Kong), Dahua Lin (Chinese University of Hong Kong)
Object DetectionVision Language ModelImageText
🎯 What it does: Propose VLM-Grounder, a zero-shot 3D visual localization agent based on vision-language models, which accomplishes 3D object localization through 2D image sequences and projects it into 3D bounding boxes.
Vocal Sandbox: Continual Learning and Adaptation for Situated Human-Robot Collaboration
Jennifer Grannen (Stanford University), Dorsa Sadigh (Stanford University)
Robotic IntelligenceTransformerLarge Language ModelImageTextMultimodality
🎯 What it does: This work proposes and implements the Vocal Sandbox framework, which supports continuous learning and real-time adaptation in practical scenarios through multi-modal teaching methods such as voice, object key points, and demonstrations for human-robot collaboration;
VoxAct-B: Voxel-Based Acting and Stabilizing Policy for Bimanual Manipulation
I-Chun Arthur Liu (University of Southern California), Gaurav S. Sukhatme (University of Southern California)
Object DetectionSegmentationRobotic IntelligenceTransformerReinforcement LearningVision Language ModelVision-Language-Action ModelImageMultimodality
🎯 What it does: Proposed a dual-arm collaboration strategy called VoxAct-B based on voxels and language, which uses VLM to crop voxel grids and learn action and stabilization actions.
What Makes Pre-Trained Visual Representations Successful for Robust Manipulation?
Kaylee Burns (Stanford University), Karol Hausman (Stanford University)
Domain AdaptationRobotic IntelligenceTransformerSupervised Fine-TuningImageVideo
🎯 What it does: Evaluate the robustness of 15 pre-trained visual models in robotic manipulation tasks, particularly their adaptability to visual distribution drifts in lighting, texture, and distractors.
What Matters in Range View 3D Object Detection
Benjamin Wilson (Georgia Institute of Technology), James Hays (Georgia Institute of Technology)
Object DetectionAutonomous DrivingConvolutional Neural NetworkPoint Cloud
🎯 What it does: This paper studies and optimizes a radar-based range view 3D object detection model, focusing on evaluating key design choices and proposing simple and effective improvement strategies.
WoCoCo: Learning Whole-Body Humanoid Control with Sequential Contacts
Chong Zhang (ETH Zurich), Guanya Shi (Carnegie Mellon University)
Robotic IntelligenceReinforcement Learning
🎯 What it does: Propose the WoCoCo framework, which utilizes contact phase decomposition to achieve reinforcement learning control for whole-body robots in sequential contact tasks.