These 32 CoRL 2024 papers come with a code repository. Each shows an AI one-line summary below — get the verified repo link + the full 6-part summary (innovation, method, data, results, limitations) and search every CoRL 2024 paper, free trial on arXivSub.
A Planar-Symmetric SO(3) Representation for Learning Grasp Detection
Tianyi Ko (Woven by Toyota), Koichi Nishiwaki (Woven by Toyota)
🎯 What it does: Designed and implemented a planar symmetric parallel gripper SO(3) rotation representation using 2D Bingham distribution, integrated into a 3D convolutional grasping detection network, addressing issues of rotational discontinuity and multimodal inconsistency in traditional regression.
A3VLM: Actionable Articulation-Aware Vision Language Model
Siyuan Huang (Shanghai Jiao Tong University), Hongsheng Li (Chinese University of Hong Kong)
CodeObject DetectionGenerationPose EstimationRobotic IntelligenceTransformerVision Language ModelVision-Language-Action ModelDiffusion modelContrastive LearningImageText
🎯 What it does: Designed and trained an object-centric, operable, and joint-aware model called A3VLM based on a vision-language model, which identifies movable parts and their joint structures using a single RGB image, and outputs operable representations directly mappable to robotic execution;
An Open-Source Soft Robotic Platform for Autonomous Aerial Manipulation in the Wild
Erik Bauer (ETH Zurich), Robert K. Katzschmann (ETH Zurich)
CodeRobotic IntelligenceSimultaneous Localization and MappingImageVideoPoint Cloud
🎯 What it does: Built and validated a fully autonomous soft aerial manipulator platform based on onboard perception, capable of autonomously locating, searching, and grasping various unknown objects in indoor and outdoor environments.
🎯 What it does: This paper proposes and implements BiDex, a low-cost, low-latency, portable dual-arm bimanual teleoperation system for performing high-degree-of-freedom bimanual grasping and manipulation in desktop and mobile environments, and collects expert demonstration data through the system to train a behavior cloning policy.
🎯 What it does: Proposed a DDFH active learning framework based on distribution differences and feature heterogeneity for efficient annotation in LiDAR 3D object detection.
🎯 What it does: Developed a pixel-level dynamic hybrid visual augmentation method called RoboSaGA based on task-related saliency, enhancing the robustness of visual behavior cloning under visual domain shift scenarios (lighting, shadows, occlusions, background).
🎯 What it does: Propose a SE(3)-equivariant 6-DoF grasping pose generation model called EquiGraspFlow based on Continuous Normalizing Flows (CNF), which can directly generate diverse grasping poses that conform to object rotations and translations from point clouds.
🎯 What it does: Propose a trajectory prediction framework based on evolutionary self-contrastive inverse reinforcement learning (ESCIRL), which can learn a general and robust reward function under the condition of only a few mixed scenario examples;
🎯 What it does: Propose the FetchBench simulation benchmark, construct diverse procedural grasping scenarios, and evaluate perception-planning-execution pipelines and end-to-end imitation learning methods on this benchmark.
🎯 What it does: This paper proposes a novel robotic behavior cloning framework called GENIMA, which maps robot joint actions to the image space, allowing a stable diffusion model to directly generate target joint positions;
Handling Long-Term Safety and Uncertainty in Safe Reinforcement Learning
Jonas Günster (TU Darmstadt), Davide Tateo (TU Darmstadt)
CodeSafty and PrivacyReinforcement Learning
🎯 What it does: By integrating ATACOM with distributed reinforcement learning, long-term safety constraints are learned, enabling risk-aware safe reinforcement learning.
JointMotion: Joint Self-Supervision for Joint Motion Prediction
Royden Wagner (Karlsruhe Institute of Technology), Carlos Fernandez
CodeAutonomous DrivingTransformerSupervised Fine-TuningAuto EncoderMultimodalityTime Series
🎯 What it does: In the joint motion prediction task for autonomous vehicles, the self-supervised pre-training framework JointMotion is proposed, which pre-trains the model using scene-level and instance-level objectives.
KOROL: Learning Visualizable Object Feature with Koopman Operator Rollout for Manipulation
Hongyi Chen (Carnegie Mellon University), Jeffrey Ichnowski (Georgia Institute of Technology)
CodeExplainability and InterpretabilityRobotic IntelligenceConvolutional Neural NetworkWorld ModelImage
🎯 What it does: Designed an imitation learning method called KOROL that leverages the Koopman operator for visual interpretable object feature learning, enabling robot manipulation without requiring real object state labels.
Large Scale Mapping of Indoor Magnetic Field by Local and Sparse Gaussian Processes
Iad ABDUL-RAOUF (Universite Paris-Saclay), Alexis Paljic (Centre De Robotique)
CodeMixture of ExpertsPhysics Related
🎯 What it does: Propose a magnetic field mapping framework based on local sparse Gaussian process (Gradient-DTC) and local Bayesian committee machine (LBCM), which can efficiently and smoothly construct magnetic field maps and estimate uncertainty in large-scale indoor environments.
Learning Differentiable Tensegrity Dynamics using Graph Neural Networks
Nelson Chen (Rutgers University), Mridul Aanjaneya (Rutgers University)
CodeExplainability and InterpretabilityComputational EfficiencyRobotic IntelligenceGraph Neural NetworkImageGraphTime SeriesOrdinary Differential Equation
🎯 What it does: Use graph neural networks to learn the dynamic behavior of tension structure robots, and achieve simulation through a hybrid analytical and learning model.
🎯 What it does: Proposed a new self-supervised learning framework for learning control barrier functions (CBF) under complex safety constraints and limited driving conditions to ensure the safety and performance of nonlinear control systems.
MaIL: Improving Imitation Learning with Selective State Space Models
Xiaogang Jia (Karlsruhe Institute of Technology), Gerhard Neumann (Karlsruhe Institute of Technology)
CodeRobotic IntelligenceReinforcement LearningVision Language ModelDiffusion modelImageBenchmark
🎯 What it does: This study proposes the MaIL architecture, which achieves visual imitation learning using the Mamba state space model, replacing Transformer.
Mobile ALOHA: Learning Bimanual Mobile Manipulation using Low-Cost Whole-Body Teleoperation
Zipeng Fu (Stanford University), Chelsea Finn (Stanford University)
CodeRobotic IntelligenceDiffusion modelImage
🎯 What it does: Designed and implemented a low-cost, portable dual-arm mobile robot called Mobile ALOHA, collecting demonstration data through whole-body teleoperation; utilized this data along with static ALOHA data for co-training, achieving successful execution of seven complex mobile dual-arm tasks.
🎯 What it does: This study integrates seven multimodal human motion datasets to construct a unified data framework and proposes the Multi-Transmotion pre-trained Transformer model, which utilizes multiple masking strategies for cross-modal prediction and is fine-tuned under multi-task scenarios.
OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning
Tairan He (Carnegie Mellon University), Guanya Shi (Shanghai Jiao Tong University)
CodeRobotic IntelligenceReinforcement Learning from Human FeedbackReinforcement LearningVision-Language-Action ModelDiffusion modelTextMultimodalitySequential
🎯 What it does: Propose the OmniH2O system, achieving full-scale humanoid robot whole-body control and autonomous learning, supporting VR, RGB, language, and other interfaces.
🎯 What it does: This study proposes and implements OpenTeleVision, an immersive teleoperation system based on VR devices, active head cameras, and stereo vision. It uses human demonstration data collected through this system to train imitation learning strategies, successfully achieving fine-grained long-term tasks (such as can classification, insertion, folding, and unloading) on two humanoid robots.
🎯 What it does: Trained a POLIFORMER agent for indoor navigation using only RGB input and a full Transformer architecture, achieving direct real-world deployment without fine-tuning after large-scale simulation training.
🎯 What it does: This work proposes the Re-Mix method, which automatically determines the mixing weights of robot datasets using distributed robust optimization, thereby enhancing model generalization in large-scale imitation learning pre-training.
🎯 What it does: This study proposes an scalable multi-agent control method to satisfy signal temporal logic (STL) specifications, combining graph neural networks (GNN), neural ODE planner, and a safety controller based on graph control barrier function (GCBF+), achieving end-to-end differentiable learning;
Sim-to-Real Transfer via 3D Feature Fields for Vision-and-Language Navigation
Zihan Wang (Institute of Computing Technology Chinese Academy of Sciences), Shuqiang Jiang (Institute of Computing Technology Chinese Academy of Sciences)
🎯 What it does: Designed semantic traversability maps and 3D feature fields based on monocular RGB-D cameras, enabling panoramic perception for monocular robots and achieving VLN model transfer from simulation to real environments.
🎯 What it does: This paper proposes SoftManiSim, a fast simulation framework for real-time simulation of multi-segment continuum manipulators, capable of co-operating with rigid robots;
🎯 What it does: Propose a transparent tableware recognition and manipulation model T-SQNet and the corresponding TablewareNet dataset, generating low-dimensional instance-level 3D geometric representations for each object using extended deformable super tetrahedrons, achieving identification and grasping from partial perspective RGB images
🎯 What it does: Propose a multi-class sparse view object-level mapping system called GOM based on a pre-trained 3D diffusion model, which can simultaneously recover the shape and pose of multiple instance objects from a few RGB-D views.
Uncertainty-Aware Decision Transformer for Stochastic Driving Environments
Zenan Li (Tsinghua), Hang Zhao (Shanghai Qi Zhi)
CodeAutonomous DrivingTransformerReinforcement LearningMixture of ExpertsSequential
🎯 What it does: Proposed a novel offline reinforcement learning decision Transformer called UNREST, which avoids the over-optimism problem in traditional decision Transformers by estimating environmental uncertainty and training with truncated returns, achieving dynamic uncertainty-guided planning in stochastic autonomous driving environments.
🎯 What it does: Propose ViPER, a multi-agent reinforcement learning method based on graph attention networks, for visualizing pursuit-evasion tasks, learning distributed collaborative strategies to quickly clear areas and detect potential escapers at arbitrary speeds in unknown/known environments.
VLM-Grounder: A VLM Agent for Zero-Shot 3D Visual Grounding
Runsen Xu (Chinese University of Hong Kong), Dahua Lin (Chinese University of Hong Kong)
CodeObject DetectionVision Language ModelImageText
🎯 What it does: Propose VLM-Grounder, a zero-shot 3D visual localization agent based on vision-language models, which accomplishes 3D object localization through 2D image sequences and projects it into 3D bounding boxes.
🎯 What it does: This paper studies and optimizes a radar-based range view 3D object detection model, focusing on evaluating key design choices and proposing simple and effective improvement strategies.