Portfolio

Fetch-and-Carry Task with Cali

Cali : An Autonomous Mobile Manipulator for Fetch-and-Carry Tasks in GPS-Denied Environments

Overview

Cali is an open-source robotic platform designed for research and education in mobile manipulation. It leverages off-the-shelf components and integrates open-source software to facilitate high-level task development for mobile manipulators.

Cali is composed of a customed 4-wheeled skid steer drive mobile platform and a commercial manipulator system: Scorbot-ER III:

Goal

Develop a modular, extensible system that simplifies mobile manipulator research and enables autonomous Fetch-and-Carry missions.

Key Features

  • Modular integration of navigation, manipulation, and perception pipelines.
  • ROS Melodic and Ubuntu 18.04-based platform.
  • Hardware includes a skid-steer drive base and Sorbot-ER III robotic arm.
  • Real-time operation with Nvidia Jetson TX2.

Technical Approach

Hardware

  • Base: Custom four-wheeled skid-steer base.
  • Manipulator: Sorbot-ER III robotic arm with 5 DOF + gripper.
  • Sensors: LiDAR, depth camera, IMU, encoders.
  • Power: 12V 50Ah lithium battery.

Software

  • Framework: ROS Melodic.
  • Simulator: Gazebo.
  • Libraries: OpenCV, Moveit

Subsystems

  • Mapping, navigation, manipulation, and perception pipelines.
  • Manipulation/Perception pipeline.

Implementation Details

Mapping

mapping

AN_v2

Manipulation

grasp_coke_can

Perception

surface_detection

Manipulation/Perception Pipeline

manipulation_perception_pipeline

Results

Simulated Environment

As we have seen Cali has successfully performed the high-level task:

  • Mapping
  • Autonomous Navigation
  • Manipulation
  • Perception
  • Manipulation/Perception pipeline

As a result it was able to successfully perform the Fetch-and-carry mission in an indoor environment.

mission

Real Robot

real_robot_v2

  • Successfully performed autonomous navigation and perception pipelines for the real robot.
  • Due to time constraints, the manipulation pipeline was not integrated as communication with the arm manipulator could not be established due to some time constraints with my university.
  • This limitation will be addressed in the Next Steps section, immediately following this one.

Here is a GIF demonstrating the real robot’s autonomous navigation:

AN_real

Now let’s observe the perception results on the real robot:

perception_real

Next Steps

There is still work yet to be done to realize the real-world application of the Fetch-and-Carry task, specifically in controlling the robot arm using the FollowJointTrajectory and GripperCommand action interfaces. Future efforts will investigate hardware interfaces or alternatives to rosserial for communication. Once communication with these action interfaces is established, Cali will be able to achieve its mission in the real world.

Here are some suggestions for potential next steps:

  • Establish communication with manipulation interfaces to enable real-world task execution.
  • Develop a hardware interface to facilitate communication with the manipulator.
  • Implement failure recovery procedures to address unexpected issues, such as unsuccessful task completion (e.g., grasping failures).

GitHub Repository

Explore the Project on GitHub