A long horizon manipulation task leveraging cuRobo is implemented as an example in this library. The block stacking task starts with 9 cubes in front of the robot, with the goal being to stack all the cubes into a tower as seen below.
The key componenet in this task is to design a Pick Place Sequence as shown in the graph below, which consists of moving the robot to a grasp pose, followed by closing the gripper, attaching the grasped object to cuRobo’s robot model, moving the object to a location on top of another cube, opening the gripper, and deleting the object from cuRobo’s robot model. Once this sequence is designed and implemented, the block stacking task can be completed by calling this sequence on all the objects one by one. After each placement, cuRobo updates its world reprensentation by reading the USD stage from Isaac Sim.
Attaching Object to Robot
cuRobo implements an attach object feature to enable reasoning about collisions between a grasped object and the world while the robot is moving. Since cuRobo represents the robot with spheres, cuRobo automatically computes bounding spheres (see Geometry Approximation to Spheres) for an object to attach to the robot. When attaching the object to the robot, the object is also deleted from the world. Similarly when the object is detached, the spheres are disabled and the object is enabled in the world model. This sequence of actions is shown in the graph below.
Detach object currently deletes all spheres attached to a link.