Known Issues

cuRobo

  1. Collision-free planning with the world represented by camera perception is an open research problem. cuRobo provides integration with nvblox for querying closest points between the robot and a voxelized map. We found that this works well in sparse obstacle environments. But as we increased the density of obstacles, the occlusions in perception can cause many failures.

Python

  1. Once a cuda graph is generated, we cannot change the dimensions of any of the tensors (e.g., dof, timesteps, number of seeds). While cuRobo library is designed to regenerate the cuda graph if a change in dimension is detected, pytorch crashes when CUDAGraph.reset() is called.

nvblox

  1. Compiling nvblox as redistributable with cxx11_ABI=1 does not work as cmake links to global glog library instead of the local one. Hence compile nvblox as standard when using with cxx11_ABI=1 as apt install glog is also compiled with this flag.

  2. Voxel sizes less then 1 cm (<0.01) can hit GPU memory limits very quickly.

  3. Reinitializing nvblox inside a python script can cause CUDA errors. Instead of reinitializing nvblox mapper instance, clear the existing map.

Isaac Sim

  1. Headless mode works reliably with Omniverse Streaming Client by setting --headless_mode=native for examples in Using with Isaac Sim. WebSocket and WebRTC can have issues.

  2. When using development docker, webrtc headless mode does not work. Use --headless_mode=native.

Warp

  1. Older versions of warp-lang<1.2.1 do not use kernel key (names) for creating the hash for a module. This prevents us from creating runtime kernels that depend on a robot’s parameters such as dof. The workaround is to create a global warp constant with value of dof. This will make the hash unique for each robot instance. This issue is fixed in warp-lang>=1.2.1 (https://github.com/NVIDIA/warp/commit/21cba5a300a81463aebcb94a15dd09c0d4937d30).