Seeking Guidance on Batch Motion Planning in Curobo

[acebulla]

Hi,

I'm working on a project involving batch motion planning with curobo and I've hit a roadblock. My environment doesn't include Isaac Sim, so I can not execute the example provided in curobo/examples/isaac_sim/batch_motion_gen_reacher.py. I'm struggling to understand how the planner distinguishes between different robots.

The MotionGenConfig is given a single robot configuration and a list of world configurations:

robot_cfg = load_yaml(join_path(get_robot_configs_path(), args.robot))["robot_cfg"]
#....
world_file = ["collision_test.yml", "collision_thin_walls.yml"]
world_cfg_list = []
for i in range(n_envs):
    world_cfg = WorldConfig.from_dict(
        load_yaml(join_path(get_world_configs_path(), world_file[i]))
    ) 
    #....
    world_cfg_list.append(world_cfg)

motion_gen_config = MotionGenConfig.load_from_robot_config(
    robot_cfg,
    world_cfg_list,
    # ....
)

Further adding to my confusion, the stacked joint states all use the same joint names:

sim_js_names = robot_list[0].dof_names # Joint names from the first robot
sim_js = robot_list[0].get_joints_state()
full_js = JointState(
  #....
  joint_names=sim_js_names, # Same names
)
for i in range(1, len(robot_list)):
  sim_js = robot_list[i].get_joints_state()
  cu_js = JointState(
     #....
      joint_names=sim_js_names, # Same names
  )
  full_js = full_js.stack(cu_js)

Any insights or a point in the right direction would be greatly appreciated. Also, it would be incredibly helpful if someone could provide an example of batch motion planning in curobo that doesn't rely on Isaac Sim.

Thanks in advance for your help

[balakumar-s]

cuRobo's batch planning mode only supports the same robot in different environments. It does not support having different robots (e.g., Franka Panda in env-1, UR10 in env-2). You have to create separate instances of cuRobo's motion gen to run planning for different robots.

Here is an example of batch motion gen without isaac sim:

curobo/examples/motion_gen_example.py

Line 222 in c09d949

def demo_motion_gen_batch():

[acebulla]

Thanks for the quick response. I had overlooked the example you mentioned.

In my project, I'm training a policy for a single robot model to perform pick-and-place tasks in different settings. Each robot, while the same model, handles a different object, which is why multiple robots are required. Being able to run these tasks in parallel across various environments would help streamline our data collection process. This could be a useful feature for cuRobo, considering the ongoing developments in Reinforcement Learning.

I appreciate the resources and examples that you have made already available to the community.

Thanks again for your help.

[balakumar-s]

Good point! We only added batched environment support later in the pipeline. We have to think about how to design the attach object API so that it can map to different batch indices. This is technically feasible, just unsure on how best to implement it.

[frederiknolte]

Hi! Is there an update regarding attaching/detaching objects in batched environments? I would like to have multiple parallel environments in IsaacSim which would mean that objects are attached/detached at different time steps. Is there anything that supports attaching/detaching only to certain environments?

[Arpitrf]

Hi @balakumar-s, thanks a lot for the well-structured codebase! I wanted to ask if there are any updates regarding attaching/detaching objects in batched environments.

[balakumar-s]

@frederiknolte, @Arpitrf, and others, how many environments are you wanting to use for batched planning with attach/detach?

We use a cpu implementation to compute spheres given a mesh, so this can slow down object attachment.

[Arpitrf]

I see. Hoping to use as much GPU memory allows. I would say in the order of 10s.