Playing a policy

To run a policy on the robot, select the policy and checkpoint you want to run and press load. The play-pause button can then be used to toggle policy execution.

The robot will now start executing the commands it receives from the policy. Each key in the policy input should be received by the Incar Skill System, otherwise this will log an error. For each key that the policy is predicting as output, you should make sure that the robot has a callback assigned. Generally, this implementation will already exist since the keys are the same as for the teleoperation. For example, the following policy:

"policy": {
    "type": "diffusion",
    "dt": 0.1,
    "input_features": {
        ...
    },
    "output_features": {
        "left.commands.arm.ee.velocity": {
            "type": "ACTION",
            "shape": [6]
        },
        "left.commands.gripper.openclose": {
            "type": "ACTION",
            "shape": [1]
        },
        "left.routines": {
            "type": "ACTION",
            "shape": [2]
        }
    },
    "preprocessing": ...
}

will output commands under the keys left.commands.arm.ee.velocity, left.commands.gripper.openclose, left.routines. Routine commands can be interpreted by the robot to e.g. execute hardcoded routines.

Overriding policy control with teleoperation

When the headset is connected, teleoperated commands to the robot will take priority over those actions from the policy. This means that when the controller is engaged, the policy commands will be overriden by the controller commands and you can control the robots with the controller. Once the controller is disengaged, the policy will take over control again seamlessly, as it has constantly been diffusing updated trajectories during the takeover control.

We call corrections made to a running policy takeovers.

Only the features provided by the teleoperation will override the policy control. This can allow for co-operation of different robot modules, e.g. let the policy keep control of hand motions while you are only correcting the end-effector movement. If you are predicting different features than you are sending via teleoperation and want to make sure that the policy does not keep any control, you should ensure that priority is configured correctly by the robot controller.

Danger

Taking over control is not a replacement for an e-stop button. Always ensure the proper safety measures are taken when operating a physical robot.

Recording takeover demos

If a dataset is selected, any data during a takeovers will automatically be saved to the dataset and flagged as a takeover demo for filtering purposes. If two or more consecutive takeovers happen within 1.5 seconds of each other, they are merged as one recording.

Hotswapping policies

To hotswap a policy, simply select the new policy and checkpoint you want to run and press select policy. The robot will seemlessly transition the policy from which it performs inference. For larger policies such as s2, there might be some loading time before the switch happens.