Training a policy

In order to train a policy, simply run the following command

LinuxWindows

# Remember to source the environment first with $ `source [path_to_env]/bin/activate`
incar train --config_path=[config_path]

# Remember to source the environment first with $ `[path_to_env]\Scripts\activate`
incar train --config_path=[config_path]

where config path is a relative or absolute path to the training config file, which contains policy and training information. It is possible to manually override parameters from the config file. For example:

LinuxWindows

# Remember to source the environment first with $ `source [path_to_env]/bin/activate`
incar train --config_path=./my_policy.json --resume=true --model_save_name=example_model

# Remember to source the environment first with $ `[path_to_env]\Scripts\activate`
incar train --config_path=./my_policy.json --resume=true --model_save_name=example_model

Info

This also works with nested configuration, e.g. passing --policy.preprocessing.validation_ratio=0.15 will set the validation ratio to 15%.

Training config

Oh, you want to know what to put in the config? Alright then... Lets take a look at an example training config:

training_example.json

{
    "workspace": "~/my_ws",
    "model_save_name": "example_policy",
    "preprocessed_dataset_name": "example_dataset",

    "policy": {
        "type": "diffusion",
        "dt": 0.1,
        "input_features": {
            "wrist_cam": {
                "type": "VISUAL",
                "shape": [3, 240, 320]
            },
            "arm.ee.pose": {
                "type": "STATE",
                "shape": [7]
            },
            "gripper.joints.position": {
                "type": "STATE",
                "shape": [1]
            }
        },
        "output_features": {
            "right.commands.arm.ee.velocity": {
                "type": "ACTION",
                "shape": [6]
            },
            "right.commands.gripper.openclose": {
                "type": "ACTION",
                "shape": [1]
            }
        },

        "preprocessing": {
            "raw_dataset_name": "example_dataset",
            "validation_ratio": 0.15,
            "steps": [
                {
                    "type": "sample_dt",
                    "dt": 0.1
                },
                {
                    "type": "downsample_video",
                    "features": ["wrist_cam"],
                    "new_size": [240, 320]
                },
                {
                    "type": "filter_by_buttons",
                    "button_feature_names": ["right.buttons"],
                    "button_names": ["gripValue"]
                },
                {
                    "type": "image_transform",
                    "features": ["wrist_cam"],
                    "noise_sigma": 0.05,
                    "transforms": {
                        "brightness": [0.6, 1.4],
                        "contrast": [0.6, 1.4],
                        "saturation": [0.6, 1.4],
                        "hue": [-0.1, 0.1]
                    }
                }
            ]
        }
    },

    "batch_size": 64,
    "num_workers": 8,
    "save_freq": 20000,
    "validate_freq": 1000
}

A little bit intimidating maybe, but worry not! The required fields are quite straight-forward:

• workspace is the path to the workspace where the dataset will be retrieved from and the model will be saved.
• model_name is the name the model will get.
• preprocessed_dataset_name is so you can save the preprocessed dataset, so that you can train multiple methods without needing to redo laborous work
• policy defines the policy. Some of the keys in this dict will depend on what type it is, as you can even implement your own policy type. However they will have things in common, namely:
  • dt: the timestep with which the policy generates a trajectory.
  • input_features are features that are inputs to the policy. They need to match a feature that is in the dataset.
  • output_features are features that are output by the policy. They need to match a feature of type ACTION in the dataset.
  • preprocessing is the preprocessing pipeline. Basically we define it here once and it ensures that the processing of data always matches during training and inference! You can add multiple steps, even write your own steps, and decide where in the pipeline they should be applied using the processing pipeline hooks. You can probably start with the steps in the config above, as they are quite standard and often good enough to get started with a simple first policy!
    • raw_dataset_name is the dataset that will be trained on, if no cached preprocessed dataset is used.
    • validation_ratio is the ratio of demos that will be reserved for validation.

Important

In order for training to work, the input and output feature keys must all be present within the dataset that is used for training, as well as match in their type and shape!

Next step: Running the policy