The August datasets

The demonstrations behind the evaluated policy

150 teleoperated demonstrations, about 58 minutes of recording: a person drives the leader arm while the cameras, the joint positions and the prompt are logged at 30 frames a second. Each demonstration is one move.

The set is sari-abdan/vial-sort-v5 on Hugging Face, and every episode can be watched in the browser.

How the demonstrations were recorded

• Three vial colours, red, cyan and green, three vials of each.
• Two six-slot racks at a diagonal to the arm, the bin on the left.
• Places are named from the side camera: the left rack is the one by the bin, and slots run 1 to 6 from left to right.
• Vials are spaced so no two stand side by side, at most three to a rack; the other vials are never the colour being moved.
• Slots 2 and 5 are never a source, so the evaluation can test two positions no demonstration started from.
• The depth stream is recorded with every episode and not used by the policy.

Every demonstration has its own wording

150 wordings for 150 demonstrations, so the policy has to read the positions rather than memorise one sentence. The 8 for R1 → bin:

• “Bring the tube at hole 1 of the rack on the right into the trash can.”
• “Carry the test tube in position 1 of the right-side rack to the bin.”
• “Move the sample tube from slot 1 of the right tube rack over to the trash.”
• “Move the vial from position 1 of the right rack to the bin.”
• “Relocate the test tube from spot 1 of the right-side rack to the rubbish bin.”
• “Take the sample tube at the first slot of the right tube rack into the waste bin.”
• “Take the vial in slot 1 of the right rack to the trash.”
• “Transfer the tube from the first slot of the rack on the right over to the waste bin.”