Teaching a Robot Arm to Play Tic-Tac-Toe

Overview

In March 2025 I teamed up with four others for a Seeed Studio robotics hackathon at Circuit Launch. Our project was to get a pair of open source SO-ARM100 robot arms to play tic-tac-toe.

To train the arms we used Hugging Face’s LeRobot library. Rather than writing code to play the game and separate code to move the pieces, we wanted to see if a single model could learn to do the whole thing from demonstrations, looking at the board, picking a move and placing the block. We knew going in that having the model learn both the grasping and the game logic was going to be a lot, but we decided to try anyway.

The code can be found here, the dataset we recorded is on Hugging Face and our presentation from the end of the hackathon is here.

One of our training demonstrations from the scene camera (left) and the gripper camera (right).

Initial Plan

The SO-ARM100s were set up as a leader and follower pair, where you move the leader arm by hand and the follower copies what it does. This makes it easy to record demonstrations of a task, which LeRobot then uses to train a model that takes in the camera images and arm positions and outputs motor commands.

Our initial plan was to run through the example wiki, set up the robots with a couple of simple USB cameras, and get a dev environment running on a desktop with an i9 and an RTX 3070 (8GB). In parallel we would train a simple model using ACT (Action Chunking with Transformers) and pick a foundation robot model to use. We went with Pi0 since it looked really interesting and had already been trained on multiple tasks.

Robots Being Robots

Unfortunately, like most hackathon projects, very little of the initial plan worked the first time.

The robot didn’t come with a camera mount for the gripper, which meant we couldn’t collect first person video for the training data. We ended up 3D printing a new camera mount, which took about 3-4 hours.

The robot would also crash if the gripper was moved too quickly. We never solved this one, we just did the teleoperation very very carefully.

Data recording with the cameras wouldn’t work on some of our computers. We debugged it for a bit and then ended up pivoting to doing everything on the desktop.

Finally, Pi0 didn’t work for us. The Pi0 model uses ALL the VRAM for training and inference, and since we didn’t have ALL the VRAM we had to drop it and use ACT instead.

There were also a number of smaller hacks to get things running, like changing the camera indexes and serial ports for each computer. When we loaded a model that had been trained on a different machine it failed on two config keys (device and use_amp) that our version didn’t recognize, so the fix was to pop them out of the config before loading.

Final Approach

With Pi0 out the final plan was to record expert demonstrations of tic-tac-toe moves with the leader and follower setup, train a model with ACT and run it locally on the desktop. Along with the gripper camera we added a scene camera looking down at the board.

We recorded 50 “expert” demonstrations evenly distributed over all the turns in a game. Each one started with a board state, then a human move, then a teleoperated robot move, covering the robot’s first, second, third and fourth moves along with the end of a game after the fifth human move. In total that came to 29,756 frames at 30 fps, or about 16 and a half minutes of data.

The initial model was trained with ACT for 5 hours (100K steps) on the RTX 3070. Another competitor was nice enough to let us use their cloud H100 to speed up training, and as we captured new data we continued training from the previous model rather than starting over to keep the training times down.

Results

It played terribly. It could pick up cubes and move them onto the board, but where it put them seemed random, with no sign that it was following the game.

This wasn’t too surprising. ACT learns to copy the motions it was shown, it doesn’t actually know the rules of tic-tac-toe, so to play well it would have had to figure out the game from 50 demonstrations spread over a lot of different board states while also learning how to pick up and place the blocks. That is a lot to ask from that little data. The more reliable approach would have been to use a simple program to pick the move and only have the model handle the picking and placing, but we wanted to see how far we could get with the model doing everything.

The board from the scene camera during one of the teleoperated training demonstrations.