Qwen3.5-4B before training failed
Qwen3.5-4B after training succeeded
GPT-6 Astra teacher succeeded
0/40before training
9/40after training
31/40the teacher, GPT-6 Astra

unseen scenes solved, out of 40

What it ran with

The model never writes motor commands. It sees two camera images and the arm’s state, writes a short plan, and makes one tool call. A harness runs that call in a LIBERO simulation of a Franka arm and sends back what happened. The simulator alone decides if the task is done.

Model

Images and arm state in. Reasoning and one tool call out.

tool call →← result + images
Harness

Checks the call, drives the arm, measures what happened.

commands at 20 Hz →← state + cameras
Simulator

LIBERO on MuJoCo. Judges the task.

Look new pictures
Point pixels to 3D positions, from depth
Move gripper to a pose, open or close
Wait let the fingers settle
Done end the run

No object positions, no simulator state and no success signal reach the model. Built on the open-source uni-robot-agent harness.

Before training

Out of the box, Qwen3.5-4B solved none of the 40 unseen scenes. It wrote long reasoning, lost track of where things were and drove the arm away from the objects. On the runs below, its median reasoning was 1,741 characters per turn. The teacher’s was about 210.

Qwen3.5-4B before training: its wrist camera faces a wall while the black bowl sits in plain view in the scene camera
I see now - maybe there ISN’T actually a separate “black bowl” visible!
Turn 44 of 46, “pick up the black bowl between the plate and the ramekin”. The wrist camera (right) faces a wall. The bowl is in plain view (left).

Training

GPT-6 Astraruns the same harness on training layouts
→
793successful runs kept
→
20,236decisions: images, reasoning, one tool call
→
Qwen3.5-4Bfull fine-tune, 3 epochs

Only successes were kept. The test scenes use starting layouts that never appear in training.

After training

It learned the short pick-and-place tasks: 6 of 10 object scenes and 3 of 10 goal scenes. It solved none of the spatial or two-step tasks.

Unseen scenes solved
Qwen3.5-4B before training Qwen3.5-4B after training GPT-6 Astra (teacher)
Objectpick the named item
0/10
6/10
8/10
Goalsame layout, different goal
0/10
3/10
8/10
Spatialpick the bowl by position
0/10
0/10
9/10
Longtwo steps in one task
0/10
0/10
6/10

It also started to reason like the teacher. Its first turn on the bowl-and-plate scene:

Before
I need to analyze the current scene and plan how to put a bowl on a plate. From the initial observation, I can see: The arm is at position [-0.21169, -0.0222, 1.17221] with orientation [-3.08354, 0.00132, 1.57123] (roll, pitch, yaw in radians)… I should first capture the scene again to get fresh observations…
After
The initial images show the bowl behind the empty plate, with the gripper open above the table. I’ll measure their visible surfaces to plan a grasp and placement that accounts for any offset from the plate center.

To see what changed in how it sees, I replayed recorded runs and asked both versions 270 questions about the frames, three times each, graded against the simulator’s ground truth. Pointing at the named object went from 16% to 42%. Judging whether an object is in place improved a little. Telling whether the gripper holds something did not improve at all.

Questions about recorded frames, share answered correctly
Qwen3.5-4B before training Qwen3.5-4B after training chance
Point at the named objectthe pixel it picks lands on the object
16%
42%
Is the object in place?yes or no, half the answers are yes
66%
74%
Is the gripper holding it?yes or no, half the answers are yes
58%
58%

What it lacks: seeing

Training taught it the teacher’s moves and habits. It did not teach it to understand what the cameras show. That gap explains a lot of its failures.

  • It can’t tell a full gripper from an empty one. 58% on yes or no, close to a coin flip. So it carries nothing and describes a grasp.
  • It points at the wrong thing. Only 42% of its points land on the object it names. Of the 255 that miss, 123 land on the table.

This is the starting point

A 4B model can learn to act through tools. Next, it learns to see, then to improve on its own.

  1. SFT on multimodal understanding. The simulator knows the true answer for every frame, so the data labels itself: point at the named object, say where it is in 3D, say whether the gripper holds it, say whether it is in place.
  2. Then RL. Let the model run the robot and learn from the simulator’s verdict, with task success as the reward.

Same two tests after each step: the questions above and the 40 unseen scenes.

Watch the runs

Three unseen scenes, all three models. Each run replays as it happened: the cameras on the left, every message the model sent and received on the right.

Qwen3.5-4B before training failed
Qwen3.5-4B after training succeeded
GPT-6 Astra teacher succeeded