What it ran with
The model never writes motor commands. It sees two camera images and the arm’s state, writes a short plan, and makes one tool call. A harness runs that call in a LIBERO simulation of a Franka arm and sends back what happened. The simulator alone decides if the task is done.
Images and arm state in. Reasoning and one tool call out.
Checks the call, drives the arm, measures what happened.
LIBERO on MuJoCo. Judges the task.
No object positions, no simulator state and no success signal reach the model. Built on the open-source uni-robot-agent harness.
Before training
Out of the box, Qwen3.5-4B solved none of the 40 unseen scenes. It wrote long reasoning, lost track of where things were and drove the arm away from the objects. On the runs below, its median reasoning was 1,741 characters per turn. The teacher’s was about 210.

I see now - maybe there ISN’T actually a separate “black bowl” visible!
Training
Only successes were kept. The test scenes use starting layouts that never appear in training.
After training
It learned the short pick-and-place tasks: 6 of 10 object scenes and 3 of 10 goal scenes. It solved none of the spatial or two-step tasks.
It also started to reason like the teacher. Its first turn on the bowl-and-plate scene:
I need to analyze the current scene and plan how to put a bowl on a plate. From the initial observation, I can see: The arm is at position [-0.21169, -0.0222, 1.17221] with orientation [-3.08354, 0.00132, 1.57123] (roll, pitch, yaw in radians)… I should first capture the scene again to get fresh observations…
The initial images show the bowl behind the empty plate, with the gripper open above the table. I’ll measure their visible surfaces to plan a grasp and placement that accounts for any offset from the plate center.
To see what changed in how it sees, I replayed recorded runs and asked both versions 270 questions about the frames, three times each, graded against the simulator’s ground truth. Pointing at the named object went from 16% to 42%. Judging whether an object is in place improved a little. Telling whether the gripper holds something did not improve at all.
What it lacks: seeing
Training taught it the teacher’s moves and habits. It did not teach it to understand what the cameras show. That gap explains a lot of its failures.
- It can’t tell a full gripper from an empty one. 58% on yes or no, close to a coin flip. So it carries nothing and describes a grasp.
- It points at the wrong thing. Only 42% of its points land on the object it names. Of the 255 that miss, 123 land on the table.
This is the starting point
A 4B model can learn to act through tools. Next, it learns to see, then to improve on its own.
- SFT on multimodal understanding. The simulator knows the true answer for every frame, so the data labels itself: point at the named object, say where it is in 3D, say whether the gripper holds it, say whether it is in place.
- Then RL. Let the model run the robot and learn from the simulator’s verdict, with task success as the reward.
Same two tests after each step: the questions above and the 40 unseen scenes.
Watch the runs
Three unseen scenes, all three models. Each run replays as it happened: the cameras on the left, every message the model sent and received on the right.