Heng Fang1 · Zehui Guo1 · Yuehan Zhang1 · Mingyang Du1,2 · Dingkang Liang1,2 · Xiang Bai1
1Huazhong University of Science and Technology 2ArcheOne AI
Learn a manipulation task in context, not in weights:
one demonstration, rehearsal in a world, no gradient updates.
RoboScript is an agent framework for robot manipulation without a trained action policy. A frozen foundation model starts from one demonstration, rehearses the task in a world such as a simulator, and writes what it learns into a Task Memory in its context instead of its weights. We call this World In-Context Learning (WICL).
Four stages: build reusable skills from demonstrations, learn a Task Memory from one demonstration, deploy the frozen Memory on new scenes with only what a real robot can sense, and improve by reflecting on past evaluations. Currently validated on dual-arm tabletop manipulation.
On RoboTwin 2.0 (50 bimanual tasks), RoboScript achieves 83.4% mean success with one demonstration per task and no gradient updates, compared to 61.7% for the strongest trained policy using 50 demonstrations per task. Rehearsal happens only on clean scenes; randomized scenes see almost no drop (83.6% → 83.2%). Results on additional benchmarks are in progress.
Full results table
† Randomized setting only, five seeds per task; N is the unreported number of demonstrations used to fine-tune the frozen base.
Works on unseen scenes. Episodes from 20 tasks on clean and randomized scenes; randomized scenes are never rehearsed. Clips are sped up.
Runs on a real robot. Memories rehearsed on a simulated ALOHA run on a dual-arm PiPER through parameter remapping, without rehearsing again. Clips are sped up.
Each component runs as its own service. Methods, benchmarks, embodiments and skillsets are plugins.
| Layer | What it does |
|---|---|
| Model gateway | Routes LLM calls to any provider, with session pinning and failover |
| Agent plane | Agent loop, tool Hub, skills — limited to what a real robot can sense |
| Control plane | Runs the learning protocol, holds ground truth. Only the official check decides success |
| Services | Motion planning (cuRobo), perception (GroundingDINO, SAM 3), blob storage |
Add a benchmark, skillset, real robot or other extension without changing the core.
Run one episode on the built-in Mock benchmark. It needs no simulator, GPU or model.
git clone https://github.com/ArcheOne-AI/RoboScript.git
cd RoboScript
uv sync --all-packages
cd examples/tutorials
uv run python -m quickstart_mock.runThen continue with the tutorials, pick a benchmark from the table below, or read the documentation. The vocabulary defines terms such as skillset, brain and recipe. To change the code, see CONTRIBUTING.md.
Framework core, three learning methods (wicl, rsi, skill_builder), the vision service, and
a real-robot package. Pre-1.0; interfaces may change.
| Benchmark | Status |
|---|---|
| Mock | Built in, no simulator needed. Used by the tutorials. |
| RoboTwin 2.0 | Full pipeline with official evaluation. |
| BEHAVIOR | Integration in development. |
To add another benchmark, see add-a-sim-benchmark.
- Real-robot deployment code.
@article{fang2026roboscript,
title = {World In-Context Learning for Agentic Robot Manipulation},
author = {Fang, Heng and Guo, Zehui and Zhang, Yuehan and Du, Mingyang and Liang, Dingkang and Bai, Xiang},
journal = {arXiv preprint arXiv:2611.12885},
year = {2026}
}Apache License 2.0. See LICENSE and NOTICE, which lists adapted third-party
code. Benchmark simulators and their assets are not part of this repository and come under their
own licenses.





