A language model has the entire internet to learn from. A model that controls a robot arm has whatever your team managed to record in the warehouse — and that usually fits in a few hundred hours.
That asymmetry is the real subject of world models. It is not about generating pretty video: it is about the fact that data is the bottleneck, and somebody worked out you can manufacture it.
🌍 Series "World Models" — 5 parts
1. What a world model is · 2. Cosmos 3 inside · 3. Synthetic data ← you are here · 4. Splitting across GPUs · 5. From text to action
Why robotics data is so expensive
1. The data is physically slow. An hour of demonstration is an hour of wall-clock time, with a robot, in a space, with someone operating it. You cannot parallelise it by buying GPUs. Ten thousand hours of demonstration are ten thousand hours.
2. The robot wears out. Every repetition consumes gripper, motor and cable life. Robotics data has a marginal cost in hardware, not just time.
3. The case that matters almost never happens. The worst of the three. You need to train behaviour for the child running after a ball, the box slipping off the stack, the sun glare that erases the lane marking. These are exactly the events you cannot record in quantity — because they are rare, and because provoking them deliberately is dangerous or unethical.
💡 The inversion that changes the game
In the real world, common data is cheap and rare data is extremely expensive — you record a thousand hours of normal traffic to get three seconds of near-miss.
In a world model, both cost exactly the same: one prompt. That inversion, not image quality, is what sustains the investment in the entire category.
The four ways to generate the data
| Technique | What it does | When to use it |
|---|---|---|
| Text → scene | Generates the situation from scratch from a description. | Covering a scenario you have never recorded. |
| Image → video | Starts from a real frame and continues the motion. | Keeping the realism of your actual environment. The most reliable path. |
| Style transfer | Takes simulator output (right geometry, plastic look) and makes it photorealistic. | You already have a simulator. Probably the best value of all four. |
| Action-conditioned | You supply the trajectory and the model shows the result. | Generating (action, consequence) pairs — the data that trains a control policy. |
The third row deserves emphasis because it is the least obvious and the most practical. If your team already has a simulator, the geometry and the physics are already right — what is missing is appearance, and appearance is exactly what generative models are good at. You use each tool for what it gets right, instead of asking physics from a model that has no physics.
The proof it works is in Cosmos's own training set
A detail from the Cosmos 3 card that almost nobody comments on. The training set has 1.3 billion data points from 393 datasets. Among them, the declared synthetic ones:
| Synthetic source | Samples |
|---|---|
| Images generated by an open image model | 15 million |
| Images generated by another open image model | 14 million |
| Captions generated by a vision-language model | 1.115 billion |
Read that last row again. Over one billion synthetic captions — the overwhelming majority of the textual description that taught the model to associate word and scene was written by another AI, not by people. NVIDIA's reference world model is itself mostly trained on generated data.
That is strong validation of the thesis. It is also the best argument for the risks in the next section.
The three ways synthetic data deceives you
1. It teaches wrong physics, convincingly. The central risk, and NVIDIA documents it: there is no physics simulator inside the model. Contact, friction and object permanence are approximated. If you train a controller on videos where the box slips in a way real boxes do not, you have not trained anything — you have trained a superstition. And it will look perfectly convincing on video.
2. It narrows the distribution silently. The generator has its own biases: certain angles, certain lighting, certain motion rhythms appear more often. Training on that pushes your system toward the subset of the world the generator likes. The symptom is treacherous: metrics on synthetic test data improve while real performance degrades.
3. It erases its own trail. Six months later, nobody on the team remembers which samples were real and which were manufactured. When the strange behaviour shows up in production, there is no way to audit the origin.
⚠️ The one discipline that fixes all three
Tag the provenance of every sample from day one, and keep an evaluation set that is exclusively real and never receives anything generated.
Synthetic data can only be judged by performance on real data. Measuring a synthetic-trained model against a synthetic test set measures how consistent the generator is with itself — the number goes up and means nothing.
It is boring metadata work, and it is the difference between a synthetic data programme that works and one that poisons the corpus for a year before anyone notices.
What it costs to manufacture that data
Unlike serving a model to a waiting user, data generation is batch work: nobody is watching, what matters is the total number of clips by morning. That completely changes the machine choice — for the better.
| Batch scale | Sensible configuration | Price/h |
|---|---|---|
| Prove the concept, dozens of clips | light model (16B) on one 96 GB card | R$ 10.26 |
| Hundreds of clips per night | large model on 2 cards of 141 GB | R$ 57.10 |
| Thousands of clips, tight deadline | block of 8 cards | R$ 228.40 |
| Style transfer over your own simulator | smaller model, one 48 GB card | R$ 7.08 |
Catalogue prices as of 1 September 2026, per single GPU, no scheduled interruption. Prices are in Brazilian reais (BRL), the billing currency. The catalogue moves within the same day — check the live one before committing to a number.
As part 2 showed, the 8-card block produces each clip faster and more expensively. For batch work — which is the case here — the 2-card configuration delivers more clips for the same money. Worth repeating because it is the opposite of the "bigger machine is more efficient" intuition.
A note on method: generate a small batch first and measure the real yield. In action-conditioned generation it is normal to discard a meaningful fraction of the output for incoherence. If you keep 60%, your cost per useful clip is 1.7× the cost per generated clip — and that is the number that belongs in the budget.
An honest starting playbook
- Pick one specific rare case you know the system gets wrong today. Do not start with "generate data" in general — start from a measured gap.
- Freeze a real evaluation set before generating anything. Without that yardstick you will not be able to prove anything afterwards.
- Generate a little and look at it yourself. A hundred clips, reviewed by a person. The discard rate you measure here sizes everything else.
- Train on a mixture, never pure synthetic. The healthy ratio varies; what does not vary is that real data has to be in there.
- Compare on the real yardstick. If it did not improve on the real set, it did not improve. However good the video looked.
It is the same rigour you would apply to any new dataset. The specific temptation here is to assume that because the video is beautiful, the data is good — and those are completely separate things.
Generate a test batch before deciding
A proof-of-concept batch fits in a few hours on one card. Billed by the hour, in Brazilian reais, no contract — you measure the real yield before sizing anything larger.
See the catalogue →Keep reading: part 1: what a world model is · part 5: from text to action