Ask a video generator for "a glass falling off a table" and you get a convincing clip. Ask the same thing and then follow up with "and what if the table were 20 cm taller?" — and the model has no way to answer, because it never modelled the table. It modelled pixels that tend to appear together.
That is the whole frontier. A world model is a model trained to answer the second question.
🌍 Series "World Models" — 5 parts
1. What a world model is ← you are here · 2. Cosmos 3 inside · 3. Synthetic data · 4. Splitting across GPUs · 5. From text to action
The question that separates the two
A video generator learns a distribution: which image sequences are plausible? That is a problem of appearance, judged by whoever watches.
A world model learns a dynamic: given the current state and an action, what is the next state? That is a problem of consequence, judged by whatever acts on it — a robot arm, a route planner, a control system.
| Video generator | World model | |
|---|---|---|
| Input | text (or image) | state + action |
| Output | a video | the next state (which may be video) |
| Success criterion | does it look good and coherent? | would this actually happen? |
| Consumer | a person | a control system |
| Typical failure | a visual artefact | a wrong decision in the real world |
This is why the word "action" shows up in every serious piece of material on the subject. Without an input channel for the action, you do not have a world model — you have a generator with better marketing.
The three requirements most systems fail
- Action conditioning. You need to be able to say "the arm moves 3 cm left" and see the effect. That is not the same as describing the action in text: it means injecting a numeric vector per frame.
- State coherence over time. The object that left the frame has to still be there when the camera comes back. This is the hardest part, and the one that fails most.
- Interactivity or a long horizon. Either the model accepts a new action at each step, or it can project tens of coherent seconds at once. Five seconds simulate nothing.
💡 Why "simulation" is both the wrong word and the right one
A classical simulator (a physics engine) computes the consequence from laws: mass, friction, collision. It gets the physics right and the appearance wrong — simulator worlds look like plastic.
A world model learns the consequence from video. It gets the appearance right and approximates the physics. Both are wrong, in opposite directions — which is exactly why serious robotics teams use both, not one.
Who is building, as of September 2026
| Bet | Who | The idea |
|---|---|---|
| Cosmos | NVIDIA | Open omnimodal model aimed at generating data and simulating scenes for robotics and autonomous vehicles. Downloadable weights. |
| Genie | Google DeepMind | An interactive environment generated from text, navigable in real time. The bet closest to "a video game that did not exist". |
| JEPA | Meta / AMI Labs | Predict in representation space, not pixel space. The thesis that generating pixels wastes model capacity. |
| Marble | World Labs | Persistent 3D scene reconstruction — the world as a navigable object, not as video. |
The JEPA thesis deserves a paragraph because it is the most contrarian: predicting every pixel spends most of the model's capacity on irrelevant detail (the exact texture of the asphalt). Predicting in an abstract representation space concentrates the model on what matters for deciding. It is Yann LeCun's line, now at AMI Labs, and it is an explicit bet against the architecture that dominates LLMs.
Who is right is still an open question. What is already fact is that the open weights available today are on the pixel-generating side — which is why the practical half of this series is about them.
What this is actually good for
1. Manufacturing training data that does not exist. The dominant use today, and the subject of part 3. Collecting 10,000 hours of real robot time is expensive; generating the rare variation — the child running across, the box that slips — is cheap.
2. Evaluating a policy without breaking anything. Before running a new controller on the physical robot, you run it against the world model and see where it gets stuck. Failing in simulation costs seconds of GPU; failing in the warehouse costs a robot arm.
3. Planning by imagination. The system mentally projects three possible actions, sees the predicted outcome of each, and picks one. The highest promise and the least mature — it depends on exactly the physical accuracy the models do not yet have.
What does not work — and NVIDIA is the one saying it
Worth quoting, because it is rare for a vendor to document its own failures this clearly. The Cosmos 3 model card lists, without hedging: objects that disappear or morph, unrealistic collisions, state drift over long sequences — and the cause: there is no physics simulator inside the model. 3D geometry, object permanence and contact dynamics are approximated, not computed.
⚠️ The sentence that belongs in every article on the subject
NVIDIA writes, on the model page itself, that the outputs should not be treated as physically accurate simulation, reliable ground truth, or safety-certified decision making.
Applications in robotics, autonomous systems or safety-critical planning require additional validation, external constraints and system-level safety analysis. A world model is a source of hypotheses, not an authority.
What this changes if you have a GPU today
World models are the most expensive category of open-weight model to run, for a structural reason: they must hold dozens of frames coherently in memory at once, with temporal attention between them. It is not an LLM at larger scale — it is a different consumption profile.
| What you want to do | Realistic card | Price/h |
|---|---|---|
| Run a 7B action model (VLA), fine-tune on your own robot | 24 GB | R$ 3.18 |
| Generate short clips with an open video model | 48 GB | R$ 7.08 |
| Serve a large world model, no parallelism | does not fit on one card | — |
| Serve a large world model, in a block | 2 to 8 cards of 141 GB | see part 4 |
Catalogue prices as of 1 September 2026, per single GPU, no scheduled interruption. Prices are in Brazilian reais (BRL), the billing currency. The catalogue moves within the same day — check the live one before committing to a number.
The third row is the surprising one: the flagship model in this category is 132.7 GB of weights alone. No single card on the market holds that — not even the 141 GB one, because activations still need room. Part 4 shows how it is split, and the counterintuitive arithmetic that more GPUs buy time, not savings.
How to read the rest of the series
- Part 2 — inside Cosmos 3: the two transformer towers, the five checkpoints, and what an hour costs.
- Part 3 — synthetic data: why it is the use that pays the bills, and where it deceives you.
- Part 4 — 132 GB will not fit on one card: the five kinds of parallelism, in plain English.
- Part 5 — from text to action: what changes when the model's output is a motor command.
Try before you commit
Large world models are expensive. A 7B action model runs on a 24 GB card for R$ 3.18 an hour, billed by the hour, in Brazilian reais, no contract. It is the honest way to find out whether the category solves your problem before sizing an eight-card block.
See the catalogue →Keep reading: part 2: inside Cosmos 3 · part 3: synthetic data