All guides

How to evaluate a world model demo without overclaiming.

Judge demos by interaction, persistence, access, and failure modes. A smooth video is not evidence of a world.

How to evaluate a world model demo without overclaiming. guide cover

Takeaways

  • Watch what happens after the camera moves or the user acts.
  • Separate public demo, gated preview, product, and paper.
  • Write down what the demo cannot do before repeating what it appears to do.

The test

A world demo has to survive user action.

If the only evidence is a camera path the authors chose, you are watching video. Oasis makes this obvious because keyboard input changes the next frame. Genie makes it obvious when the prototype lets a person navigate and prompt events.

Marble and HappyOyster should be judged the same way, even when the interface looks like a product: can the visitor direct, revisit, or export anything that remains a place?

Access label

Name the surface before praising the capability.

Genie is a research preview with gated Project Genie access. Oasis is a public demo. HappyOyster is early access. Marble is a public product. Those labels are part of the evaluation, not marketing tone.

A gated photorealistic preview can still be the most important research signal. It cannot be cited as something most readers can try unless the current official page says they can.

Failure modes

Look for the first broken turn, not the hero shot.

Useful notes include: identity drift, geometry collapse, ignored controls, memory loss after a few seconds, and export formats that do not match the demo. Write those down.

This site's news pages use overclaim warnings for the same reason. A launch clip can be true and still not support pricing, physics, or availability claims.

After the demo

Open the dossier, then the comparison, not another clip.

The demo wall is an entry point. The model page holds availability, limits, and sources. The comparison page holds the decision a reader is actually making.

If the demo was about inhabitants rather than scenery, Project Sid is the adjacent page, not a visual-generation alternative.

FAQ

Should I trust FPS or photorealism claims from a trailer?

Only when a primary source names the model variant, hardware, and setting. Oasis and LingBot-World-Fast are examples where latency claims should be tied to specific cards or papers, not to a highlight reel.

Decision tables

Use a comparison when two lanes are still competing.

Sources

Discussion

Reader discussion

Add source-backed corrections, questions, or notes for this page.

Loading comments
Loading discussion...

Loading comments...