Text-to-Video Game

Single Player

Evaluate models and tools that generate playable single-player game experiences from text descriptions. Tests game logic, visual quality, input handling, and code quality across 6 models.

Lobster Test

"A 2D platformer where a lobster on a dirt bike collects golden shells while avoiding seagulls."

“A 2D platformer where a lobster on a dirt bike collects golden shells while avoiding seagulls”

Scoring Rubric

Metric	1 (Low)	5 (High)
Playability	Crashes / unplayable	Polished, fun to play
Visual Quality	Broken / no graphics	Clean, consistent art style
Input Handling	Unresponsive	Smooth, intuitive controls
Rule Adherence	Ignores described mechanics	All mechanics implemented correctly
Complexity	Trivially simple	Full game loop with polish
Bug Density	Constant crashes	No bugs in 5 min playtest
Code Quality	Unreadable / broken	Clean, idiomatic, well-structured

Models Tracked

Claude Opus 4

API

Strong spatial reasoning, full HTML5/JS games

Claude Sonnet 4

API

High-value tier, test against Opus

GPT-4o

API

Canvas/code interpreter game generation

Gemini 2.5 Pro

API

Strong code gen, long context

Bolt.new

UI Only

Full-stack app builder, web games

Lovable

UI Only

React app generation, game-like experiences

Metrics Evaluated

PlayabilityVisual qualityLogic coherenceBug densityCode quality

Results Coming Soon

Benchmark results will be published here after the first monthly run. Scores, rankings, trend charts, and the lobster gallery — all in one place.