Text-to-Image

Evaluate image generation models on photorealism, prompt adherence, compositional understanding, typography, style control, and artifact detection across 6 leading models.

Lobster Test

"A lobster riding a dirt bike" — can the model compose two unrelated concepts in a novel spatial relationship?

“A lobster riding a dirt bike”

Scoring Rubric

Metric	1 (Low)	5 (High)
Photorealism	Obviously fake / incoherent	Indistinguishable from photo
Prompt Adherence	Ignores prompt	Every element present and correct
Composition	Objects randomly placed	Correct spatial relationships
Typography	No readable text	Perfect text rendering
Style Control	Wrong style entirely	Nails the requested style
Artifacts	Major artifacts (mangled hands, extra limbs)	No detectable artifacts

Models Tracked

GPT-4o (native)

API

OpenAI — native LLM image generation

Midjourney v7

UI Only

Discord/web UI — no reliable API

Flux 1.1 Pro

API

Black Forest Labs via Replicate/fal.ai

Ideogram 2.0

API

Best-in-class text rendering

Imagen 3

API

Google via Vertex AI

Recraft V3

API

Strong on design/vector

Metrics Evaluated

Prompt adherenceCompositional accuracyVisual fidelityText renderingArtifact score

Results Coming Soon

Benchmark results will be published here after the first monthly run. Scores, rankings, trend charts, and the lobster gallery — all in one place.