Updated Monthly

VCL AI Benchmarks

Same prompts. Every major model. Every month. No marketing fluff — just results.

The “Lobster on a Dirt Bike” Test

Our signature compositional benchmark, run across every category and every model, every month.

Inspired by Simon Willison's “pelican on a bicycle.” The simplicity is the point — failure modes are endlessly informative.

View Gallery

Text-to-Image

Photorealism, prompt adherence, compositional understanding, typography, and artifact detection.

Prompt adherenceCompositional accuracyVisual fidelity+2

6 models tracked

Text-to-Text

Reasoning, writing, coding, instruction following, factual accuracy, and long-context understanding.

ReasoningCode generationInstruction following+2

8 models tracked

Text-to-3D Scene

Video Export

3D scene generation with flythrough video export. Geometry, textures, lighting, and atmosphere.

Geometric coherenceTexture qualityLighting & atmosphere+2

4 models tracked

Text-to-Video Game

Single Player

Generate playable games from text. Game logic, visual quality, input handling, and complexity.

PlayabilityVisual qualityLogic coherence+2

6 models tracked

Research

Deep research and synthesis. Source retrieval, citation accuracy, and handling conflicting info.

Source qualitySynthesis depthCitation accuracy+2

5 models tracked

Cross-Cutting

Applied to All

Measured across every category and every model for apples-to-apples comparison.

Speed & Latency

Time to first output and time to completion

Cost Efficiency

Output quality per dollar spent

Consistency

Same prompt 3x — how much do results vary?

Instruction Following

Does the output match prompt constraints?

Playable Reference — Text-to-Video-Game

Ship Hand

Our hand-built benchmark anchor: create a crew member (seven species, three uniform colors) and free-roam a room-by-room Enterprise-D with a living crew, collectible items, procedural music, and full sound effects.

Play it

Community Polling

Coming Soon

Wabbit-style blind polls — community members see outputs side-by-side without model names and vote on which is better. Elo-style ranking from votes.

Which is better?

Model A

Vote

Model B

Vote

Monthly Cadence

Every major model. Same prompts. First week of every month. Trends tracked over time.

Run Benchmarks

All models tested with identical prompts across all five categories.

Community Polls

Blind side-by-side comparisons. Community votes on which output wins.

Publish Results

Scores, rankings, and the lobster test results — all published openly.

Track Trends

Month-over-month and model-over-model trends. See who's improving.