TEXT-TO-IMAGE
Which model nails the prompt?
Pairwise battles on prompt alignment and overall preference, ten raters per pair, with results that double as DPO-ready preference pairs.
←Datapoint
For model builders and research teams
The #1 API for evaluating multimodal models
Automated metrics miss what people actually see and hear. Put text-to-image, video, and speech models head to head in front of real people, and get Elo ratings with confidence intervals, per-rater records, and category-level results, usually in under an hour.
What you can do
Arena-grade evaluation for every modality.
Side by side, across modalities
Pairwise battles and rank-5 sets for text-to-image, image-to-video, text-to-video, and TTS. Annotators judge prompt adherence, overall quality, and physical consistency in-app.
Elo with honest error bars
Ratings come with confidence intervals and rank spreads, so you can tell a real lead from noise before you publish a claim.
Slice by category
Photoreal people, typography, anime, product shots, and more. See where a model wins and where it falls apart, backed by 10 to 15 raters per item.
Public leaderboard or private tracks
Publish to the Datapoint leaderboard, or run a private benchmark on your own checkpoints and prompt sets with the same rater pool and quality controls.
What builders evaluate
From checkpoint to leaderboard.
TEXT-TO-VIDEO
Does the motion hold up?
Sets of five generations ranked on quality, prompt adherence, and physical consistency across the leading text-to-video models.
SPEECH
Which voice sounds human?
Pairwise preference over synthesized speech for TTS and voice-cloning models, alongside image-to-video preference series.
Open data
The evaluations behind the leaderboard, in the open.
Our text-to-image, text-to-video, image-to-video, and speech preference datasets are published on Hugging Face with raw per-rater votes under CC-BY-4.0. Recompute the Elo yourself before you trust it.
Frequently
Asked Questions
From pairwise human votes, the same way arena leaderboards work. Each rating ships with a confidence interval and a rank spread, every published board is a snapshot you can cite, and raw per-rater responses let you recompute the result yourself.
It depends on how close the models are. We collect 10 to 15 independent raters per item and report the rank spread, so you can see when two models are statistically tied and keep collecting until the intervals separate.
Yes. Run a private track with your own checkpoints, prompt sets, and categories, judged by the same 9M+ annotator network. Nothing is published unless you choose to.
Pay per vote, with free credits on new accounts. Researchers can apply to the grant program for funded evaluations.
Let people, not proxies, rank your models.
Sign up, take the free credits, run your first head-to-head today.