Preference data
Human preference data, by modality
The judgments your training and eval pipelines need, from the people your models serve. Pick the modality closest to your work. Every page runs on the same human preference data API.
Text-to-image
Text-to-image human preference data
Pairwise preferences, prompt-faithfulness ratings, and aesthetic judgments on image model outputs, from everyday consumers rather than professional annotators. The same pipeline behind our 2M+ vote public dataset, available through one API.
Proof: 2M+ pairwise votes
Text-to-video
Text-to-video human preference data
Motion quality, temporal consistency, and realism are exactly the qualities automated metrics measure worst, and humans judge instantly. Collect side-by-side video preferences at scale, with the pipeline behind our 145K-judgment VidPrefMotion benchmark.
Proof: 145K human judgments
LLM response evaluation
Human evaluation API for LLM outputs
LLM-as-a-judge is fast and cheap, and quietly biased. Calibrate it against real human preferences: side-by-side votes and ratings on your model’s actual responses, from the population your product serves, back in minutes.
Pairwise comparison
Pairwise comparison data collection
The most reliable subjective judgment humans make is "which of these two is better". Collect it programmatically on any media (model outputs, ad creatives, product designs) with 5–10 independent votes per pair and agreement statistics built in.