# Text-to-image human preference data

> Collect human preference data on text-to-image model outputs via API: pairwise comparisons, prompt-faithfulness ratings, and aesthetic judgments from real consumers. 2M+ votes already published on Hugging Face.

Pairwise preferences, prompt-faithfulness ratings, and aesthetic judgments on image model outputs, from everyday consumers rather than professional annotators. The same pipeline behind our 2M+ vote public dataset, available through one API.

## Published proof

- datapointai/text-2-image-human-preferences-2m: 2M+ pairwise votes (Open on Hugging Face) at https://huggingface.co/datasets/datapointai/text-2-image-human-preferences-2m

## What teams collect

### Model vs model side-by-sides
Two generations for the same prompt, one vote for the better image. Aggregates into win rates and Elo rankings across model versions, checkpoints, or providers.

### Prompt faithfulness
Does the image actually contain what the prompt asked for? Ratings or A/Bs focused on adherence rather than aesthetics. This is the failure mode automated metrics miss most.

### Aesthetic quality and style
Which image is more beautiful, more on-brand, more like the reference style? Pure taste judgments where representative consumers are the only valid ground truth.

### DPO training pairs
High-agreement chosen/rejected image pairs, filtered by vote threshold, ready for Diffusion-DPO-style fine-tuning or reward model training.

## FAQ

### How is text-to-image preference data collected?
You submit image pairs (or single images with a rating rubric) through the Datapoint API; verified respondents recruited via a global ad network judge them within seconds. Each datapoint collects as many independent votes as you specify, and results return with per-item agreement statistics.

### Can I reproduce the 2M-vote dataset methodology on my own models?
Yes. The public dataset was collected with the same jobs API available to every account: pairwise comparison tasks, 5+ votes per pair, quality gates on every response. Point the same setup at your own model outputs.

### How many votes do I need per image pair?
For leaderboards, 5 votes per pair across many pairs beats many votes on few pairs. For DPO training data, 5–10 votes with a high-agreement filter (e.g. keep pairs where 80%+ agree) is a common recipe.

### Who are the judges?
Everyday consumers across 200+ countries, recruited live through in-app ads and verified at the device level, not career annotators. You can target by geography, age, or interest when preferences should reflect a specific market.

---

All of this runs on the Datapoint human preference data API: https://trydatapoint.com/human-preference-data-api (docs: https://trydatapoint.com/docs).
