Text-to-speech · Preference data
Text-to-speech human evaluation
Naturalness, prosody and pronunciation are judged by ear, and automatic MOS predictors rank TTS models differently from listeners. Collect pairwise preferences and MOS ratings from native speakers through one API, with the pipeline behind our 315K-vote Audio Bench dataset.
What teams collect
Judgment tasks that work for text-to-speech
Pairwise naturalness
Two clips of the same text from different models or checkpoints, one vote for the more natural voice. Aggregates into win rates and Elo rankings.
MOS ratings
A 1 to 5 rating per clip for naturalness or overall quality, with every listener’s score returned so you can compute MOS and its confidence interval yourself.
Native-listener panels by locale
Target listeners by country and screen for language, so Hindi is judged in India and Spanish in Mexico or Spain. Screened-out listeners are not billed.
Voice-agent and ad-copy prompts
Test on the text your product reads aloud: support dialogue, numbers and addresses, ad scripts. Rankings change with the prompt set.
Frequently asked
You pay a flat rate per response. On platforms that bill participant hours, a 40-second speech comparison comes to about $190 per 1,000 responses; Datapoint is about a quarter of that. Book a call for a quote.
Pairwise when you are choosing between models or checkpoints: listeners agree more on which clip is better than on an absolute score, so you need fewer votes to separate close systems. MOS when you need an absolute number to report or to track one system over time.
Listeners have to play both clips before they can vote, and playback and response timing are recorded with each vote.
MP3, WAV, OGG, M4A and FLAC, uploaded or as public HTTPS URLs. Listeners get a player control for each clip.
Not for ranking current models. Predicted MOS is useful for catching broken audio, but the top TTS systems differ on prosody and expressiveness, which listeners hear and predictors score poorly.
Pilot a real text-to-speech study today.
Free credits on new accounts, pay-per-response after. No contracts, no annotation ops.