Definition

Annotator bias

Annotator bias is the systematic skew introduced into labeled or preference data by the particular people who produced it: their demographics, culture, expertise, and incentives. Models trained on biased preferences inherit them, which makes the composition of the annotator pool as consequential as the size of the dataset.

Professional-annotator drift

Full-time labelers become experts at labeling, and that is precisely the problem: they internalize rubrics, develop shared conventions, learn what reviewers reward, and grow numb to outputs that would delight or repel a normal person. Their judgments drift from the end-user taste the model is supposed to learn. The workforce is also demographically narrow, concentrated in a few labor markets, so "human preference" quietly becomes "the preference of a specific labor pool".

Panel bias

Research panels have their own version: members self-selected into survey-taking years ago, are rewarded for volume, and are sampled repeatedly. Studies repeatedly find professional survey-takers answering faster, straight-lining more, and differing measurably from fresh respondents.

Mitigations

No pool is bias-free, but the levers are known: recruit fresh respondents rather than career labelers, sample broadly across geography and demographics, target the population the model actually serves, rotate the pool continuously, and report who judged alongside what they judged. Ad-network recruitment, which pulls in everyday consumers at the moment of the study, is the structural alternative to standing workforces, trading rubric consistency for representativeness.