Multiple choice

Classify a question into one or several labels from a fixed set, optionally with a media subject shown above the options.

Multiple-choice tasks present a question and a fixed list of text options. Annotators select one (single mode) or several (multi mode) of those options.

Each datapoint can optionally carry a single media subject — one already-uploaded video or image (audio is also supported) rendered above the options, so annotators classify what they see rather than only what they read. The options themselves are always plain text.

When to use it

Tagging / categorical labeling
Classification tasks with a known label space (e.g. "is this claim: True / False / Unverifiable")
Multi-label tagging (e.g. "which of these attributes apply?")
Classifying a media clip (e.g. "which action is the person performing?") by attaching it as the subject

Datapoint shape

A text-only datapoint carries just the question and the options:

{
  "context": "Which of these best describes the video's motion quality?",
  "media": {
    "options": [
      { "id": "smooth", "text": "Smooth and natural" },
      { "id": "slight_jitter", "text": "Slight jitter, but watchable" },
      { "id": "heavy_jitter", "text": "Heavy jitter or artifacts" },
      { "id": "static", "text": "Appears static or broken" }
    ]
  }
}

datapoint

Field	Type	Description
`context`*	`string`	The question text shown to the annotator. Standalone multiple-choice datapoints must have a non-empty context. In a chain job the step's instruction supplies the question and `context` is optional.
`media.options`*	`array of { id, text }`	At least 2 options. Each item has a stable string `id` (used in results) and visible `text`. Both are required and non-empty.
`media.subject`	`array of { url \| media_id, type? }`	Optional. 0 or 1 already-uploaded media item, rendered above the options. Reference it by a dp:// url or media_id — not a public https URL (see below).

Option IDs must be unique within a datapoint and can't have leading or trailing whitespace. In multi mode they also can't contain commas; single mode allows them. The only keys media accepts are options and subject; any other key rejects with 400.

Adding a media subject

To show a clip or image above the options, upload it first with POST /media, then reference the returned handle from the datapoint's media.subject. This is a two-step flow:

Upload the subject via POST /media — you get back a dp:// reference and a media_id.
Put a single-item subject array on the datapoint, referencing that media by media_id (or its dp:// url).

{
  "context": "Which action is the worker performing?",
  "media": {
    "subject": [
      { "media_id": "b6a1cd3f-1234-4a2b-9c8d-0e1f2a3b4c5d" }
    ],
    "options": [
      { "id": "lifting", "text": "Lifting" },
      { "id": "pushing", "text": "Pushing" },
      { "id": "carrying", "text": "Carrying" }
    ]
  }
}

media.subject[]

Field	Type	Description
`url`	`string`	A dp:// reference returned by POST /media. Provide either url or media_id.
`media_id`	`string`	The id of an item returned by POST /media, as an alternative to a dp:// url.
`type`	`"video" \| "image" \| "audio"`	Optional hint; ignored in practice. The subject's modality is taken from the uploaded media item itself.

A subject does not change the submission format or response_options (single/multi, min_selections/max_selections, shuffle); the only result-side difference is that the media array is populated.

The subject can be any uploaded item — video, image, or audio; the modality comes from the uploaded media. Video is the most heavily exercised path today.

Mode configuration

Multiple-choice jobs require a response_options.mode:

{
  "task_type": "multiple_choice",
  "response_options": {
    "mode": "single"
  }
}

For multi-select:

{
  "response_options": {
    "mode": "multi",
    "min_selections": 1,
    "max_selections": 3
  }
}

response_options

Field	Type	Description
`mode`*	`"single" \| "multi"`	single: annotator picks one option. multi: annotator picks one or more.
`min_selections`	`int`	multi mode only. Minimum options the annotator must pick. Defaults to 1.
`max_selections`	`int`	multi mode only. Cap on how many options can be picked. Must be ≥ min_selections and ≤ option count.
`shuffle`	`bool` default: true	Randomize option order per annotator to reduce position bias. Set to false when option text references position or is an ordered scale.

Result shape

Same envelope for text-only and subject-bearing datapoints, and for both single and multi. votes is keyed by your option id:

{
  "datapoint_index": 0,
  "votes": {
    "smooth": 6,
    "slight_jitter": 3,
    "heavy_jitter": 1,
    "static": 0
  },
  "total_responses": 10,
  "consensus": "smooth",
  "confidence": 0.6,
  "weighted_votes": {
    "smooth": 5.2,
    "slight_jitter": 2.7,
    "heavy_jitter": 0.9,
    "static": 0
  },
  "weighted_consensus": "smooth",
  "weighted_confidence": 0.591,
  "media": [
    {
      "media_id": "b6a1cd3f-1234-4a2b-9c8d-0e1f2a3b4c5d",
      "type": "video",
      "role": "subject",
      "label": null,
      "url": "/media/v2/b6a1cd3f-1234-4a2b-9c8d-0e1f2a3b4c5d?sig=...",
      "thumbnail_url": "/media/v2/b6a1cd3f-1234-4a2b-9c8d-0e1f2a3b4c5d?preview=1&sig=..."
    }
  ]
}

Field	Type	Description
`votes`	`{ [option_id: string]: count }`	In single mode, counts sum to total_responses. In multi mode, counts can exceed total_responses since each annotator may pick several options.
`total_responses`	`int`	Number of annotator sessions that produced at least one valid selection.
`consensus`	`string \| "tie" \| null`	The option with the highest vote count. 'tie' when 2+ options are tied for the lead; null before any responses.
`confidence`	`float \| null`	top_option_votes / total_responses, rounded to 3 decimals; null before any responses.
`weighted_votes`	`{ [option_id: string]: float }`	Selection totals where each selection contributes its annotator's platform-computed reliability score instead of 1. Rounded to 3 decimals.
`weighted_consensus`	`string \| "tie" \| null`	Option with the highest weighted vote total; 'tie' on a weighted tie; null when no eligible responses.
`weighted_confidence`	`float \| null`	Top option's weighted votes divided by the total response weight — the sum of the reliability scores of the responding annotators, each counted once. In multi mode that denominator is NOT the sum of weighted_votes (an annotator picking several options adds their weight to each), so weighted_confidence can sit well below the naive ratio. Rounded to 3 decimals; null when no eligible responses.
`media`	`array`	The datapoint's media items — the subject, when one is present (empty for text-only datapoints). See the fields below.

When a subject is present, each entry in the media array describes it:

media[]

Field	Type	Description
`media_id`	`string`	Stable identifier for the media item.
`type`	`"video" \| "image" \| "audio"`	The subject's media type.
`role`	`"subject"`	Where the item sits on the datapoint. For multiple choice this is always `subject` (context shown above the options).
`label`	`null`	Always null — the subject is context, not a voted-on option.
`url`	`string`	Signed, relative proxy URL for the media; valid for roughly one hour.
`thumbnail_url`	`string`	Signed, relative preview URL; valid for roughly one hour.

Translating option ids to text

By default the aggregation keys and consensus use your stable option ids. Pass labels=true to have the API translate them to the visible option text instead:

GET /jobs/{id}/results?labels=true

With labels=true (default false), the keys of votes / weighted_votes and the consensus / weighted_consensus values are rendered as option text, and each result row also echoes the job's response_options. Leave it off (false) to keep stable ids for machine consumers. The /results per_page query parameter has no upper bound, so you can pull an entire job in one page.

Example (single)

curl -X POST https://api.trydatapoint.com/data-labelling/v1/jobs \
  -H "X-API-Key: $DATAPOINT_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "motion-labels-v1",
    "instruction": "Pick the single best description.",
    "task_type": "multiple_choice",
    "response_options": { "mode": "single" },
    "max_responses_per_datapoint": 10,
    "datapoints": [
      {
        "context": "Which of these best describes the motion?",
        "media": {
          "options": [
            { "id": "smooth", "text": "Smooth" },
            { "id": "jittery", "text": "Jittery" },
            { "id": "static", "text": "Static" }
          ]
        }
      }
    ]
  }'

Example (media subject)

Upload the clip with POST /media first, then reference its media_id:

curl -X POST https://api.trydatapoint.com/data-labelling/v1/jobs \
  -H "X-API-Key: $DATAPOINT_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "action-classify-v1",
    "instruction": "Watch the clip, then classify the action.",
    "task_type": "multiple_choice",
    "response_options": { "mode": "single" },
    "max_responses_per_datapoint": 10,
    "datapoints": [
      {
        "context": "Which action is the worker performing?",
        "media": {
          "subject": [
            { "media_id": "b6a1cd3f-1234-4a2b-9c8d-0e1f2a3b4c5d" }
          ],
          "options": [
            { "id": "lifting", "text": "Lifting" },
            { "id": "pushing", "text": "Pushing" },
            { "id": "carrying", "text": "Carrying" }
          ]
        }
      }
    ]
  }'

Example (multi)

{
  "task_type": "multiple_choice",
  "response_options": {
    "mode": "multi",
    "min_selections": 1,
    "max_selections": 2
  },
  "datapoints": [
    {
      "context": "Which attributes apply? (pick up to 2)",
      "media": {
        "options": [
          { "id": "photoreal", "text": "Photorealistic" },
          { "id": "coherent",  "text": "Motion-coherent" },
          { "id": "on_prompt", "text": "Follows the prompt" },
          { "id": "artistic",  "text": "Stylized / artistic" }
        ]
      }
    }
  ]
}

Example (fixed order)

Disable shuffling so every annotator sees the same order:

{
  "task_type": "multiple_choice",
  "response_options": { "mode": "single", "shuffle": false },
  "datapoints": [
    {
      "context": "How often do you shop online?",
      "media": {
        "options": [
          { "id": "always",    "text": "Always" },
          { "id": "often",     "text": "Often" },
          { "id": "sometimes", "text": "Sometimes" },
          { "id": "rarely",    "text": "Rarely" },
          { "id": "never",     "text": "Never" }
        ]
      }
    }
  ]
}

Common errors

Status	Cause
`400`	Missing `response_options.mode`, or mode not `single`/`multi`.
`400`	Datapoint has no `context`, or `media.options` has fewer than 2 entries.
`400`	`media` contains a key other than `options` or `subject`.
`400`	`media.subject` has 2+ items (0 or 1 allowed).
`400`	Option IDs are duplicated within a datapoint.
`400`	An option ID has leading or trailing whitespace.
`400`	In `multi` mode, an option ID contains a comma.
`400`	`max_selections` exceeds the number of options in the datapoint.
`422`	`max_responses_per_datapoint` outside range, or a required top-level field is missing or malformed.

Rating: score a media or text subject on a numeric scale, optionally against reference media
Comparison: when the answer is one of two media items, not a label
API Reference: Jobs: full endpoint reference

← PreviousRanking Next →Text answer