Multiple choice

Classify a question into one or several labels from a fixed set, optionally with media subjects shown above the options.

Multiple-choice tasks present a question and a fixed list of text options. Annotators select one (single mode) or several (multi mode) of those options.

Each datapoint can optionally carry media subjects — one to four already-uploaded videos or images (audio is also supported) of the same type, rendered above the options, so annotators classify what they see rather than only what they read. The options themselves are always plain text.

When to use it

Tagging / categorical labeling
Classification tasks with a known label space (e.g. "is this claim: True / False / Unverifiable")
Multi-label tagging (e.g. "which of these attributes apply?")
Classifying media clips (e.g. "which action is the person performing?") by attaching them as subjects

Datapoint shape

A text-only datapoint carries just the question and the options:

{
  "context": "Which of these best describes the video's motion quality?",
  "media": {
    "options": [
      { "id": "smooth", "text": "Smooth and natural" },
      { "id": "slight_jitter", "text": "Slight jitter, but watchable" },
      { "id": "heavy_jitter", "text": "Heavy jitter or artifacts" },
      { "id": "static", "text": "Appears static or broken" }
    ]
  }
}

datapoint

Field	Type	Description
`context`*	`string`	The question text shown to the annotator. Standalone multiple-choice datapoints must have a non-empty context. In a chain job the step's instruction supplies the question and `context` is optional.
`media.options`*	`array of { id, text }`	At least 2 options. Each item has a stable string `id` (used in results) and visible `text`. Both are required and non-empty.
`media.subjects`	`array of { url \| media_id, type? }`	Optional. 1 to 4 already-uploaded media items of the same type, rendered above the options. Reference each by a dp:// url or media_id — not a public https URL (see below). `subject` (singular) is accepted as a legacy alias for the same list.

Option IDs must be unique within a datapoint and can't have leading or trailing whitespace. In multi mode they also can't contain commas; single mode allows them. The only keys media accepts are options and subjects (or its legacy alias subject, never both); any other key rejects with 400.

Adding media subjects

To show clips or images above the options, upload them first with POST /media, then reference the returned handles from the datapoint's media.subjects. This is a two-step flow:

Upload each subject via POST /media — you get back a dp:// reference and a media_id.
Put a subjects array (1 to 4 items) on the datapoint, referencing each media item by media_id (or its dp:// url).

{
  "context": "Which action appears in both clips?",
  "media": {
    "subjects": [
      { "media_id": "b6a1cd3f-1234-4a2b-9c8d-0e1f2a3b4c5d" },
      { "media_id": "7a8b9c0d-dddd-4eee-9fff-2a3b4c5d6e7f" }
    ],
    "options": [
      { "id": "lifting", "text": "Lifting" },
      { "id": "pushing", "text": "Pushing" },
      { "id": "carrying", "text": "Carrying" }
    ]
  }
}

media.subjects[]

Field	Type	Description
`url`	`string`	A dp:// reference returned by POST /media. Provide either url or media_id.
`media_id`	`string`	The id of an item returned by POST /media, as an alternative to a dp:// url.
`type`	`"video" \| "image" \| "audio"`	Optional hint. The served modality is taken from the uploaded media item itself; the hint is what the create-time same-type check reads, so omit it or make sure it matches the upload.
`label`	`string`	Optional caption shown above this subject (e.g. "A", "B") so options can refer to it, and echoed as `label` on the matching `media[]` entry in results. Must be non-empty and unique within the datapoint. No label is applied by default.

To let options refer to a specific subject — for example an A / B / "no difference" question over two clips — give each subject a label and name it in the option text:

{
  "context": "Which clip looks better, or are they the same?",
  "media": {
    "subjects": [
      { "media_id": "b6a1cd3f-1234-4a2b-9c8d-0e1f2a3b4c5d", "label": "A" },
      { "media_id": "7a8b9c0d-dddd-4eee-9fff-2a3b4c5d6e7f", "label": "B" }
    ],
    "options": [
      { "id": "a", "text": "Video A" },
      { "id": "b", "text": "Video B" },
      { "id": "same", "text": "No difference" }
    ]
  }
}

Subjects are always shown in the order you submit them (only options shuffle), and each result's media[] entry carries the subject's label, so a vote for a maps back to the clip labelled A.

All subjects in a datapoint must share one media type (all videos, all images, or all audio); mixing types rejects with 400. Subjects do not change the submission format or response_options (single/multi, min_selections/max_selections, shuffle); the only result-side difference is that the media array is populated, one entry per subject.

Annotators see every subject stacked above the options. For video subjects, the options stay locked until each clip has been watched; image and audio subjects are static. Video is the most heavily exercised path today.

Mode configuration

Multiple-choice jobs require a response_options.mode:

{
  "task_type": "multiple_choice",
  "response_options": {
    "mode": "single"
  }
}

For multi-select:

{
  "response_options": {
    "mode": "multi",
    "min_selections": 1,
    "max_selections": 3
  }
}

response_options

Field	Type	Description
`mode`*	`"single" \| "multi"`	single: annotator picks one option. multi: annotator picks one or more.
`min_selections`	`int`	multi mode only. Minimum options the annotator must pick. Defaults to 1.
`max_selections`	`int`	multi mode only. Cap on how many options can be picked. Must be ≥ min_selections and ≤ option count.
`shuffle`	`bool` default: true	Randomize option order per annotator to reduce position bias. Set to false when option text references position or is an ordered scale.

Result shape

Same envelope for text-only and subject-bearing datapoints, and for both single and multi. votes is keyed by your option id:

{
  "datapoint_index": 0,
  "votes": {
    "smooth": 6,
    "slight_jitter": 3,
    "heavy_jitter": 1,
    "static": 0
  },
  "total_responses": 10,
  "consensus": "smooth",
  "confidence": 0.6,
  "weighted_votes": {
    "smooth": 5.2,
    "slight_jitter": 2.7,
    "heavy_jitter": 0.9,
    "static": 0
  },
  "weighted_consensus": "smooth",
  "weighted_confidence": 0.591,
  "media": [
    {
      "media_id": "b6a1cd3f-1234-4a2b-9c8d-0e1f2a3b4c5d",
      "type": "video",
      "role": "subject",
      "label": null,
      "url": "/media/v2/b6a1cd3f-1234-4a2b-9c8d-0e1f2a3b4c5d?sig=...",
      "thumbnail_url": "/media/v2/b6a1cd3f-1234-4a2b-9c8d-0e1f2a3b4c5d?preview=1&sig=..."
    }
  ]
}

Field	Type	Description
`votes`	`{ [option_id: string]: count }`	In single mode, counts sum to total_responses. In multi mode, counts can exceed total_responses since each annotator may pick several options.
`total_responses`	`int`	Number of annotator sessions that produced at least one valid selection.
`consensus`	`string \| "tie" \| null`	The option with the highest vote count. 'tie' when 2+ options are tied for the lead; null before any responses.
`confidence`	`float \| null`	top_option_votes / total_responses, rounded to 3 decimals; null before any responses.
`weighted_votes`	`{ [option_id: string]: float }`	Selection totals where each selection contributes its annotator's platform-computed reliability score instead of 1. Rounded to 3 decimals.
`weighted_consensus`	`string \| "tie" \| null`	Option with the highest weighted vote total; 'tie' on a weighted tie; null when no eligible responses.
`weighted_confidence`	`float \| null`	Top option's weighted votes divided by the total response weight — the sum of the reliability scores of the responding annotators, each counted once. In multi mode that denominator is NOT the sum of weighted_votes (an annotator picking several options adds their weight to each), so weighted_confidence can sit well below the naive ratio. Rounded to 3 decimals; null when no eligible responses.
`media`	`array`	The datapoint's media items — one entry per subject, in submission order (empty for text-only datapoints). See the fields below.

When subjects are present, each entry in the media array describes one of them:

media[]

Field	Type	Description
`media_id`	`string`	Stable identifier for the media item.
`type`	`"video" \| "image" \| "audio"`	The subject's media type.
`role`	`"subject"`	Where the item sits on the datapoint. For multiple choice this is always `subject` (the classified media shown above the options), whether the request used `subjects` or the legacy `subject` key.
`label`	`string \| null`	The subject's `label` from the request, or null when none was set.
`url`	`string`	Signed, relative proxy URL for the media; valid for roughly one hour.
`thumbnail_url`	`string`	Signed, relative preview URL; valid for roughly one hour.

Translating option ids to text

By default the aggregation keys and consensus use your stable option ids. Pass labels=true to have the API translate them to the visible option text instead:

GET /jobs/{id}/results?labels=true

With labels=true (default false), the keys of votes / weighted_votes and the consensus / weighted_consensus values are rendered as option text, and each result row also echoes the job's response_options. Leave it off (false) to keep stable ids for machine consumers. The /results per_page query parameter has no upper bound, so you can pull an entire job in one page.

Example (single)

curl -X POST https://api.trydatapoint.com/data-labelling/v1/jobs \
  -H "X-API-Key: $DATAPOINT_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "motion-labels-v1",
    "instruction": "Pick the single best description.",
    "task_type": "multiple_choice",
    "response_options": { "mode": "single" },
    "max_responses_per_datapoint": 10,
    "datapoints": [
      {
        "context": "Which of these best describes the motion?",
        "media": {
          "options": [
            { "id": "smooth", "text": "Smooth" },
            { "id": "jittery", "text": "Jittery" },
            { "id": "static", "text": "Static" }
          ]
        }
      }
    ]
  }'

Example (media subjects)

Upload the clips with POST /media first, then reference their media_ids:

curl -X POST https://api.trydatapoint.com/data-labelling/v1/jobs \
  -H "X-API-Key: $DATAPOINT_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "action-classify-v1",
    "instruction": "Watch the clips, then classify the action.",
    "task_type": "multiple_choice",
    "response_options": { "mode": "single" },
    "max_responses_per_datapoint": 10,
    "datapoints": [
      {
        "context": "Which action appears in both clips?",
        "media": {
          "subjects": [
            { "media_id": "b6a1cd3f-1234-4a2b-9c8d-0e1f2a3b4c5d" },
            { "media_id": "7a8b9c0d-dddd-4eee-9fff-2a3b4c5d6e7f" }
          ],
          "options": [
            { "id": "lifting", "text": "Lifting" },
            { "id": "pushing", "text": "Pushing" },
            { "id": "carrying", "text": "Carrying" }
          ]
        }
      }
    ]
  }'

Example (multi)

{
  "task_type": "multiple_choice",
  "response_options": {
    "mode": "multi",
    "min_selections": 1,
    "max_selections": 2
  },
  "datapoints": [
    {
      "context": "Which attributes apply? (pick up to 2)",
      "media": {
        "options": [
          { "id": "photoreal", "text": "Photorealistic" },
          { "id": "coherent",  "text": "Motion-coherent" },
          { "id": "on_prompt", "text": "Follows the prompt" },
          { "id": "artistic",  "text": "Stylized / artistic" }
        ]
      }
    }
  ]
}

Example (fixed order)

Disable shuffling so every annotator sees the same order:

{
  "task_type": "multiple_choice",
  "response_options": { "mode": "single", "shuffle": false },
  "datapoints": [
    {
      "context": "How often do you shop online?",
      "media": {
        "options": [
          { "id": "always",    "text": "Always" },
          { "id": "often",     "text": "Often" },
          { "id": "sometimes", "text": "Sometimes" },
          { "id": "rarely",    "text": "Rarely" },
          { "id": "never",     "text": "Never" }
        ]
      }
    }
  ]
}

Common errors

Status	Cause
`400`	Missing `response_options.mode`, or mode not `single`/`multi`.
`400`	Datapoint has no `context`, or `media.options` has fewer than 2 entries.
`400`	`media` contains a key other than `options` or `subjects` (or the legacy `subject`), or both `subjects` and `subject` are present.
`400`	`media.subjects` has 5+ items (1 to 4 allowed), or the items are not all the same media type.
`400`	A subject `label` is empty, not a string, or repeated within the datapoint.
`400`	Option IDs are duplicated within a datapoint.
`400`	An option ID has leading or trailing whitespace.
`400`	In `multi` mode, an option ID contains a comma.
`400`	`max_selections` exceeds the number of options in the datapoint.
`422`	`max_responses_per_datapoint` outside range, or a required top-level field is missing or malformed.

Rating: score a media or text subject on a numeric scale, optionally against reference media
Comparison: when the answer is one of two media items, not a label
API Reference: Jobs: full endpoint reference

← PreviousRanking Next →Text answer