# Skills

> Create your own skill from labelled examples and require raters to pass it before their votes count.

A skill is a set of questions with known answers. A job or stream that requires a skill gives each rater a short assessment from that set first, and only counts votes from raters whose score is above your threshold.

You can require a shared skill, or create your own from examples you have labelled yourself. Your own skills are private to your account. For how modes, scores, and qualification behave, see [Skills and qualification](/docs/skills).

Every endpoint below is under `https://api.trydatapoint.com/data-labelling/v1`, takes your API key in the `X-API-Key` header and counts against your key's [rate limit](/docs/rate-limits).

## Create a skill in three calls

1. `POST /skills` to name the skill. You get back its `skill` identifier, which looks like `tenant:<account id>:<name>`.
2. `POST /media` to [upload](/docs/api/media) any images, videos or audio the examples use.
3. `POST /skills/{skill}/calibration-sets` with 20 to 500 labelled examples.

The set is checked automatically and a `draft` skill becomes `active` in the same call. There is no manual review. You can then name the skill in `skill_requirements` on a [job](#require-a-skill-on-a-job) or a [stream](/docs/api/streams#skill-requirements).

<CodeTabs>
  <CodeTab lang="python" label="Python">

```python
import requests

API = "https://api.trydatapoint.com/data-labelling/v1"
HEADERS = {"X-API-Key": "dp_..."}

skill = requests.post(
    f"{API}/skills",
    headers=HEADERS,
    json={"name": "Artifact spotting", "description": "Finds the image with a rendering artifact."},
).json()["skill"]

# labelled: [(first_ref, second_ref, "image_1" or "image_2"), ...], refs from POST /media.
examples = [
    {
        "media": {
            "candidates": [
                {"url": first, "type": "image"},
                {"url": second, "type": "image"},
            ]
        },
        "gold_submission": {"choice": answer},
        "provenance": {"source": "Labelled in house", "license": "client-owned"},
    }
    for first, second, answer in labelled
]

created = requests.post(
    f"{API}/skills/{skill}/calibration-sets",
    headers=HEADERS,
    json={
        "name": "Artifacts v1",
        "task_type": "comparison",
        "instruction": "Which image has a visible artifact?",
        "examples": examples,
    },
)
created.raise_for_status()
```

  </CodeTab>
</CodeTabs>

## Create a skill

<EndpointHeader method="POST" path="/skills" />

### Request body

<ParamTable
  params={[
    { name: "name", type: "string (1–120)", required: true, description: "Display name. Must be unique among your skills; a duplicate is a `409`." },
    { name: "description", type: "string (up to 2,000)", required: false, description: "What the skill checks." }
  ]}
/>

### Response (`201 Created`)

The [skill object](#the-skill-object), with `status: "draft"`. A draft skill can't be required yet; it becomes `active` when its first set is accepted.

## Add labelled examples

<EndpointHeader method="POST" path="/skills/{skill}/calibration-sets" />

Adds a set of examples to one of your skills. You can add more sets later; raters are assessed on examples drawn from all of the skill's active sets that can serve in the job's environment. Each rater sees each example at most once, so larger sets support more re-checks.

### Request body

<ParamTable
  params={[
    { name: "name", type: "string (1–120)", required: true, description: "Name of this set, unique among the skill's sets." },
    { name: "task_type", type: '"comparison" | "multiple_choice"', required: true, description: "How each example is asked. Other task types aren't supported for skills." },
    { name: "instruction", type: "string (1–5,000)", required: true, description: "The question raters see on every example." },
    { name: "response_options", type: "object", required: false, description: "As on `POST /jobs`. Required for `multiple_choice` (`mode`)." },
    { name: "serving_environment", type: '"all" | "sandbox" | "prod"', required: false, description: "Where the examples can be served. Defaults to the environment of the API you call (`prod` on the production API). A `prod` job can use `prod` and `all` sets, a `sandbox` job `sandbox` and `all` sets, and an `all` job only `all` sets. Use `all` to try the skill in sandbox before going live." },
    { name: "examples", type: "array (20–500 items)", required: true, description: "The labelled examples. See below." }
  ]}
/>

<ParamTable
  title="examples[]"
  params={[
    { name: "media", type: "object", required: true, description: "The same shape as a datapoint's `media` for the task type: `candidates` for comparison, `options` and optional `subjects` for multiple choice." },
    { name: "context", type: "string (up to 20,000)", required: false, description: "Required for `multiple_choice`, where it is the question. For `comparison`, per-example text shown where the instruction has `{context}`." },
    { name: "gold_submission", type: "object", required: true, description: "The correct answer. See [Gold answers](#gold-answers)." },
    { name: "provenance", type: "object", required: true, description: "Where the example came from: `source` (required), `license` (required) and an optional `uri`. For your own data, `\"client-owned\"` is a fine `license`." }
  ]}
/>

<DocCallout type="warning" title="Upload media first">
Media in an example must be a `dp://` reference in `url`, or a `media_id`, from your own [`POST /media`](/docs/api/media) uploads, and it must have passed content review: `{ "url": "dp://..." }` or `{ "media_id": "..." }`. A public `https://` URL is a `400`.
</DocCallout>

The `media` shapes are on the [comparison](/docs/task-types/comparison) and [multiple choice](/docs/task-types/multiple-choice) pages.

### Gold answers

| Task type | `gold_submission` | Notes |
|-----------|-------------------|-------|
| `comparison` | `{"choice": "image_1"}` | The first entry in `candidates` is `image_1`, `video_1` or `audio_1` by its uploaded media type; the second ends in `_2`. Raters may see them in either order. Any `id` or `type` you set on a candidate is ignored. |
| `multiple_choice`, `mode: "single"` | `{"choice": "<option id>"}` | An `id` from `media.options`. |
| `multiple_choice`, `mode: "multi"` | `{"choices": ["<option id>", ...]}` | Every correct option. |

For multiple choice, vary which option is correct across the set: if one option is right for most examples, a rater who always picks it scores well without the skill. Each example's media and `context` must be distinct from every other example's.

A multiple-choice example with an image:

```json
{
  "context": "What is wrong with this image?",
  "media": {
    "subjects": [{ "url": "dp://b6a1cd3f1234/sample_017.png", "type": "image" }],
    "options": [
      { "id": "hands", "text": "Malformed hands" },
      { "id": "text", "text": "Garbled text" },
      { "id": "none", "text": "Nothing" }
    ]
  },
  "gold_submission": { "choice": "hands" },
  "provenance": { "source": "Labelled in house", "license": "client-owned" }
}
```

### Response (`201 Created`)

```json
{
  "set_id": "0b9f6c0e-3a7d-4f0e-9d1b-2f5f7f0c8a11",
  "skill": "tenant:7c1e0d52-9b3a-4c1f-8e21-5a4d1f0b6c33:artifact-spotting",
  "name": "Artifacts v1",
  "status": "passed",
  "active": true,
  "item_count": 20,
  "submitted_for_review": false
}
```

### Error cases

| Status | When |
|--------|------|
| `400` | The skill isn't one of yours or is retired; the set's name is already used on the skill; an example fails the task type's validation, its answer isn't one of its candidates or options, or its media isn't an uploaded reference that passed content review; or examples repeat. The `detail` says which. |
| `422` | The body doesn't match the schema, such as fewer than 20 examples or an unknown field, or its text didn't pass content review (`detail.code` is `content_blocked`). |
| `503` | Skill checks or content review are temporarily unavailable. Nothing was stored; retry later, after `Retry-After` when it is sent. |

## List skills

<EndpointHeader method="GET" path="/skills" />

Returns `{ "skills": [...] }`: every shared skill plus the ones your account created.

### The skill object

```json
{
  "skill": "tenant:7c1e0d52-9b3a-4c1f-8e21-5a4d1f0b6c33:artifact-spotting",
  "name": "Artifact spotting",
  "description": "Finds the image with a rendering artifact.",
  "scope": "client",
  "status": "active",
  "calibration_sets": 1,
  "validated_sets": 1,
  "active_items": 20,
  "created_at": "2026-11-12T10:15:00+00:00"
}
```

<ParamTable
  params={[
    { name: "skill", type: "string", description: "The identifier to pass in `skill_requirements`. It never changes, even if you rename the skill." },
    { name: "scope", type: '"platform" | "client"', description: "`platform` is a shared skill, free to require. `client` is one your account created." },
    { name: "status", type: '"draft" | "active" | "paused" | "retired"', description: "Only an `active` skill can be required." },
    { name: "active_items", type: "int", description: "Examples raters can currently be assessed on." }
  ]}
/>

## Update, pause or retire a skill

<EndpointHeader method="PATCH" path="/skills/{skill}" />

Takes any of `name`, `description` and `status` (`"active"`, `"paused"` or `"retired"`), and returns the skill object. Setting `active` needs at least one accepted set, otherwise it is a `400`. A paused skill stays paused when you add sets to it. A retired skill takes no new sets. A skill that isn't yours is a `404`.

Pausing or retiring a skill stops every job and stream that requires it from serving new work, including to raters who already qualified, until you set it back to `active`. Those jobs stay active and report no error.

## Require a skill on a job

Add these to the [`POST /jobs`](/docs/api/jobs) body:

<ParamTable
  params={[
    { name: "skill_requirements", type: "array (1–10 items)", required: false, description: "Each item has a `skill` identifier and `reject_at_or_below`, from 0 up to but not including 1. A vote counts only when the rater's score on every listed skill is strictly above its threshold." },
    { name: "skill_gate_mode", type: '"adaptive" | "enforced"', required: false, description: "`adaptive` (the default) reuses a rater's existing score and rechecks it from time to time. `enforced` assesses every rater on each required skill for this job." },
    { name: "skill_budget_credits", type: "int", required: false, description: "The most the job may spend on assessments of your own skills, reserved on top of the normal responses. Required when a listed skill is one you created and the job is charged, and must cover at least one assessment per such skill. Omit it, or send `0`, for shared skills and sandbox jobs; a non-zero budget there is a `400`." }
  ]}
/>

```json
{
  "name": "artifact-eval-run-3",
  "task_type": "comparison",
  "instruction": "Which image looks more realistic?",
  "datapoints": [...],
  "skill_requirements": [
    { "skill": "tenant:7c1e0d52-9b3a-4c1f-8e21-5a4d1f0b6c33:artifact-spotting", "reject_at_or_below": 0.6 }
  ],
  "skill_budget_credits": 5000
}
```

A rater's score on a skill runs from 0 to 1, reflects their recent graded answers on it, and is shared by every job that requires the skill. In `adaptive` mode, a rater who falls below your threshold can be assessed again later. In `enforced` mode, each rater is assessed on every required skill for the job, and a rater who doesn't qualify isn't assessed again for that job.

Votes from raters who don't pass are left out of results and billing. A datapoint keeps being served until enough votes count, up to a serving limit, so if many raters fail it can close with fewer than `max_responses_per_datapoint`. An unknown, inactive or empty skill is a `400` naming it, and while skill checks are temporarily unavailable a job with skills is a `503`.

To raise the assessment budget on a running job, `POST /jobs/{job_id}/skill-budget` with `{ "skill_budget_credits": <new total> }`. It can only go up; a `402` means your balance doesn't cover the increase. See [Billing](/docs/billing#increase-a-jobs-skill-budget).

<EndpointHeader method="GET" path="/jobs/{job_id}/skill-quality" />

Returns `job_id`, `enabled`, `mode`, and a `requirements` array. Each requirement has the `skill`, its `reject_at_or_below`, and counts of `completed_assessments`, `billed_assessments`, `passed_assessments` and `failed_assessments`, plus `skill_cost_credits`. The passed and failed counts record enforced-mode threshold checks, not answer accuracy, so they stay at 0 on adaptive jobs; use the performance report below for accuracy.

## What assessments cost

- Shared skills are free.
- Each answered assessment of a skill you created is charged once at your account's assessment rate, whether the rater passes or not. The rate is fixed when the job or stream is created; `GET /billing/pricing` returns your current rate.
- In `adaptive` mode, a rater who already has a passing score usually starts real work without a new assessment, but is re-checked from time to time, and each re-check is charged. In `enforced` mode, every rater is assessed on every job.
- Unanswered assessments are free, and so are answers that arrive after the assessment expires.
- Sandbox jobs have a rate of 0.
- When the job settles, unused assessment budget is returned.

[Billing](/docs/billing#skill-assessment-pricing) covers budgets and spend reports.

## Check how a skill is doing

<EndpointHeader method="GET" path="/skills/{skill}/quality" />

Returns the skill's `status` and counts, and a `validation` array listing each set with its `set_id`, `status`, whether it is `active`, its `item_count` and `active_items`, and a summary of the automated checks. Works for shared skills too.

<EndpointHeader method="GET" path="/skills/{skill}/performance" />

Reports how raters did on the skill across your jobs: assessments assigned and scored, accuracy, qualification pass rate and credits charged, in total and per set or per question.

<ParamTable
  title="Query parameters"
  params={[
    { name: "window_days", type: "int (1–90)", required: false, description: "Days to report on, ending at `end_at`. Default 30." },
    { name: "end_at", type: "datetime", required: false, description: "End of the window, with a timezone. Defaults to now." },
    { name: "job_id", type: "string", required: false, description: "Only assessments served for this job." },
    { name: "set_id", type: "uuid", required: false, description: "Only this set." },
    { name: "mode", type: '"adaptive" | "enforced"', required: false, description: "Only jobs in this gate mode." },
    { name: "serving_environment", type: '"all" | "sandbox" | "prod"', required: false, description: "Only jobs with this `serving_environment` setting." },
    { name: "group_by", type: '"set" | "question"', required: false, description: "Breakdown level. Default `set`. Use `question` to find examples most raters get wrong, which often means the label or the question is unclear." },
    { name: "page, page_size", type: "int", required: false, description: "Paging over the breakdown; `page_size` up to 100, default 25." }
  ]}
/>

Comparison and multiple-choice answers are graded right or wrong, so `accuracy` is the share answered correctly. The qualification pass rate counts enforced-mode threshold checks only, and is `null` without them; `null` never means 0%.

An unknown skill, job or set is a `404`, an invalid filter or a future `end_at` is a `422`, and a report that takes too long is a `503`; retry with a shorter window or a `job_id` or `set_id` filter.

## Next

- [Skills and qualification](/docs/skills)
- [Streams: skill requirements](/docs/api/streams#skill-requirements)
- [Billing: assessment pricing](/docs/billing#skill-assessment-pricing)
- [Media](/docs/api/media)
- [Task types](/docs/task-types)