Skills

A skill is a set of questions with known answers. A job or stream that requires a skill gives each rater a short assessment from that set first, and only counts votes from raters whose score is above your threshold.

You can require a shared skill, or create your own from examples you have labelled yourself. Your own skills are private to your account. For how modes, scores, and qualification behave, see Skills and qualification.

Every endpoint below is under https://api.trydatapoint.com/data-labelling/v1, takes your API key in the X-API-Key header and counts against your key's rate limit.

Create a skill in three calls

1. POST /skills to name the skill. You get back its skill identifier, which looks like tenant:<account id>:<name>.
2. POST /media to upload any images, videos or audio the examples use.
3. POST /skills/{skill}/calibration-sets with 20 to 500 labelled examples.

The set is checked automatically and a draft skill becomes active in the same call. There is no manual review. You can then name the skill in skill_requirements on a job or a stream.

import requests
 
API = "https://api.trydatapoint.com/data-labelling/v1"
HEADERS = {"X-API-Key": "dp_..."}
 
skill = requests.post(
    f"{API}/skills",
    headers=HEADERS,
    json={"name": "Artifact spotting", "description": "Finds the image with a rendering artifact."},
).json()["skill"]
 
# labelled: [(first_ref, second_ref, "image_1" or "image_2"), ...], refs from POST /media.
examples = [
    {
        "media": {
            "candidates": [
                {"url": first, "type": "image"},
                {"url": second, "type": "image"},
            ]
        },
        "gold_submission": {"choice": answer},
        "provenance": {"source": "Labelled in house", "license": "client-owned"},
    }
    for first, second, answer in labelled
]
 
created = requests.post(
    f"{API}/skills/{skill}/calibration-sets",
    headers=HEADERS,
    json={
        "name": "Artifacts v1",
        "task_type": "comparison",
        "instruction": "Which image has a visible artifact?",
        "examples": examples,
    },
)
created.raise_for_status()

Create a skill

Request body

Response (201 Created)

The skill object, with status: "draft". A draft skill can't be required yet; it becomes active when its first set is accepted.

Add labelled examples

Adds a set of examples to one of your skills. You can add more sets later; raters are assessed on examples drawn from all of the skill's active sets that can serve in the job's environment. Each rater sees each example at most once, so larger sets support more re-checks.

Request body

The media shapes are on the comparison and multiple choice pages.

Gold answers

For multiple choice, vary which option is correct across the set: if one option is right for most examples, a rater who always picks it scores well without the skill. Each example's media and context must be distinct from every other example's.

A multiple-choice example with an image:

{
  "context": "What is wrong with this image?",
  "media": {
    "subjects": [{ "url": "dp://b6a1cd3f1234/sample_017.png", "type": "image" }],
    "options": [
      { "id": "hands", "text": "Malformed hands" },
      { "id": "text", "text": "Garbled text" },
      { "id": "none", "text": "Nothing" }
    ]
  },
  "gold_submission": { "choice": "hands" },
  "provenance": { "source": "Labelled in house", "license": "client-owned" }
}

Response (201 Created)

{
  "set_id": "0b9f6c0e-3a7d-4f0e-9d1b-2f5f7f0c8a11",
  "skill": "tenant:7c1e0d52-9b3a-4c1f-8e21-5a4d1f0b6c33:artifact-spotting",
  "name": "Artifacts v1",
  "status": "passed",
  "active": true,
  "item_count": 20,
  "submitted_for_review": false
}

Error cases

List skills

Returns { "skills": [...] }: every shared skill plus the ones your account created.

The skill object

{
  "skill": "tenant:7c1e0d52-9b3a-4c1f-8e21-5a4d1f0b6c33:artifact-spotting",
  "name": "Artifact spotting",
  "description": "Finds the image with a rendering artifact.",
  "scope": "client",
  "status": "active",
  "calibration_sets": 1,
  "validated_sets": 1,
  "active_items": 20,
  "created_at": "2026-11-12T10:15:00+00:00"
}

Update, pause or retire a skill

Takes any of name, description and status ("active", "paused" or "retired"), and returns the skill object. Setting active needs at least one accepted set, otherwise it is a 400. A paused skill stays paused when you add sets to it. A retired skill takes no new sets. A skill that isn't yours is a 404.

Pausing or retiring a skill stops every job and stream that requires it from serving new work, including to raters who already qualified, until you set it back to active. Those jobs stay active and report no error.

Require a skill on a job

Add these to the POST /jobs body:

{
  "name": "artifact-eval-run-3",
  "task_type": "comparison",
  "instruction": "Which image looks more realistic?",
  "datapoints": [...],
  "skill_requirements": [
    { "skill": "tenant:7c1e0d52-9b3a-4c1f-8e21-5a4d1f0b6c33:artifact-spotting", "reject_at_or_below": 0.6 }
  ],
  "skill_budget_credits": 5000
}

A rater's score on a skill runs from 0 to 1, reflects their recent graded answers on it, and is shared by every job that requires the skill. In adaptive mode, a rater who falls below your threshold can be assessed again later. In enforced mode, each rater is assessed on every required skill for the job, and a rater who doesn't qualify isn't assessed again for that job.

Votes from raters who don't pass are left out of results and billing. A datapoint keeps being served until enough votes count, up to a serving limit, so if many raters fail it can close with fewer than max_responses_per_datapoint. An unknown, inactive or empty skill is a 400 naming it, and while skill checks are temporarily unavailable a job with skills is a 503.

To raise the assessment budget on a running job, POST /jobs/{job_id}/skill-budget with { "skill_budget_credits": <new total> }. It can only go up; a 402 means your balance doesn't cover the increase. See Billing.

Returns job_id, enabled, mode, and a requirements array. Each requirement has the skill, its reject_at_or_below, and counts of completed_assessments, billed_assessments, passed_assessments and failed_assessments, plus skill_cost_credits. The passed and failed counts record enforced-mode threshold checks, not answer accuracy, so they stay at 0 on adaptive jobs; use the performance report below for accuracy.

What assessments cost

• Shared skills are free.
• Each answered assessment of a skill you created is charged once at your account's assessment rate, whether the rater passes or not. The rate is fixed when the job or stream is created; GET /billing/pricing returns your current rate.
• In adaptive mode, a rater who already has a passing score usually starts real work without a new assessment, but is re-checked from time to time, and each re-check is charged. In enforced mode, every rater is assessed on every job.
• Unanswered assessments are free, and so are answers that arrive after the assessment expires.
• Sandbox jobs have a rate of 0.
• When the job settles, unused assessment budget is returned.

Billing covers budgets and spend reports.

Check how a skill is doing

Returns the skill's status and counts, and a validation array listing each set with its set_id, status, whether it is active, its item_count and active_items, and a summary of the automated checks. Works for shared skills too.

Reports how raters did on the skill across your jobs: assessments assigned and scored, accuracy, qualification pass rate and credits charged, in total and per set or per question.

Comparison and multiple-choice answers are graded right or wrong, so accuracy is the share answered correctly. The qualification pass rate counts enforced-mode threshold checks only, and is null without them; null never means 0%.

An unknown skill, job or set is a 404, an invalid filter or a future end_at is a 422, and a report that takes too long is a 503; retry with a shorter window or a job_id or set_id filter.

Next

• Skills and qualification
• Streams: skill requirements
• Billing: assessment pricing
• Media
• Task types