How Datapoint works

Datapoint turns a labelling request into aggregated human judgments. This page walks the lifecycle so you know what happens after a POST /jobs call and what the moving parts are.

The lifecycle

1. You submit a job

A job describes:

• what to show annotators (datapoints, each with media and an optional context),
• what to ask them (instruction or multi-dimensions),
• how many responses to collect per datapoint (max_responses_per_datapoint, 1–10000),
• the task type: comparison, image-to-video comparison, rating, ranking, or multiple-choice.

Creating a job is synchronous: the API validates the payload against the task-type handler, reserves credit from your account, and queues one background task per datapoint.

2. We distribute tasks to annotators

Datapoint delivers tasks through its ads-based distribution network, so you don't manage a workforce or a labelling tool. Each datapoint is sent to real humans until it has collected max_responses_per_datapoint responses.

Every job runs through the same built-in sampled quality checks and a platform-computed reliability score automatically — you don't configure them, and they apply to every job.

3. You watch progress

GET /jobs/{job_id} returns a live counter of the job's state: its overall status, how many datapoints are still processing, completed, failed, or blocked, and how many responses have been received overall.

The typical shape of a running job:

{
  "job_id": "job_abc123def456",
  "status": "active",
  "task_type": "comparison",
  "total_datapoints": 50,
  "processing_datapoints": 12,
  "ready_datapoints": 38,
  "completed_datapoints": 30,
  "failed_datapoints": 0,
  "blocked_datapoints": 0,
  "total_responses": 412,
  "max_responses_per_datapoint": 10,
  "cost_credits": 2060,
  "credits_per_response": 5,
  "errors": []
}

• status: the job's overall state. While it runs the status is processing or active; it settles to one of the terminal values completed, failed, blocked, or cancelled. Treat completion as status-driven: poll GET /jobs/{job_id} until status is terminal rather than inferring it from the counters.
• processing_datapoints: datapoints not yet dispatched to annotators (no task row yet).
• ready_datapoints: active + completed. Datapoints that have been dispatched and are collecting or done. Partial results are fetchable for every ready datapoint.
• completed_datapoints: reached max_responses_per_datapoint.
• blocked_datapoints: datapoints removed by content review; their reason is surfaced in errors.
• total_responses: the capped, billable count of responses collected across the job so far.
• cost_credits: computed from total_responses collected so far at the job's per-response rate (your account rate plus any audience-targeting surcharge, fixed at creation), not the full reservation.

4. We aggregate responses

Each task type has its own aggregator (see Task types). Aggregation is computed on demand every time you call GET /jobs/{id}/results, so the numbers are always current — there's no separate aggregation step to trigger, and no cached summary that can go stale.

5. You pull results

Two endpoints:

• GET /jobs/{id}/results: one aggregated object per datapoint (votes, mean rating, rank order, etc.). This is what you'd train a model on or log.
• GET /jobs/{id}/responses: raw per-annotator responses (useful for audits and custom analyses).

Both are paginated (page, per_page).

Concurrency and rate of collection

Your job runs as fast as the annotator pool can supply responses for its task type. Typical timings:

• First response within a few minutes.
• Full consensus for a single datapoint within roughly an hour in most regions.
• A job with hundreds of datapoints completes in a few hours end-to-end.

These are steady-state estimates; edge cases (rare language pools, very long media) take longer.

Next

• Task types: comparison, image-to-video, rating, ranking, multiple-choice
• Media handling: upload + dp:// references
• Billing: how credits are reserved and charged