FR
Copied
Concepts

Jobs & lifecycle

A job is one unit of work. This page describes its states, transitions, events, and retry semantics.

A job is one unit of work. Every module runs as a job. Jobs are isolated, observable, resumable.

State machine

   ┌─────────┐    queue picks    ┌─────────┐    success    ┌──────┐
   │ pending │ ────────────────► │ running │ ────────────► │ done │
   └─────────┘                   └─────────┘               └──────┘
        │                             │
        │     user cancels            │     fatal error
        ▼                             ▼
   ┌───────────┐                 ┌────────┐
   │ cancelled │                 │ failed │
   └───────────┘                 └────────┘

   done / failed / cancelled  ──── (after 7 days) ────►  expired
State Meaning
pending Created, sitting in the FIFO queue
running Picked by a worker, executing
done Completed successfully, results downloadable
failed Errored out (see error_message)
cancelled Cancelled via the UI or API
expired More than 7 days since terminal state: result files purged

Transitions and queue assignment are atomic; a job is never picked twice.

Creation

POST /api/jobs             { "queries": [...], "zones": [...] }   # creates a scrap job
POST /api/jobs/{type}      { ...module-specific params }          # typed shortcut

See Jobs API.

Observability

GET /api/jobs/{id}            # status, counters, metadata
GET /api/jobs/{id}/stream     # SSE: status / log / done

The stream closes when the job terminates. Safety timeout: 6 hours. Event payloads: see States & SSE events.

Results

GET /api/jobs/{id}/download?format=csv|json|xlsx
GET /api/jobs/{id}/items?offset=0&limit=200

Results live 7 days after terminal state, then are purged. The job record remains.

Errors & retries

A failed job exposes error_message and error_count (items that errored inside the job: a job can be done with error_count > 0).

POST /api/jobs/{id}/resume

Creates a new attempt resuming from the last successful item.

Only one resume of a given job can be pending or running at a time: a second call returns 400 until the first one reaches a terminal state.

Maximum run time

A single job runs for at most 24 hours. When the limit is reached, the job stops cleanly and ends in failed, with an error_message stating that the maximum duration was reached. Results obtained so far are kept and downloadable.

For resumable jobs, a follow-up job is queued automatically: same settings, source_job_id pointing to the stopped job, starting from where it stopped and billed only for the remainder. No call to /resume is needed. A long job therefore appears as a chain of jobs of up to 24 hours each, the last one carrying the complete result file.

If the job made no progress during its run, or if the account already has 5 jobs pending or running, no follow-up is queued: the error_message says so and the job can be resumed with POST /api/jobs/{id}/resume.

Cancellation

POST /api/jobs/{id}/cancel    # keeps partial results
DELETE /api/jobs/{id}         # cancels and removes record

Concurrency

What's next