FR
Copied
Concepts

Module registry

A single source of truth describes every module: its inputs, outputs, category, and where it appears in the UI.

Every module outsend exposes is declared in a single registry. It powers the dashboard module grid, the new-job picker, the pipeline editor, and the landing page listing.

Guarantees: a module visible in the dashboard has an endpoint (and vice versa); a machine-readable snapshot is published; categories are hints, while slug, needs and produces are stable.

The endpoint

GET /api/modules-registry

Returns the full registry as JSON. Each entry:

{
  "slug": "scrap",
  "category": "extraction",
  "label": { "fr": "Scrap Google Maps", "en": "Scrape Google Maps" },
  "needs": null,
  "produces": "poi_list",
  "pipelinable": true,
  "coming_soon": false,
  "api_endpoint": "/api/jobs/scrap"
}
Field Meaning
slug Stable identifier, used in URLs and API paths
category extraction | enrichment | intelligence | verification | pipeline | meta
label User-facing display names per language
needs Input shape (poi_list, csv_rows, …): null if produced from scratch
produces Output shape
pipelinable Usable as a node in a pipeline
coming_soon If true, listed for visibility only; interest can be voted
alpha_unavailable If true, the module is built and listed as active everywhere, but frozen during alpha: its create endpoint returns 503
alpha_reduced If true, the module can be launched but is shown with a "reduced functionality during the alpha" notice
api_endpoint Shortcut to start a job of this type

Flexible input matching

needs and produces describe canonical column names (nom, telephone, site_web, email, lien_google_maps, …). You never have to format your data to match them exactly: inputs are resolved against a shared table of accepted aliases, so columns named Website, url, e-mail, name or raison sociale map to the right canonical field. Header-less files are auto-detected and columns are inferred from their content.

Part of produces names signals rather than literal CSV headers: the pipeline builder, the filters and the AI planner use it to know what a module brings. For example dead_check declares site_alive, while its file carries a status column that the alive_only filter rule reads. The exact columns written to each result file are listed in the Outputs section of every module page. Your input columns are always kept, and the module's columns are appended after them.

Every job is transparent about it. Each run reports a non-blocking notice (shown as an info banner on the job page and as a discreet ⓘ on the dashboard) describing what was auto-mapped, guessed, or ignored: for example rows skipped because they had no website. A job only fails when a required column is genuinely absent (e.g. an enrichment that needs site_web finds it on zero rows), and that error explicitly names the accepted aliases so you know what header to provide.

Categories

Category What it does Examples
extraction Produces data from public sources scrap
enrichment Augments existing rows with new fields emails, socials, legal_ids
intelligence Computes signals on existing rows pricing, techstack, ads_intelligence
verification Validates or scores existing rows verify_emails, delivery_check
pipeline Orchestration utilities import, filter, sort
meta Not a job: describes pipelines or veilles (no API endpoint)

Lifecycle of a module

  1. Coming soon: landing page only, no backend, interest votable
  2. Active: fully backed by an endpoint
  3. Available (alpha-frozen): built and presented as an active module across every surface, but not launchable during alpha: the UI shows a maintenance banner with a disabled launch button, and the create endpoint returns 503. Unlike coming soon, it is not a placeholder and carries no interest vote: it is a finished module held back only by alpha capacity.
  4. Deprecated: still callable but flagged

Phase changes appear in the registry via coming_soon, alpha_unavailable, and deprecated_at.

Adding a module (contributors)

Adding a module = 2 files in the codebase: a JS registry entry (UI surfaces) and a Python registry entry (API + worker dispatcher). The runtime then plugs the module everywhere automatically.

What's next