Skip to main content

User Guide

Synapse is a UI and API for building ETL pipelines that read maritime and ISR data, validate it with JSON Schema, transform it, and write it to a sink.

Mental model​

  1. Datasource — a named pair of source (where items are discovered) and sink (where successful records are written). The API calls this a configuration (/api/v1/configs).
  2. Schema — a versioned JSON Schema under a subject. Pipelines pin a specific schema id on each classifier.
  3. Custom transformer (optional) — a saved Python transform(record) script referenced by id.
  4. Pipeline — datasource + reader + ordered classifier/transformer chain.
  5. Playground — dry-run the chain on pasted text (no datasource I/O).
  6. Execution — one finite run. Not a cron job. Status lives on the Executions page.

Typical order: datasource → schemas → optional custom transformer → pipeline → playground → Run → Executions.

In-app help icons next to fields repeat this guidance. This guide covers the same workflows with engine caveats the UI does not always surface.

What this product is not​

  • No cron / recurring schedule UI (run_at exists on the API only)
  • No schema auto-discovery from sample files
  • No visualization product