Workers and the execution queue
There is one job kind: pipeline execution. The queue is the Postgres executions table, not Kafka.
The API must run with FALCONTALON_WORKER_MODE=cluster. Worker processes (command: ["/worker"]) poll the API. If the API is also set to run jobs itself, it races those workers.
Lifecycle
scheduled → claimed → running → completed | failed | cancelled
POST /api/v1/pipelines/{id}/executions returns 202. Optional body: max_retries (default 3), priority, run_at. Queue overflow is HTTP 429.
Retry re-sets status to scheduled with exponential backoff (default base 30s). Cancel sets cancel_requested; the worker notices on heartbeat (about 1s).
Worker loop
POST /api/v1/worker/register— upsertworker_heartbeatsPOST /api/v1/worker/claim— atomic claim; response is a JobBundlePOST /api/v1/worker/executions/{id}/running- Heartbeat every 10s — extends lease, returns
cancel_requested POST .../finish—completed/cancelled/failed(API may requeue)
Auth is API key only.
A JobBundle includes the pipeline definition, decrypted datasource, and schemas. Workers need no database.
Defaults
| Knob | Typical |
|---|---|
| GlobalSlots | 4 per process (FALCONTALON_GLOBAL_SLOTS) |
| PerPipelineSlots | 1 |
| Heartbeat | 10s |
| Lease TTL | 60s (≥ 2× heartbeat) |
| Reaper | 90s without heartbeat |
| Service / probes | no HTTP Service. TCP/UDP listen runs create a per-execution ClusterIP when listener_services is on |
Reaper
The API deletes workers whose last_seen_at is older than FALCONTALON_WORKER_REAP_AFTER_MS and unassigns their claimed/running executions. Expired leases also requeue.
Partial sink writes from abandoned runs are not rolled back.
Console: Workers page
Polls GET /api/v1/worker/stats (queue counts, instances, leases). An empty connected-workers table means pods are not registering — check FALCONTALON_API_URL, FALCONTALON_KEY_API, and worker logs.