Skip to main content

Custom transformers

Saved Python scripts that pipelines reference by id (type: custom). At run time the engine expands them to type: python with the stored code.

Contract​

def transform(record):
    # record is a dict; return a dict
    return record

Default editor stub matches this. Names are unique.

Editor​

  • Timeout — per-record wall-clock in milliseconds (default 5000, max 60000). Batch runs scale this up to a 5 minute ceiling.
  • Batch size — records run per Python process (default 100, max 1000). A batch runs when it fills, or 500 ms after its first record arrives, so a slow stream is not held back waiting for 100 records. Use a small value for low latency and a large one for bulk throughput.
  • Test panel — POST /api/v1/custom-transformers/test with sample JSON. Does not persist. 30s request cap; concurrency 4 (HTTP 429 if exceeded).

Sandbox (runtime)​

  • python3 subprocess (FALCONTALON_PYTHON_BIN)
  • 256 MiB memory cap
  • Fixed child PATH (/usr/local/bin:/usr/bin:/bin)

After save, pick Custom (Python) in a pipeline transformer step and select this name.