Create a Pipeline

A brief guide on how to spin up a pipeline

Pipelines let you easily and securely transfer real-time event data between a Source and a Destination. Creating pipelines is easy. They can also be created either via the Developer UI or the SDK.

1. Name your Pipeline

The first thing you want to do is create a unique identifier for your pipeline.

2. Configure the source

Select the Source of data and the Event you'd like to use as the pipeline's trigger. The trigger event will be used to invoke the pipeline logic whenever the event is collected.

3. Configure the destination

Select the Destination for your event data. You'll also need to select the action you'd like to use when your trigger event is received.

4. Enrich the data

All pipelines come with Enrichment functionality by default. To enrich the event data, you can use an Extractor to grab data from another part of your system.

To learn more on how to use Extractors, visit the full Extractors Documentation:

Enrich your Data

5. Transform the data

All pipelines come with Transformation functionality by default. To transform your event data in flight and shape it according to the Destination's data structure, simply write your Transformation in JavaScript.

6. Add a Retry Policy

Retry Policies let you set rules for how many times (and for how long) you'd like to retry your pipeline if it originally fails. Each pipeline has a unique retry policy.