Import WarpStream

WarpStream Orbit Pipelines can be imported into the suite to monitor data replication between WarpStream virtual clusters and standard Kafka clusters. The import flow connects to the WarpStream Console API to discover and import active pipelines.

Import Workflow

1. Enter your WarpStream Console URL, API Key, and Virtual Cluster ID to authenticate with the WarpStream platform.

   

2. The suite connects to the WarpStream Console and lists discovered pipelines. Choose the Orbit Pipeline you want to import.

   

3. Map the Source Kafka cluster by selecting from your registered clusters or adding a new one. The source cluster corresponds to the origin of the data flow in the pipeline.

   

4. Map the Destination Kafka cluster to complete the cluster pairing for the pipeline.

   

5. Toggle Schema Link to enable schema replication for the pipeline. If your pipeline does not involve schema-managed topics, leave this disabled and skip to Step 8.

   

6. Discover Schema Link Pipelines. The suite scans for schema link pipelines associated with the selected Orbit Pipeline and lists them for selection.

   

7. Configure Source and Destination schema registries by providing the registry URLs, authentication details, and credentials for both the source and destination registries.

8. Assign a Job Name for the imported pipeline and click Import and Start Monitoring to finalize.

   

Tip

The WarpStream Console URL, API Key, and Virtual Cluster ID are available from your WarpStream Console dashboard. Ensure the API Key has sufficient permissions to list pipelines and read pipeline configuration.

Caution

Schema registry configuration is optional but recommended if your topics use Avro, Protobuf, or JSON Schema serialization. Skipping this step means schema synchronization will not be monitored.