Core Configuration

The Core Configuration step defines the foundational replication parameters for the selected tool. These settings control how data is replicated, how offsets are managed, and how the replication infrastructure behaves.

MirrorMaker 2

MirrorMaker 2 configuration is organized into four sections.

Replication Settings

Field	Required	Description
Max Tasks	No	Maximum number of tasks for the MirrorMaker 2 connector. More tasks increase parallelism.
Replication Factor	Yes	The replication factor for internal topics created by MirrorMaker 2 on the destination cluster.
Policy Separator	No	The character used to separate source cluster alias from topic names in the replication policy.
Policy Class	No	The fully qualified class name of the replication policy. Controls how topic names are mapped between source and destination.

Consumer Properties

Field	Required	Description
Group ID	Yes	The consumer group ID used by MirrorMaker 2’s internal consumers.
Auto Offset Reset	No	Where to start consuming when no committed offset exists. Default: Earliest.
Poll Timeout ms	Yes	Maximum time in milliseconds that the consumer waits for new records during each poll cycle.
Request Timeout ms	No	Maximum time in milliseconds for consumer requests to the source cluster.

Producer Overrides

Field	Required	Description
Override Bootstrap Servers	No	Custom bootstrap servers for the producer. When set, the producer connects to these servers instead of the destination cluster’s default.
Override Security Protocol	No	Custom security protocol for the producer connection.
Retry Backoff ms	No	Time in milliseconds between producer retry attempts.

Offsets and Heartbeats

Field	Required	Description
Offsets Topic	No	The name of the internal topic used to store translated consumer group offsets.
Max Offset Lag	No	The maximum offset lag allowed before MirrorMaker 2 flags a warning.
Heartbeat Topics	No	Configuration for heartbeat topics used to monitor MirrorMaker 2 liveness and replication health.

Caution

MirrorMaker 2 requires multiple interdependent components (source connector, checkpoint connector, heartbeat connector). Misconfiguration of any component can cause replication loops, data duplication, or silent failures.

Cluster Linking

Field	Required	Default	Description
Link Name	Yes		A unique name for the cluster link.
Link Mode	No	Destination Initiated	The direction of the link. Matches the linking mode configured during infrastructure setup.
Include Data Rollback Support	No	Off	When enabled, the suite configures the link to support rolling back migrated data to the source cluster.

Topic Settings:

Field	Description
Auto-create Mirror Topics	When enabled, mirror topics are automatically created on the destination when new topics appear on the source.
Topic Config Sync Interval ms	How frequently topic configurations are synchronized from the source to the destination.
Topic Config Sync Include	A tags / multi-select input listing the topic configs the cluster link syncs onto mirror topics. It is pre-populated with the standard set of synced configs, all selected by default. Deselect any config your destination version does not support; the resulting selection becomes the link’s synced-config allowlist when the link is created.

A mandatory subset — max.message.bytes, cleanup.policy, and message.timestamp.type — is always synced and cannot be deselected. Leaving the default selection unchanged preserves existing behavior for non-affected version combinations.

Caution

Cross-version topic configs can silently stall a migration. Confluent Platform 8.0.0+ removed message.timestamp.difference.max.ms (replaced by message.timestamp.before.max.ms / message.timestamp.after.max.ms), but source topics on Confluent Cloud and other Kafka clusters still carry the old config. If it is synced to a CP 8.0.0+ destination, the broker rejects mirror-topic creation with Unknown topic config name: message.timestamp.difference.max.ms. The cluster link still reports healthy, so the migration appears stuck at 0% with no mirror topics created.

For CC → CP and OSK → CP migrations into CP 8.0.0+, deselect message.timestamp.difference.max.ms from Topic Config Sync Include so mirror topics are created successfully.

Note

When the destination is Confluent Platform 7.8, message.timestamp.difference.max.ms is a required topic config and must be kept in (or added to) the Topic Config Sync Include selection alongside the defaults.

Consumer Offset Sync:

Field	Description
Enable Consumer Offset Sync	Toggle to enable consumer offset synchronization through the cluster link.
Sync Interval ms	How frequently consumer offsets are synchronized.

ACL Sync:

Toggle to enable ACL synchronization through the cluster link. When enabled, ACL changes on the source are replicated to the destination.

Replicator

Topic Creation Mode:

Option	Default	Description
Pre-create topics (recommended)	Yes	The suite creates destination topics via its Kafka Admin path before starting the Replicator connector. Required for Confluent Platform 8.0.1+ destinations. Topics are created with the source partition count and explicitly-set source configs, minus any entries in the exclusions list. Use Sync Resources on the Monitor page to pick up new wildcard-matched topics after the job has started.
Let Replicator auto-create	No	Legacy behavior. Replicator creates destination topics at runtime by copying source topic configs verbatim.

Caution

When Let Replicator auto-create is selected, if the source carries topic configs that the destination does not support, Replicator may fail to create or sync those topics and the migration can get stuck in the Sync In Progress state with no error surfaced. Migrating from MSK to Confluent Platform 8.0+ involves configs such as message.format.version that the destination does not support. Select Pre-create topics for that case.

Exclude Topic Configs (Pre-create mode only):

Field

Description

Exclude Topic Configs

A tags-style input where entries are selected from a dropdown of known topic config keys. Shown only when Pre-create is selected. Pre-populated with message.format.version, log.message.format.version, and message.timestamp.difference.max.ms — all three selected by default. Add or remove entries as needed. Configs in this list are not applied to the created destination topics; all other explicitly-set source configs are.

Note

Config exclusions apply only in Pre-create mode. The list is not shown for auto-create migrations, where Replicator controls topic creation directly.

Other Settings:

Field	Required	Description
License Key	No	Confluent Enterprise license key. Without a key, Replicator runs in trial mode for 30 days.
Tasks Max	Yes	Maximum number of replication tasks. This field accepts 1-5. Higher values increase parallelism.
Topic Poll Interval ms	Yes	How frequently Replicator polls the source cluster for new topics.
Preserve Partitions	No	When enabled, the destination topic maintains the same partition count as the source.
Offset Commit Topic	No	The internal topic name used for storing replication offset commits.
Destination Replication Factor	No	The replication factor for topics created on the destination in Pre-create mode. Auto-defaults based on the destination broker count, capped at 3. For Confluent Cloud destinations the replication factor is not specified; Confluent Cloud assigns its managed value.

WarpStream Orbit

Field	Required	Description
Pipeline Name	Yes	A unique name for the Orbit replication pipeline.
Enable Consumer Group Sync	No	When enabled, consumer group offsets are replicated to the WarpStream cluster.
Copy Source Cluster Configuration	No	When enabled, topic-level configurations from the source cluster are applied to the destination.