avn mirrormaker

Here you’ll find the full list of commands for avn mirrormaker.

Create and manage Aiven for Apache Kafka MirrorMaker2 replication flows

Commands for managing Aiven for Apache Kafka MirrorMaker2 replication flows.

avn mirrormaker replication-flow create

Creates a new Aiven for Apache Kafka MirrorMaker2 replication flow.

Warning

Before creating a replication flow, an integration needs to be created between the Aiven for Apache Kafka MirrorMaker2 service and each of the source and the target services.

E.g. An integration with alias kafka-target-alias between an Aiven for Apache Kafka service named kafka-target and an Aiven for Apache Kafka MirrorMaker2 named kafka-mm can be created with:

avn service integration-create \
  -s kafka-target              \
  -d kafka-mm                  \
  -t kafka_mirrormaker         \
  -c cluster_alias=kafka-target-alias

At most one replication flow can be build between any two Aiven for Apache Kafka services.

Parameter

Information

service_name

The Aiven for Apache Kafka MirrorMaker2 service where to create the replication flow

--source-cluster

The Aiven for Apache Kafka service to be used as source for replication

--target-cluster

The Aiven for Apache Kafka service to be used as target for replication

replication_flow_config

JSON string or path (preceded by @) to a JSON configuration file for the replication flow definition

Example: In the service kafka-mm create a new replication flow from an Aiven for Apache Kafka service with integration alias kafka-source-alias to a service named kafka-target-alias with the following settings:

  • include all topics with name starting with my-src-topic (topic name patterns can be defined using Java patterns)

  • exclude all topics with name ending with not-include

  • DefaultReplicationPolicy as replication policy class

  • enable MirrorMaker2 heartbeats

  • enable synching of consumer groups offset every 60 seconds

avn mirrormaker replication-flow create kafka-mm \
  --source-cluster kafka-source-alias \
  --target-cluster kafka-target-alias \
  '
    {
        "emit_heartbeats_enabled": true,
        "enabled": true,
        "replication_policy_class": "org.apache.kafka.connect.mirror.DefaultReplicationPolicy",
        "source_cluster": "kafka-source-alias",
        "sync_group_offsets_enabled": true,
        "sync_group_offsets_interval_seconds": 60,
        "target_cluster": "kafka-target-alias",
        "topics": [
            "my-src-topic.*"
        ],
        "topics.blacklist": [
            ".*not-include"
        ]
    }
  '

avn mirrormaker replication-flow delete

Deletes an existing Aiven for Apache Kafka MirrorMaker2 replication flow.

Parameter

Information

service_name

The Aiven for Apache Kafka MirrorMaker2 service where to delete the replication flow

--source-cluster

The Aiven for Apache Kafka service to be used as source for replication

--target-cluster

The Aiven for Apache Kafka service to be used as target for replication

Example: In the service kafka-mm delete the replication flow from an Aiven for Apache Kafka service with integration alias kafka-source-alias to the service named kafka-target-alias.

avn mirrormaker replication-flow delete kafka-mm \
  --source-cluster kafka-source-alias \
  --target-cluster kafka-target-alias

avn mirrormaker replication-flow get

Retrieves the configuration details of an existing Aiven for Apache Kafka MirrorMaker2 replication flow.

Parameter

Information

service_name

The Aiven for Apache Kafka MirrorMaker2 service where to get the replication flow details

--source-cluster

The Aiven for Apache Kafka service to be used as source for replication

--target-cluster

The Aiven for Apache Kafka service to be used as target for replication

Example: In the service kafka-mm retrieve the details of the replication flow from an Aiven for Apache Kafka service with integration alias kafka-source-alias to the service named kafka-target-alias.

avn mirrormaker replication-flow get kafka-mm \
  --source-cluster kafka-source-alias \
  --target-cluster kafka-target-alias

An example of the avn mirrormaker replication-flow get command output:

{
    "emit_heartbeats_enabled": true,
    "enabled": true,
    "replication_policy_class": "org.apache.kafka.connect.mirror.DefaultReplicationPolicy",
    "source_cluster": "kafka-source-alias",
    "sync_group_offsets_enabled": true,
    "sync_group_offsets_interval_seconds": 60,
    "target_cluster": "kafka-target-alias",
    "topics": [
        "my-src-topic.*"
    ],
    "topics.blacklist": [
        ".*not-include"
    ]
}

avn mirrormaker replication-flow list

Lists the configuration details for all replication flows defined in an existing Aiven for Apache Kafka MirrorMaker2 service.

Parameter

Information

service_name

The Aiven for Apache Kafka MirrorMaker2 service where to list the replication flow

Example: List the configuration details for all replication flows defined in an existing Aiven for Apache Kafka MirrorMaker2 named kafka-mm.

avn mirrormaker replication-flow list kafka-mm

An example of the avn mirrormaker replication-flow list command output:

[
      {
          "emit_heartbeats_enabled": true,
          "enabled": true,
          "replication_policy_class": "org.apache.kafka.connect.mirror.DefaultReplicationPolicy",
          "source_cluster": "kafka-source-alias",
          "sync_group_offsets_enabled": true,
          "sync_group_offsets_interval_seconds": 60,
          "target_cluster": "kafka-target-alias",
          "topics": [
              "my-src-topic.*"
          ],
          "topics.blacklist": [
              ".*not-include"
          ]
      }
 ]

avn mirrormaker replication-flow update

Updates an existing Aiven for Apache Kafka MirrorMaker2 replication flow.

Parameter

Information

service_name

The Aiven for Apache Kafka MirrorMaker2 service where to update the replication flow

--source-cluster

The Aiven for Apache Kafka service to be used as source for replication

--target-cluster

The Aiven for Apache Kafka service to be used as target for replication

replication_flow_config

JSON string or path (preceded by @) to a JSON configuration file for the replication flow definition

Example: In the service kafka-mm update the replication flow from an Aiven for Apache Kafka service with integration alias kafka-source-alias to a service named kafka-target-alias with the settings contained in a file named replication-flow.json.

avn mirrormaker replication-flow update kafka-mm \
  --source-cluster kafka-source-alias \
  --target-cluster kafka-target-alias \
  @replication-flow.json