How to change the number of replicas of a Kafka topic
Appearance
1. Specify the extra replicas in a custom reassignment json file
[edit]For example, you could create increase-replication-factor.json and put this content in it:
{"version":1,
"partitions":[
{"topic":"telegraf","partition":0,"replicas":[0,1,2]},
{"topic":"telegraf","partition":1,"replicas":[1,2,0]},
{"topic":"telegraf","partition":2,"replicas":[2,0,1]}
]}
2. Use the file with the --execute option of the kafka-reassign-partitions tool
[edit][or kafka-reassign-partitions.sh - depending on the kafka package]
For example:
$ ./kafka-reassign-partitions.sh --bootstrap-server kafka.kafka:9092 --reassignment-json-file increase-replication-factor.json --execute
3. Verify the replication factor with the kafka-topics tool
[edit][or kafka-topics.sh - depending on the kafka package]
bash-5.0# ./kafka-topics.sh --bootstrap-server kafka.kafka:9092 --topic telegraf --describe
Topic: telegraf TopicId: O4smIJNBRyqmm0z2CwunNA PartitionCount: 3 ReplicationFactor: 1 Configs:
cleanup.policy=delete,segment.bytes=5242880,retention.ms=43200000,message.timestamp.type=LogAppendTime,retention.bytes=-1,delete.retention.ms=43200000
Topic: telegraf Partition: 0 Leader: 0 Replicas: 0,1 Isr: 0 Adding Replicas: 1 Removing Replicas:
Topic: telegraf Partition: 1 Leader: 1 Replicas: 1,2 Isr: 1 Adding Replicas: 2 Removing Replicas:
Topic: telegraf Partition: 2 Leader: 2 Replicas: 2,0 Isr: 2 Adding Replicas: 0 Removing Replicas:
bash-5.0# ./kafka-topics.sh --bootstrap-server kafka.kafka:9092 --topic telegraf --describe
Topic: telegraf TopicId: O4smIJNBRyqmm0z2CwunNA PartitionCount: 3 ReplicationFactor: 2 Configs:
cleanup.policy=delete,segment.bytes=5242880,retention.ms=43200000,message.timestamp.type=LogAppendTime,retention.bytes=-1,delete.retention.ms=43200000
Topic: telegraf Partition: 0 Leader: 0 Replicas: 0,1 Isr: 0,1
Topic: telegraf Partition: 1 Leader: 1 Replicas: 1,2 Isr: 1,2
Topic: telegraf Partition: 2 Leader: 2 Replicas: 2,0 Isr: 2,0
bash-5.0#
See also: the part of the official documentation that describes how to increase the replication factor.