kafka replication tools 博客分类： kafka kafkareplication

kafka工具

 

replication tool工作步骤（异步过程，执行完1就结束）：

1.更新zk上的/admin/preferred_replica_election节点，内容为副本的几个位置（不包含已经奔溃的leader）

 

2.controller中的zk的listener读取topic partition的副本的几个位置

 

3.controller获取每个topic partition的副本的几个replica中的第一个，作为referred replica；

  如果这个preferred replica不是leader且在isr中，则controller发出request到这个preferred replica的broker，通知其作为topic partition的leader

  

  

bin/kafka-preferred-replica-election.sh --zookeeper localhost:81821/kafka-zk --path-to-json-file topicPartitionList.json

 

topicPartitionList.json格式为：

{

 "partitions":

  [

    {"topic": "topic1", "partition": "0"},

    {"topic": "topic1", "partition": "1"},

    {"topic": "topic1", "partition": "2"},

 

    {"topic": "topic2", "partition": "0"},

    {"topic": "topic2", "partition": "1"},

  ]

}

 

FAQ

What happens if the preferred replica is not in the ISR?

The controller will fail to move the leadership to the preferred replica if it is not in the ISR. This is to ensure that there is no dataloss. When the replica becomes "in-sync" with the leader, the tool can be run again to move the leader.

则移动失败，避免数据丢失

 

How to find if all the partitions have been moved to the "preferred replica" after running the tool?

ListTopicCommand is an excellent tool that provides an overview of all the topic partitions in the cluster. For each topic partition, it displays the leader, assigned replicas and current "in-sync" replica set. If the leader and the first replica in the assigned replica set are the same then the Preferred replica leader election" tool succeeded. If not, the tool failed and may have to be run again.

跟isr的第一个replica相同