rebuild command
Jump to Common Topic Operations
rebuild is used for:
- replacing brokers
- removing brokers (scaling down or preparing a broker to be decommissioned)
- adding brokers (scaling up or replacing one broker with two or more brokers)
- isolating one or more topics to a specific set of brokers
- building optimized shard placements from scratch (by partition count or by storage bin-packing)
- changing replication factors
The primary input is a reference of topics to rebuild (either looked up in ZooKeeper by topic name via --topics or provided literally as JSON via --map-string flags) and a list of brokers (via the the --brokers flag). Topicmappr builds a partition placement map that ensures that the referenced topics are mapped to the listed brokers. The output is a kafka-reassign-partitions.sh compatible JSON file for each referenced topic that could then be executed.
Subsequent topicmappr runs should always produce the same output map (*provided the same input; a change in topic state in ZooKeeper would be a different input). In order to minimize movement, topicmappr by default only maps a partition to a new broker if an existing broker isn't provided in the broker list or wasn't found in ZooKeeper.
For instance, if all partitions for test_topic exist on brokers 1001,1002, running topicmappr with the provided broker list 1001,1002,1003,1004 would be a no-op. An example that can be tested locally using a mock map:
$ topicmappr rebuild --map-string '{"version":1,"partitions":[{"topic":"test_topic","partition":0,"replicas":[1001,1002]},{"topic":"test_topic","partition":1,"replicas":[1002,1001]},{"topic":"test_topic","partition":2,"replicas":[1001,1002]},{"topic":"test_topic","partition":3,"replicas":[1002,1001]}]}' --brokers=1001,1002 --use-meta=false
Topics:
test_topic
Broker change summary:
Replacing 0, added 0, missing 0, total count changed by 0
Action:
no-op
WARN:
[none]
Partition map changes:
test_topic p0: [1001 1002] -> [1001 1002] no-op
test_topic p1: [1002 1001] -> [1002 1001] no-op
test_topic p2: [1001 1002] -> [1001 1002] no-op
test_topic p3: [1002 1001] -> [1002 1001] no-op
...
However if broker 1002 failed, we would exclude it from the list, indicating that those positions be filled with the best fitting broker in the list:
$ topicmappr rebuild --map-string '{"version":1,"partitions":[{"topic":"test_topic","partition":0,"replicas":[1001,1002]},{"topic":"test_topic","partition":1,"replicas":[1002,1001]},{"topic":"test_topic","partition":2,"replicas":[1001,1002]},{"topic":"test_topic","partition":3,"replicas":[1002,1001]}]}' --brokers=1001,1003,1004 --use-meta=false
Topics:
test_topic
Broker change summary:
Broker 1002 marked for removal
New broker 1003
New broker 1004
-
Replacing 1, added 2, missing 0, total count changed by 1
Action:
Rebuild topic with 1 broker(s) marked for replacement
WARN:
[none]
Partition map changes:
test_topic p0: [1001 1002] -> [1001 1003] replaced broker
test_topic p1: [1002 1001] -> [1003 1001] replaced broker
test_topic p2: [1001 1002] -> [1001 1004] replaced broker
test_topic p3: [1002 1001] -> [1004 1001] replaced broker
Broker distribution:
degree [min/max/avg]: 1/1/1.00 -> 1/2/1.33
-
Broker 1001 - leader: 2, follower: 2, total: 4
Broker 1003 - leader: 1, follower: 1, total: 2
Broker 1004 - leader: 1, follower: 1, total: 2
New partition maps:
test_topic.json
Essentially, if all brokers housing a topic are a subset of the provided list, nothing changes. The default action is to only fix what's broken.
This can be overridden with the --force-rebuild option. This tells topicmappr to rebuild an ideal map from the broker list, discarding the current state of the topic:
$ topicmappr rebuild --map-string '{"version":1,"partitions":[{"topic":"test_topic","partition":0,"replicas":[1001,1002]},{"topic":"test_topic","partition":1,"replicas":[1002,1001]},{"topic":"test_topic","partition":2,"replicas":[1001,1002]},{"topic":"test_topic","partition":3,"replicas":[1002,1001]}]}' --brokers=1001,1002,1003,1004 --use-meta=false --force-rebuild
Topics:
test_topic
Broker change summary:
New broker 1003
New broker 1004
-
Replacing 0, added 2, missing 0, total count changed by 2
Action:
Expanding/rebalancing topic with 2 additional broker(s) (this is a no-op unless --force-rebuild is specified)
WARN:
[none]
Partition map changes:
test_topic p0: [1001 1002] -> [1001 1002] no-op
test_topic p1: [1002 1001] -> [1002 1003] replaced broker
test_topic p2: [1001 1002] -> [1003 1001] replaced broker
test_topic p3: [1002 1001] -> [1004 1002] replaced broker
Broker distribution:
degree [min/max/avg]: 1/1/1.00 -> 1/3/2.00
-
Broker 1001 - leader: 1, follower: 1, total: 2
Broker 1002 - leader: 1, follower: 2, total: 3
Broker 1003 - leader: 1, follower: 1, total: 2
Broker 1004 - leader: 1, follower: 0, total: 1
...
In this scenario, the newly provided brokers 1003 and 1004 are mapped in alongside the previous 1001 and 1002, providing perfect leader and follower balance.
The --topics method of specifying target topics takes a comma delimited list of topic names. The current partition map for all matching topic is fetched from ZooKeeper. Brokers in the provided --brokers list are also looked up in ZooKeeper. The topic map(s) are then compared against the broker list to make mapping decisions.
Topic name regex patterns is also supported*. For instance, providing --topics="test_topic[0-9]" might return test_topic1, test_topic2, and test_topic3. Name strings and regex patterns can be combined: --topics="test_topic,numbered_topics[0-9]".
Example:
Replacing broker ID 1 with ID 2 for all partitions in test_topic.
$ topicmappr rebuild --topics test_topic -brokers "0,2" -zk-addr "localhost:2181"
Topics:
test_topic
Broker change summary:
Broker 1 marked for removal
Replacing 1, added 1, total count changed by 0
Action:
Rebuild topic with 1 broker(s) marked for removal
WARN:
[none]
Partition map changes:
test_topic p0: [0 1] -> [0 2] replaced broker
test_topic p1: [1 0] -> [2 0] replaced broker
test_topic p2: [0 1] -> [0 2] replaced broker
test_topic p3: [1 0] -> [2 0] replaced broker
test_topic p4: [0 1] -> [0 2] replaced broker
test_topic p5: [1 0] -> [2 0] replaced broker
test_topic p6: [0 1] -> [0 2] replaced broker
test_topic p7: [1 0] -> [2 0] replaced broker
Broker distribution:
degree [min/max/avg]: 1/1/1.00 -> 1/1/1.00
-
Broker 0 - leader: 4, follower: 4, total: 8
Broker 2 - leader: 4, follower: 4, total: 8
New parition maps:
test_topic.json
*Take note of how regex is interpreted: all topics included in the --topics list ultimately become regex. Topic names where all characters are those allowed by Kafka (a-z, A-Z, 0-9, _, -, .) sans ., are assumed to be literal names and thus become the regex /^topic$/. The inclusion of a . or any other character assumes that the entry is to be interpreted as regex and is compiled as is. This means that if you want to rebuild the literal topic my.topic, it's best to provide --topics="my\.topic". Without escaping the . (--topics="my.topic"), both my.topic and my1topic would be targeted.
The --map-string flag takes a literal partition map as JSON (this is the same format that Kafka uses, i.e. the input and output of kafka-reassign-partitions). Brokers in the provided --brokers list are looked up in ZooKeeper. The input topic map(s) are then compared against the broker list to make mapping decisions.
Multiple topics can be included in the topic map.
Example:
Rebuilding test_topic which was originally mapped to broker IDs 1001,1002,1003, but lost broker 1003. The newly supplied ID list 1001,1002,1004 includes the replacement broker ID 1004.
$ topicmappr rebuild-map --map-string '{"version":1,"partitions":[{"topic":"test_topic","partition":0,"replicas":[1005,1006]},{"topic":"test_topic","partition":2,"replicas":[1007,1001]},{"topic":"test_topic","partition":7,"replicas":[1007,1002]},{"topic":"test_topic","partition":6,"replicas":[1006,1001]},{"topic":"test_topic","partition":4,"replicas":[1002,1005]},{"topic":"test_topic","partition":5,"replicas":[1005,1007]},{"topic":"test_topic","partition":3,"replicas":[1001,1002]},{"topic":"test_topic","partition":1,"replicas":[1006,1007]}]}' -brokers="1001,1002,1003,1004,1005,1006,1008" -use-meta=false
Topics:
test_topic
Broker change summary:
Broker 1007 marked for removal
Replacing 1, added 3, total count changed by 2
Action:
Rebuild topic with 1 broker(s) marked for removal
WARN:
[none]
Partition map changes:
test_topic p0: [1005 1006] -> [1005 1006]
test_topic p1: [1006 1007] -> [1006 1003] replaced broker
test_topic p2: [1007 1001] -> [1003 1001] replaced broker
test_topic p3: [1001 1002] -> [1001 1002]
test_topic p4: [1002 1005] -> [1002 1005]
test_topic p5: [1005 1007] -> [1005 1008] replaced broker
test_topic p6: [1006 1001] -> [1006 1001]
test_topic p7: [1007 1002] -> [1004 1002] replaced broker
Broker distribution:
degree [min/max/avg]: 3/4/3.20 -> 1/3/2.29
-
Broker 1001 - leader: 1, follower: 2, total: 3
Broker 1002 - leader: 1, follower: 2, total: 3
Broker 1003 - leader: 1, follower: 1, total: 2
Broker 1004 - leader: 1, follower: 0, total: 1
Broker 1005 - leader: 2, follower: 1, total: 3
Broker 1006 - leader: 2, follower: 1, total: 3
Broker 1008 - leader: 0, follower: 1, total: 1
New parition maps:
test_topic.json
Topicmappr has two methods of determining where partition replicas go, otherwise known as the placement strategy. This is controlled via the --placement flag, which takes one of two values: count or storage. In all previous examples, we chose the default count method.
The count strategy balances partitions in a way that results in the most even number across brokers. This is simple and reliable if imbalances in data volumes among partitions is not anticipated.
When repairing a topic, for example a one that was originally mapped to brokers 1,2,3,4,5 where broker 5 failed and the user provides the new broker list 1,2,3,4,6, the count strategy will attempt even partition balance when choosing replacements. This means that if broker 1 were lowly utilized (in terms of partitions held), it may take up a few of the partitions along with the newly provided broker 6.
The count method can be used to repair missing brokers from a topic that was previously storage balanced, but in this case it may be desirable to maintain the uneven partition counts (since the storage strategy places by size rather than quantity). Using the optional substitution affinity feature (via the --sub-affinity flag) allows users to ensure 1:1 replacements for specific brokers. In the above example, if the replacement broker 6 has the same rack ID as the previously failed broker 5, enabling substitution affinity would ensure that only broker 6 will be used used to replace all of the ISR slots left by broker 5.
When this feature is used, it's indicated in the information output:
Broker change summary:
Broker 1002 marked for removal
New broker 1003
New broker 1004
-
Substitution affinity: 1002 -> 1003
-
Replacing 1, added 2, missing 0, total count changed by 1
This means that every slot left behind by 1002 will be replaced with 1003. Disabling this feature (the default) would allow both 1003 and 1004 to fill those slots.
Substitution affinity has two ways of choosing a substitution broker. If the broker being replaced is still registered in ZooKeeper, it prefers a newly added broker (a broker in the --brokers list that is not currently holding any replicas for topics in the --rebuild list) with the same rack ID. If the broker being replaced has failed and is no longer in ZooKeeper, substitution affinity will infer a suitable replacement and indicate this by appending (inferred) to the info output.
The storage strategy chooses brokers based on free space and partition size (using an algorithm modeled on first-fit descending bin packing). In each placement decision, the broker with the most available free space that satisfies all other constraints is chosen. The storage strategy is best used if large imbalances among partitions is anticipated.
The storage placement strategy is tunable as to whether it biases for maximum partition dispersion or maximum storage balance, via the --optimize param. The default is distribution and is suitable for most storage placements. The storage optimization is used when a few partitions are disproportionately large and result in undesirable range spreads in broker free storage when using the default distribution optimization.
Additionally, the --partition-size-factor (defaults to 1.0) flag allows placement decisions using increased or decreased partition sizes. For instance, if a topic were being migrated to a larger broker set with the intent of later increasing the topic retention by 50%, specifying --partition-size-factor 1.5 would multiply the partition size cost by 1.5x during bin-packing. This allows more accurate placement decisions and early storage exhaustion warnings. Similarly, a placement that accommodates for an eventual 50% reduction in retention could be configured by specifying --partition-size-factor 0.5.
When using the storage placement strategy, an estimate of changes in free storage is printed in the topicmappr summary output, including the change in range, range spread and standard deviation of free storage across all referenced brokers:
...
Broker 1015 - leader: 5, follower: 5, total: 10
Broker 1016 - leader: 6, follower: 6, total: 12
Broker 1017 - leader: 5, follower: 6, total: 11
Broker 1018 - leader: 5, follower: 4, total: 9
Storage free change estimations:
range: 344.02GB -> 31.11GB
range spread: 42.44% -> 3.02%
std. deviation: 89.14GB -> 13.82GB
-
Broker 1002: 1130.20 -> 1030.99 (-99.21GB, -8.78%)
Broker 1003: 1061.89 -> 1058.72 (-3.17GB, -0.30%)
Broker 1005: 1005.35 -> 1037.43 (+32.08GB, 3.19%)
Broker 1006: 1135.51 -> 1056.87 (-78.65GB, -6.93%)
Broker 1009: 810.59 -> 1059.90 (+249.32GB, 30.76%)
Broker 1012: 1056.41 -> 1029.81 (-26.60GB, -2.52%)
...
The storage strategy requires complete metrics data in order to operate. Topicmappr will check for the following znodes as children of /topicmappr (configurable via --zk-metrics-prefix):
/topicmappr/partitionmeta
The znode data must be formatted as JSON with the structure {"<topic name>": {"<partition number>": {"Size": <bytes>}}}. Metrics data for all topics being mapped must be present for all partitions.
/topicmappr/brokermetrics
The znode data must be formatted as JSON with the structure {"<broker ID>": {"StorageFree": <bytes>}}. Metrics data for all brokers participating brokers in a mapping operation must be present.
This data can be populated from any metrics system as long as it conforms to these standards. The provided metricsfetcher is a simple Datadog implementation.
Ensure that recent data is being used. If stale metrics data is being used, a placement map could be built that's suboptimal. Topics that have been storage balanced can be repaired (e.g. if a broker fails) using the count method combined with the --sub-affinity flag.
Topicmappr attempts to ensure safe operations where possible. One way it does this is by referencing Kafka cluster state from ZooKeeper to enforce the following:
- The
broker.rackKafka attribute is used as a placement constraint. No replica set for a given partition can hold more than one broker for a givenbroker.rackvalue. - All provided brokers exist and are visible in ZooKeeper.
- The topic exists in ZooKeeper.
- Enough brokers are available for the given replication factor.
- Storage placements do not over-run available storage (when using the storage placement strategy).
When confronted with an error condition or inability to make a sound decision, topicmappr avoids producing any output maps and advises the user of the fault.
Example, attempting to rebuild a topic with a non-existent broker:
$ topicmappr rebuild-topic --topics test_topic --brokers "0,3"
Broker change summary:
Broker 1 marked for removal
Broker 3 not found in ZooKeeper
Replacing 1, added 0, total count changed by -1
Action:
Shrinking topic by 1 broker(s)
WARN:
test_topic p0: No additional brokers that meet constraints
test_topic p1: No additional brokers that meet constraints
test_topic p2: No additional brokers that meet constraints
test_topic p3: No additional brokers that meet constraints
test_topic p4: No additional brokers that meet constraints
test_topic p5: No additional brokers that meet constraints
test_topic p6: No additional brokers that meet constraints
test_topic p7: No additional brokers that meet constraints
Partition map changes:
test_topic p0: [0 1] -> [0] replaced broker
test_topic p1: [1 0] -> [0] replaced broker
test_topic p2: [0 1] -> [0] replaced broker
<...>
-
Topicmappr's placement decisions and statistical output (such as degree distribution or partition leadership counts) is always scoped to the input topics and brokers. For instance, total partition counts in the "Broker distribution" summary will not include partitions that the broker may hold if they belong to topics that were not included by the
--topicsinput. -
Statistics in the "Broker distribution" summary refers to the min, max and average relationships among brokers in terms of vertex degrees. An edge is defined as a common partition replica between two brokers. For instance, if broker 1001 held 20 partitions with a degree value of 8, the other replicas for those 20 partitions held would be dispersed over 8 other brokers in the cluster. Degree distribution doesn't explain exactly how data is distributed, but it can be used as an indicator of the level of distribution. If broker 1001 failed and were replaced, there would be at most 8 replication sources for the replacement broker to stream in data from. More replication sources generally means more recovery bandwidth and less impact on any individual source.
Take note that optimal node degree is a function of both partition replica and broker counts, so a decrease isn't always a sign of worse distribution or recovery capacity. For instance, increasing brokers over a constant number of total partitions would result in a lower average node degree, as each broker becomes responsible for fewer partitions (reducing the ceiling of possible relationships).
These procedures are presented as clear, quick references to common tasks and the associated commands. As a result, required but not immediately relevant inputs, such as the --zk-addr param, are excluded. Likewise, the wealth of detailed topicmappr output information for each task is excluded from the examples but will be available to the user at run time.
Data replication occurs in most cases of the following operations. Check out autothrottle to dynamically control the replication throttle rates.
Use case: A topic experiences one or more broker failures resulting in incomplete ISRs.
Topicmappr can be used to repair a topic no matter how it was previously managed. The simplest way to build a repair map is to provide a list of the healthy brokers that excludes failed brokers and includes a suitable number of replacements.
Procedure: If test_topicwere mapped to brokers 1,2,3 and 3 failed, the topic could be rebuilt with the replacement broker 4:
$ topicmappr rebuild --topics test_topic --brokers 1,2,4
Depending on previous partition counts held, it's possible that brokers 1 or 2 could also fill some of the previously held replicas for broker 3. If it's desired to guarantee that only broker 4 does a 1:1 replacement for all replicas previously held by 3, enable substitution affinity:
$ topicmappr rebuild --topics test_topic --brokers 1,2,4 --sub-affinity
This is useful if the topic were previously storage balanced, which could result in an intentional imbalance by count of partition assignments per broker.
Use case: The number of replicas per partition needs to be changed or made consistent.
The replication factor for a topic can be changed without affecting any existing leader/follower assignment.
Procedure: Iftest_topic were currently configured with a replication factor of 2, it could be changed to a replication factor of 3 by providing the existing broker list and an updated --replication value:
$ topicmappr rebuild --topics test_topic --brokers 1,2,3 --replication 3
This can also be used to normalize the replication factor for a topic with varying replication factors across partitions. If test_topic had many partitions with a replication factor of 3 but some partitions configured to 2, the above command would ensure that all partitions were consistent with a replication factor of 3.
Use case: A topic is outgrowing the resources for the set of brokers on which it resides, requiring more resources.
See the rebalance command for an alternative (and generally recommended) scaling procedure.
The following scaling examples show count based placement. It's possible to simultaneously scale and perform a storage balancing across brokers. See the storage rebalancing section.
Topicmappr provides two ways to scale a topic over more brokers:
Full migration procedure: Provide topicmappr a completely new list of brokers (e.g. none of the provided brokers are any of those already hosting the topic). The disadvantage of this method is that 100% of the data must be moved. The advantage is a reduced risk of running a broker out of storage (compared to an in place scaling) and more total bandwidth will be available for the data transfers.
If test_topic were mapped to brokers 1, 2, 3, we could scale to 5 brokers by providing an entirely new list of 5 brokers:
$ topicmappr rebuild --topics test_topic --brokers 4,5,6,7,8
In-place scaling procedure: Provide topicmappr with a broker list that includes those already hosting the topic plus additional brokers. Run topicmappr with --force-rebuild. The advantage of this method is requiring fewer new brokers than a full migration scale out would require. If possible, it's recommended to temporarily reduce topic retention when doing in place scaling to avoid the risk of running a broker out of space.
If test_topic were mapped to brokers 1,2,3, we could scale to 5 brokers in-place by providing 2 additional brokers with --force-rebuild:
$ topicmappr rebuild --topics test_topic --brokers 1,2,3,4,5 --force-rebuild
Use case: A topic is underutilizing the resources for the set of brokers on which it resides, requiring fewer resources.
Procedure: If test_topic were mapped to brokers 1,2,3,4,5, it could be scaled down by 2 brokers by excluding 2 from the broker list (topicmappr will automatically rebalance partitions over the remaining brokers):
$ topicmappr rebuild --topics test_topic --brokers 1,2,3
Scaling down also supports storage bin-packing; partitions located on brokers marked for removal can be targeted to remaining brokers by storage free space and partition size. See the Storage Rebalancing section for reference. The scale down procedure is the same except that brokers targeted for removal are simply omitted from the list. The above example would become:
$ topicmappr rebuild --topics test_topic --brokers 1,2,3 --placement storage
Use case: Ensuring maximum balance and distribution of partition replicas and leadership over a set of brokers by count.
A topic may have been previously mapped in way that yields uneven partition ownership and leadership across brokers. Topicmappr can build a map that optimizes partition ownership and leadership across brokers either in-place or over new brokers in a similar fashion to Scaling Up with the same tradeoffs. An in-place procedure will be described here.
Procedure: If test_topic were mapped to brokers 1,2,3, optimum partition ownership, leadership and replica distribution can be ensured by passing the current configuration in unchanged with the--force-rebuild option:
$ topicmappr rebuild --topics test_topic --brokers 1,2,3 --force-rebuild
NOTE: this is a 'full storage rebalance' and moves all partitions. See the rebalance command for limited movement of partitions from the most to least utilized brokers.
Use case: Ensuring even topic storage utilization over a set of brokers.
Storage Rebalancing is performed using an algorithm based first-fit descending bin-packing and requires both metrics data on partition sizes and broker storage. See the Storage Placement section along with the metricsfetcher reference tool for further details. Storage rebalancing can be done in-place or over new brokers in a similar fashion to Scaling Up with the same tradeoffs (an in-place rebalance can be performed with the following procedure by maintaining the existing broker list and including the --force-rebuild option). A procedure using metricsfetcher and a new broker list will be described here.
Procedure: If test_topic were mapped over brokers 1,2,3,4,5 in a way that lead to high storage utilization on some brokers and low utilization other brokers, a storage-balanced mapping can be generated for the new set of brokers 6,7,8,9,10. When the mapping is applied, test_topic will be moved to the new broker set in a way that ensures even storage balance across brokers while honoring locality (rack.id) placement constraints.
First, recent metrics data (the last 3600s) is populated into ZooKeeper via metricsfetcher:
$ metricsfetcher --broker-storage-query "avg:system.disk.free{service:kafka-1,device:/data}" \
--partition-size-query "max:kafka.log.partition.size{service:kafka-1,topic:test_topic} by {topic,partition}" \
--span 3600
Next, topicmappr is ran with the new broker list and the --placement param switched from the default count method to storage, which makes decisions using the stored metrics data:
$ topicmappr rebuild --topics "test_topic" --brokers 6,7,8,9,10 --placement storage
Once applied, the topic will be migrated to the new broker set with the storage rebalance in effect:
While running any of the above operations, it's possible to finally optimize each broker's leader to follower ratio using the --optimize-leadership flag. This option is always applied as a final step on any output partition map and works by iterating over each replica set, sorting the brokers by their leader/follower ratios ascending. This helps even out network bandwidth whether you're using the count placement (assuming somewhat even writes among partitions) or the storage placement strategy. In the latter, partition sizes are coupled to throughput since partition size is a function of retention and throughput; if brokers own even volumes of partitions (rather than counts) and leadership is well distributed, the network flow should be well distributed.
The goal of this option is to approach a 50/50 split of leadership vs follower counts for all brokers. For example, if a broker owns 30 partitions, the broker should hold a leader role for 15 of those partitions and be a follower for the other 15. Take note that due to placement constraints (i.e. rack awareness) or a low ratio of partitions to brokers may limit the optimality of the --optimize-leadership flag.
It's also possible to run a leadership optimization without any other changes to the partition map. By default, if topicmappr's input is both a topic and the exact brokers housing the topic (this can be automatically done by specifying -1 as value to the --brokers flag), the operation is a no-op but to return the current partition map. By introducing the --optimize-leaders flag to an otherwise no-op rebuild, you can perform a leadership optimization.
No-op rebuild:
$ topicmappr rebuild --topics test-topic --brokers -1
Topics:
test-topic
Broker change summary:
Replacing 0, added 0, missing 0, total count changed by 0
Action:
no-op
Partition map changes:
test-topic p0: [1018 1004] -> [1018 1004] no-op
test-topic p1: [1017 1015] -> [1017 1015] no-op
test-topic p2: [1004 1020] -> [1004 1020] no-op
[truncated]
Broker distribution:
degree [min/max/avg]: 6/10/8.80 -> 6/10/8.80
-
Broker 1001 - leader: 113, follower: 2, total: 115
Broker 1002 - leader: 5, follower: 3, total: 8
Broker 1003 - leader: 46, follower: 69, total: 115
Broker 1004 - leader: 8, follower: 9, total: 17
Broker 1005 - leader: 51, follower: 74, total: 125
Broker 1006 - leader: 49, follower: 66, total: 115
Broker 1007 - leader: 8, follower: 3, total: 11
Broker 1009 - leader: 55, follower: 54, total: 109
Broker 1012 - leader: 53, follower: 64, total: 117
Broker 1013 - leader: 54, follower: 55, total: 109
Broker 1014 - leader: 55, follower: 61, total: 116
Broker 1015 - leader: 48, follower: 65, total: 113
Broker 1017 - leader: 54, follower: 56, total: 110
Broker 1018 - leader: 65, follower: 45, total: 110
Broker 1020 - leader: 61, follower: 49, total: 110
WARN:
[none]
New partition maps:
test-topic.json
Rebuild with leadership optimization:
$ topicmappr rebuild --topics test-topic --brokers -1 --optimize-leadership
Topics:
test-topic
Broker change summary:
Replacing 0, added 0, missing 0, total count changed by 0
Action:
Optimizing leader/follower ratios
Partition map changes:
test-topic p0: [1018 1004] -> [1018 1004] no-op
test-topic p1: [1017 1015] -> [1015 1017] preferred leader
test-topic p2: [1004 1020] -> [1004 1020] no-op
[truncated]
Broker distribution:
degree [min/max/avg]: 6/10/8.80 -> 6/10/8.80
-
Broker 1001 - leader: 76, follower: 39, total: 115
Broker 1002 - leader: 5, follower: 3, total: 8
Broker 1003 - leader: 58, follower: 57, total: 115
Broker 1004 - leader: 9, follower: 8, total: 17
Broker 1005 - leader: 63, follower: 62, total: 125
Broker 1006 - leader: 58, follower: 57, total: 115
Broker 1007 - leader: 6, follower: 5, total: 11
Broker 1009 - leader: 55, follower: 54, total: 109
Broker 1012 - leader: 59, follower: 58, total: 117
Broker 1013 - leader: 55, follower: 54, total: 109
Broker 1014 - leader: 58, follower: 58, total: 116
Broker 1015 - leader: 57, follower: 56, total: 113
Broker 1017 - leader: 56, follower: 54, total: 110
Broker 1018 - leader: 55, follower: 55, total: 110
Broker 1020 - leader: 55, follower: 55, total: 110
WARN:
[none]
New partition maps:
test-topic.json
In the second example, the ratio of leadership to follower positions per broker is optimized as best as possible given all other constraints that must be satisfied.