-
Notifications
You must be signed in to change notification settings - Fork 3.2k
Statistics
Thijs Cadier edited this page Mar 2, 2017
·
25 revisions
librdkafka optionally emits internal metrics every statistics.interval.ms if a stats_cb has been registered.
The stats are provided as a JSON object string.
Note: This page documents the metrics available in librdkafka master, some fields may be missing in older versions of librdkafka.
All fields that contain sizes are are in bytes unless otherwise noted.
{
<Top-level fields>
"brokers": {
<brokers fields>,
"toppars": { <toppars fields> }
},
"topics": {
<topic fields>,
"partitions": {
<partitions fields>
}
}
[, "cgrp": { <cgrp fields> } ]
}
| Field | Type | Example | Description |
|---|---|---|---|
| name | string | "rdkafka#producer-1" |
Handle instance name |
| type | string | "producer" |
Instance type (producer or consumer) |
| ts | int | 12345678912345 | librdkafka's internal monotonic clock (micro seconds) |
| time | int | Wall clock time in seconds since the epoch | |
| replyq | int | Number of ops waiting in queue for application to serve with rd_kafka_poll() | |
| msg_cnt | int | Current number of messages in instance queues | |
| msg_size | int | Current total size of messages in instance queues | |
| msg_max | int | Threshold: maximum number of messages allowed | |
| msg_size_max | int | Threshold: maximum total size of messages allowed | |
| simple_cnt | int | Internal tracking of legacy vs new consumer API state | |
| brokers | object | Dict of brokers, key is broker name, value is object. See brokers below | |
| topics | object | Dict of topics, key is topic name, value is object. See topics below | |
| cgrp | object | Consumer group metrics. See cgrp below |
| Field | Type | Example | Description |
|---|---|---|---|
| name | string | "example.com:9092/13" |
Broker hostname, port and broker id |
| nodeid | int | 13 | Broker id (-1 for bootstraps) |
| state | string | Broker state | |
| stateage | int | Time since last broker state change (microseconds) | |
| outbuf_cnt | int | Number of requests awaiting transmission to broker | |
| outbuf_msg_cnt | int | Number of messages in outbuf_cnt | |
| waitresp_cnt | int | Number of requests in-flight to broker awaiting response | |
| waitresp_msg_cnt | int | Number of messages in waitresp_cnt | |
| tx | int | Total number of requests sent | |
| txbytes | int | Total number of bytes sent | |
| txerrs | int | Total number of transmission errors | |
| txretries | int | Total number of request retries | |
| req_timeouts | int | Total number of requests timed out | |
| rx | int | Total number of responses received | |
| rxbytes | int | Total number of bytes received | |
| rxerrs | int | Total number of receive errors | |
| rxcorriderrs | int | Total number of unmatched correlation ids in response (typically for timed out requests) | |
| rxpartial | int | Total number of partial messagesets received | |
| zbuf_grow | int | Total number of decompression buffer size increases | |
| buf_grow | int | Total number of buffer size increases | |
| wakeups | int | Broker thread poll wakeups | |
| int_latency | object | Internal producer queue latency in microseconds. See Window stats below | |
| rtt | object | Broker latency / round-trip time. See Window stats below | |
| throttle | object | Broker throttling time. See Window stats below | |
| toppars | object | Partitions handled by this broker handle. Key is "topic-partition". See brokers.toppars below |
Rolling window statistics
| Field | Type | Example | Description |
|---|---|---|---|
| min | int | Smallest value | |
| max | int | Largest value | |
| avg | int | Average value | |
| sum | int | Sum of values | |
| cnt | int | Number of value samples |
| Field | Type | Example | Description |
|---|---|---|---|
| topic | string | atopic | Topic name |
| partition | int | 3 | Partition id |
| Field | Type | Example | Description |
|---|---|---|---|
| topic | string | atopic | Topic name |
| metadata_age | int | Age of metadata from broker for this topic (milliseconds) | |
| partitions | object | Partitions dict, key is partition id. See partitions below. |
| Field | Type | Example | Description |
|---|---|---|---|
| partition | int | 3 | Partition Id (-1 for internal UA/UnAssigned partition) |
| leader | int | Current leader broker id | |
| desired | bool | Partition is explicitly desired by application | |
| unknown | bool | Partition not seen in topic metadata from broker | |
| msgq_cnt | int | Number of messages waiting to be produced in first-level queue | |
| msgq_bytes | int | Number of bytes in msgq_bytes | |
| xmit_msgq | int | Number of messages ready to be produced in transmit queue | |
| xmit_msgq_bytes | int | Number of bytes in xmit_msgq | |
| fetchq_cnt | int | Number of pre-fetched messages in fetch queue | |
| fetchq_size | int | Bytes in fetchq | |
| fetch_state | string | Consumer fetch state for this partition | |
| query_offset | int | Current/Last logical offset query | |
| next_offset | int | Next offset to fetch | |
| app_offset | int | Offset of last message passed to application | |
| stored_offset | int | Offset to be committed | |
| committed_offset | int | Last committed offset | |
| eof_offset | int | Last PARTITION_EOF signaled offset | |
| lo_offset | int | Partition's low watermark offset on broker | |
| hi_offset | int | Partition's high watermark offset on broker | |
| consumer_lag | int | Difference between hi_offset - app_offset | |
| txmsgs | int | Total number of messages transmitted (produced) | |
| txbytes | int | Total number of bytes transmitted | |
| msgs | int | Total number of messages received (consumed) | |
| rx_ver_drops | int | Dropped outdated messages |
| Field | Type | Example | Description |
|---|---|---|---|
| rebalance_age | int | Time elapsed since last rebalance (assign or revoke) (milliseconds) | |
| rebalance_cnt | int | Total number of rebalances (assign or revoke) | |
| assignment_size | int | Current assignment's partition count |
This example output is from a short-lived high level consumer using the following command:
rdkafka_performance -G myfinegroup -b 0 -t test -o beginning -T 2000 -Y 'cat > stats.json'
{
"name": "rdkafka#consumer-1",
"type": "consumer",
"ts": 895747604205,
"time": 1479659343,
"replyq": 0,
"msg_cnt": 0,
"msg_size": 0,
"msg_max": 0,
"msg_size_max": 0,
"simple_cnt": 0,
"brokers": {
"0:9092/bootstrap": {
"name": "0:9092/bootstrap",
"nodeid": -1,
"state": "UP",
"stateage": 5989882,
"outbuf_cnt": 0,
"outbuf_msg_cnt": 0,
"waitresp_cnt": 0,
"waitresp_msg_cnt": 0,
"tx": 2,
"txbytes": 56,
"txerrs": 0,
"txretries": 0,
"req_timeouts": 0,
"rx": 2,
"rxbytes": 31692,
"rxerrs": 0,
"rxcorriderrs": 0,
"rxpartial": 0,
"zbuf_grow": 0,
"buf_grow": 0,
"rtt": {
"min": 0,
"max": 0,
"avg": 0,
"sum": 0,
"cnt": 0
},
"throttle": {
"min": 0,
"max": 0,
"avg": 0,
"sum": 0,
"cnt": 0
},
"toppars": {}
},
"localhost:9092/2": {
"name": "localhost:9092/2",
"nodeid": 2,
"state": "UP",
"stateage": 5958663,
"outbuf_cnt": 0,
"outbuf_msg_cnt": 0,
"waitresp_cnt": 1,
"waitresp_msg_cnt": 0,
"tx": 54,
"txbytes": 3650,
"txerrs": 0,
"txretries": 0,
"req_timeouts": 0,
"rx": 53,
"rxbytes": 89546,
"rxerrs": 0,
"rxcorriderrs": 0,
"rxpartial": 0,
"zbuf_grow": 0,
"buf_grow": 0,
"rtt": {
"min": 721,
"max": 106064,
"avg": 87530,
"sum": 1925664,
"cnt": 22
},
"throttle": {
"min": 0,
"max": 0,
"avg": 0,
"sum": 0,
"cnt": 19
},
"toppars": {
"test-1": {
"topic": "test",
"partition": 1
}
}
},
"localhost:9094/4": {
"name": "localhost:9094/4",
"nodeid": 4,
"state": "UP",
"stateage": 5958663,
"outbuf_cnt": 0,
"outbuf_msg_cnt": 0,
"waitresp_cnt": 1,
"waitresp_msg_cnt": 0,
"tx": 40,
"txbytes": 3042,
"txerrs": 0,
"txretries": 0,
"req_timeouts": 0,
"rx": 39,
"rxbytes": 87058,
"rxerrs": 0,
"rxcorriderrs": 0,
"rxpartial": 0,
"zbuf_grow": 0,
"buf_grow": 0,
"rtt": {
"min": 100169,
"max": 101198,
"avg": 100730,
"sum": 2014600,
"cnt": 20
},
"throttle": {
"min": 0,
"max": 0,
"avg": 0,
"sum": 0,
"cnt": 20
},
"toppars": {
"test-3": {
"topic": "test",
"partition": 3
},
"test-0": {
"topic": "test",
"partition": 0
}
}
},
"localhost:9093/3": {
"name": "localhost:9093/3",
"nodeid": 3,
"state": "UP",
"stateage": 5958647,
"outbuf_cnt": 0,
"outbuf_msg_cnt": 0,
"waitresp_cnt": 1,
"waitresp_msg_cnt": 0,
"tx": 44,
"txbytes": 2688,
"txerrs": 0,
"txretries": 0,
"req_timeouts": 0,
"rx": 43,
"rxbytes": 90161,
"rxerrs": 0,
"rxcorriderrs": 0,
"rxpartial": 0,
"zbuf_grow": 0,
"buf_grow": 0,
"rtt": {
"min": 99647,
"max": 101254,
"avg": 100612,
"sum": 2012247,
"cnt": 20
},
"throttle": {
"min": 0,
"max": 0,
"avg": 0,
"sum": 0,
"cnt": 20
},
"toppars": {
"test-2": {
"topic": "test",
"partition": 2
}
}
}
},
"topics": {
"test": {
"topic": "test",
"metadata_age": 4957,
"partitions": {
"0": {
"partition": 0,
"leader": 4,
"desired": true,
"unknown": false,
"msgq_cnt": 0,
"msgq_bytes": 0,
"xmit_msgq_cnt": 0,
"xmit_msgq_bytes": 0,
"fetchq_cnt": 0,
"fetchq_size": 0,
"fetch_state": "active",
"query_offset": -2,
"next_offset": 427,
"app_offset": 427,
"stored_offset": 427,
"commited_offset": 427,
"committed_offset": 427,
"eof_offset": 427,
"lo_offset": -1001,
"hi_offset": 427,
"consumer_lag": 0,
"txmsgs": 0,
"txbytes": 0,
"msgs": 0,
"rx_ver_drops": 0
},
"1": {
"partition": 1,
"leader": 2,
"desired": true,
"unknown": false,
"msgq_cnt": 0,
"msgq_bytes": 0,
"xmit_msgq_cnt": 0,
"xmit_msgq_bytes": 0,
"fetchq_cnt": 0,
"fetchq_size": 0,
"fetch_state": "active",
"query_offset": -2,
"next_offset": 436,
"app_offset": 436,
"stored_offset": 436,
"commited_offset": 436,
"committed_offset": 436,
"eof_offset": 436,
"lo_offset": -1001,
"hi_offset": 436,
"consumer_lag": 0,
"txmsgs": 0,
"txbytes": 0,
"msgs": 0,
"rx_ver_drops": 0
},
"2": {
"partition": 2,
"leader": 3,
"desired": true,
"unknown": false,
"msgq_cnt": 0,
"msgq_bytes": 0,
"xmit_msgq_cnt": 0,
"xmit_msgq_bytes": 0,
"fetchq_cnt": 0,
"fetchq_size": 0,
"fetch_state": "active",
"query_offset": -2,
"next_offset": 458,
"app_offset": 458,
"stored_offset": 458,
"commited_offset": 458,
"committed_offset": 458,
"eof_offset": 458,
"lo_offset": -1001,
"hi_offset": 458,
"consumer_lag": 0,
"txmsgs": 0,
"txbytes": 0,
"msgs": 0,
"rx_ver_drops": 0
},
"3": {
"partition": 3,
"leader": 4,
"desired": true,
"unknown": false,
"msgq_cnt": 0,
"msgq_bytes": 0,
"xmit_msgq_cnt": 0,
"xmit_msgq_bytes": 0,
"fetchq_cnt": 0,
"fetchq_size": 0,
"fetch_state": "active",
"query_offset": -2,
"next_offset": 497,
"app_offset": 497,
"stored_offset": 497,
"commited_offset": 497,
"committed_offset": 497,
"eof_offset": 497,
"lo_offset": -1001,
"hi_offset": 497,
"consumer_lag": 0,
"txmsgs": 0,
"txbytes": 0,
"msgs": 0,
"rx_ver_drops": 0
},
"-1": {
"partition": -1,
"leader": -1,
"desired": false,
"unknown": false,
"msgq_cnt": 0,
"msgq_bytes": 0,
"xmit_msgq_cnt": 0,
"xmit_msgq_bytes": 0,
"fetchq_cnt": 0,
"fetchq_size": 0,
"fetch_state": "none",
"query_offset": 0,
"next_offset": 0,
"app_offset": -1001,
"stored_offset": -1001,
"commited_offset": -1001,
"committed_offset": -1001,
"eof_offset": -1001,
"lo_offset": -1001,
"hi_offset": -1001,
"consumer_lag": -1,
"txmsgs": 0,
"txbytes": 0,
"msgs": 0,
"rx_ver_drops": 0
}
}
}
},
"cgrp": {
"rebalance_age": 5251,
"rebalance_cnt": 2,
"assignment_size": 4
}
}