You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Server should send to the client the data in form of collections of host_name, current_time, thread_id, type, name, value records.
type is either increment or gauge.
This data contains info from ProfileEvents (in form of increments) and from MemoryTracker (in form of gauge).
The data is sent at some intervals similarly to the Progress packets.
New type of packet will be introduced to send this data and it will be sent only to supporting clients (by checking the client version).
In distributed query processing, every server sends data about itself and also forwards all already received data from other servers. The server does not sum increments but only sends (forwards) them to the client (but integration during intervals between consecutive sends is ok). It also does not sum data neither by hosts or by threads. It is responsibility of the final client application (clickhouse-client or similar) to interpret and possibly sum increments. The gauges and increments are sent if they are non-zero.
The data of some metrics can be not per-thread, then thread_id = 0 is provided.
The data will be used for the following purposes:
display some metrics in clickhouse-client in realtime (e.g. total number of CPU cores participated in query processing will look cool and help for users of a cluster to maintain awareness of the vast amount of computing resources);
extend quotas to allow arbitrary metrics to limit query complexity (imagine you want to limit the amount of page faults in 5 minutes interval for some user - not a practical use case but you've got the idea);
Note that all these data is already available in the query_log and query_thread_log.
Additional notes
The implementation can be somewhat similar to the logs sending. The data can be serialized in form of a Block.
We can also introduce another system log that will contain these values. E.g. to draw a graph of the metrics of selected query with fine-grained resolution.
A small refactoring of ThreadStatus/ThreadGroupStatus will be needed to obtain the values of ProfileEvents from every thread.
The text was updated successfully, but these errors were encountered:
Server should send to the client the data in form of collections of
host_name, current_time, thread_id, type, name, value
records.type is either
increment
orgauge
.This data contains info from ProfileEvents (in form of increments) and from MemoryTracker (in form of gauge).
The data is sent at some intervals similarly to the
Progress
packets.New type of packet will be introduced to send this data and it will be sent only to supporting clients (by checking the client version).
In distributed query processing, every server sends data about itself and also forwards all already received data from other servers. The server does not sum increments but only sends (forwards) them to the client (but integration during intervals between consecutive sends is ok). It also does not sum data neither by hosts or by threads. It is responsibility of the final client application (clickhouse-client or similar) to interpret and possibly sum increments. The gauges and increments are sent if they are non-zero.
The data of some metrics can be not per-thread, then thread_id = 0 is provided.
The data will be used for the following purposes:
Note that all these data is already available in the
query_log
andquery_thread_log
.Additional notes
The implementation can be somewhat similar to the logs sending. The data can be serialized in form of a Block.
We can also introduce another system log that will contain these values. E.g. to draw a graph of the metrics of selected query with fine-grained resolution.
A small refactoring of ThreadStatus/ThreadGroupStatus will be needed to obtain the values of ProfileEvents from every thread.
The text was updated successfully, but these errors were encountered: