You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Proposal] Get the hot spot of the doris cluster. #3517
Nowadays, We have no insight about the hotspot the cluster.
During the time, we don't know the scanner rows of one table during time.
Now we should collect the scanner rows of the table and scanner bytes of the table for query.
Using this information, we can got the insight about the hot spot of cluster.
Supposing there are 10000 tablets in one BE node and collecting metrics at every 15 seconds.
Every time we will gather 10000 * 200 Bytes = 2MB, one hundred nodes will collect 200MB, this will be consume huge bandwidth. And also user will take care of table granularity than tablet. So we will collect table metrics instead of tablet metrics.
For metric in one BE node, I will use a map to store the information.
There also exists another function using log.
If every query collecting the tablet scanner rows/bytes, and write it to log.
We can access the log to get the message, it needs log analysis which is not integrated in Doris.
So I will use the above method to solve this problem.
Activity
chaoyli commentedon May 8, 2020
The metric is like this
Supposing there are 10000 tablets in one BE node and collecting metrics at every 15 seconds.
Every time we will gather 10000 * 200 Bytes = 2MB, one hundred nodes will collect 200MB, this will be consume huge bandwidth. And also user will take care of table granularity than tablet. So we will collect table metrics instead of tablet metrics.
For metric in one BE node, I will use a map to store the information.
The incremental metric value should be calculated at every collecting interval.
Like this
chaoyli commentedon May 8, 2020
There also exists another function using log.
If every query collecting the tablet scanner rows/bytes, and write it to log.
We can access the log to get the message, it needs log analysis which is not integrated in Doris.
So I will use the above method to solve this problem.
imay commentedon May 8, 2020
This is a good job.
However I think something maybe discussed clearly before writing code.
chaoyli commentedon May 9, 2020
morningman commentedon May 9, 2020
I think we could refer to
https://pingcap.com/blog-cn/observability-of-distributed-system/and
https://cloud.google.com/bigtable/docs/keyvis-overviewto learn something