Details
-
Bug
-
Status: Done
-
Medium
-
Resolution: Fixed
-
2.13.0
-
None
-
1
-
Yes
-
Yes
-
[obsolete] C/S Core
Description
Issue: Using PMM v2.13.0, based in docker, to monitor ~200 servers with 2-3 services each. Accessing TimeSeries Information panels dashboards return the following error:
"error when executing query=\"topk(10, count({__name__=~\\\".+\\\"}) by (__name__))\" for (time=1610117211741, step=300000): search error after reading 0 data blocks: error when searching for tagFilters=[{__name__=~\".+\"}] on the time range [2021-01-08 14:36:51.741 +0000 UTC - 2021-01-08 14:46:51.741 +0000 UTC]: error when searching tsids: the number of matching unique timeseries exceeds 1500000; either narrow down the search or increase -search.maxUniqueTimeseries"
The issue can be seen for Dashboards,
PMM Home page
Nodes Overview
MySQL Instances Overview
Steps to Reproduce:
The same issue can be seen for 200+ services. tested as follows,
Install PMM_SERVER
docker create -v /srv --name pmm-data percona/pmm-server:2.13.0 /bin/true
docker run -d -p 80:80 -p 443:443 --volumes-from pmm-data --name pmm-server --restart always percona/pmm-server:2.13.0
Configure PMM client
pmm-admin config --server-insecure-tls --server-url=https://admin:[email protected]:443
For testing rather than installing 200+ mysql, start few mysql servers and add them in pmm monitoring with the different service names.
Add it at once:
for i in {1..202} ; do pmm-admin add mysql nodelong_string_something_big_$i 127.0.0.1:3306 --username=msandbox --password=msandbox; done
try to see time-series data for services on the dashboard (MySQL /MySQL Instances Overview)
you will see "error when executing query="
The same will happen for 200+ nodes and related dashboards.
Related Issue: https://github.com/Percona-Lab/pmm-submodules/pull/1374
Actual result: Grafana dashboard gives error while loading when monitoring a large number of nodes/services like 200 to 300.
Expected Result: Grafana dashboard should load without error when monitoring a large number of nodes/services like 200 to 300.
Attachments
Issue Links
- links to