Deploying pmm(ver 2.39.0 server and client) and mongodb(4.4.19) in k8s via helm chart, our large cluster that has 20 replsets and 20 mongos instances leads to a situation where the mongos pmm-clients are not properly monitored. In the inventory, on the services tab, all instances state is N/A. On the node tab, some random number are in the Down state. All replset instances states are fine.
I've tried restarting all pmm-client containers to no avail. Restarting pmm-server has no effect. I've then deleted all mongos instances from the inventory and slowly added them back by restarting their pmm-client one by one, which is how I descovered that 9th client causes the issue. Deleting the 9th client and restarting pmm-server gets the mongos clients to list UP state in the inventory.