PoC for Vacuum monitoring was implemented in
Everyone can take a look at changes and test it in PMM by using the following images
My inputs for further steps
I think it'll be good to have
- Collector of those metrics with additional parameters for databases and tables that have more than 1000 tables
- Labels like oid, table, schema, and database can produce high cardinality issues in the future on large-scale databases with many tables. Or many clusters with many tables.
We also discussed that Postgres exporter needs to work with PG connections more gentle because PG creates one process per connection and in the case of 50 databases PG exporter needs to open 50 connections (I'm not sure about this point)
When it comes to autovacuum, the ultimate source of information is the PostgreSQL logs.
Generally every PostgreSQL user will set log_autovacuum_min_duration parameter and capture the details to PostgreSQL logs.
It is important to understand which tables are becoming too frequent candidate for autovacuum and the load it creates.
- We need to be careful with the load will be added to the database we are monitoring. Few comments above databases with large number of tables are mentioned but there are other cases, for example, an instance with many databases. Postgres requires 1 connection per database, and the more databases in the same instance, the more connections PMM will open. It's a global problem for PMM and PG but can be even more harmful here;
- Some queries can take longer depending on the number of objects (schemas, tables, etc) in the database and the 1k table cap may not solve the issue;
- This is great to start the vacuum monitoring but we need start thinking about other sources of information, for example the one mentioned by Jobin, the log files. It may be for another POC or features updates but are they in the road map?
- Configuration and granularity: Would it be possible for the user to have different granularity of the collected information? For example, would it be possible to enable the collector only for one specific database (the similar to schema in MySQL), and or disable for a list of them? Maybe add for future improvements?