Uploaded image for project: 'Percona Monitoring and Management'
  1. Percona Monitoring and Management
  2. PMM-9156

`pmm-agent` paths-base option not working for pmm2-client binary installation in PMM 2.23.0

Details

    • 2
    • Yes
    • Yes
    • [obsolete] C/S Core

    Description

      Issue: pmm-agent paths-base option not working for pmm2-client binary installation in PMM 2.23.0, Starting pmm-agent process gives following error,

       

      /pmm2# ./bin/pmm-agent --config-file=${PMM_DIR}/config/pmm-agent.yaml
      INFO[2021-11-16T19:32:13.702+05:30] time="2021-11-16T19:32:13+05:30" level=error msg="Error reading textfile collector directory \"/usr/local/percona/pmm2/collectors/textfile-collector/high-resolution\": open /usr/local/percona/pmm2/collectors/textfile-collector/high-resolution: no such file or directory" source="textfile.go:225" agentID=/agent_id/a8593487-917d-45f3-929d-86513e38227f component=agent-process type=node_exporter
      

       

      Reproducible Test:

      PMM SERVER 2.23.0

       

      Installing pmm2-client to the custom path:

       

      Download pmm 2.23.0 tar.gz binary
      /pmm2# export PMM_DIR=/pmm2
      /pmm2/pmm2-client-2.23.0# ./install_tarball 
      Installing into /pmm2...
      
      /pmm2# .
      /bin/pmm-agent setup --config-file=${PMM_DIR}/config/pmm-agent.yaml --server-address=172.17.0.2:443 --server-insecure-tls --server-username=admin --server-password=admin --paths-tempdir=${PMM_DIR}/tmp --paths-base=${PMM_DIR} 
      INFO[2021-11-16T19:14:26.900+05:30] Loading configuration file /pmm2/config/pmm-agent.yaml. component=setup
      INFO[2021-11-16T19:14:26.900+05:30] Using /pmm2/exporters/node_exporter component=setup
      INFO[2021-11-16T19:14:26.900+05:30] Using /pmm2/exporters/mysqld_exporter component=setup
      INFO[2021-11-16T19:14:26.900+05:30] Using /pmm2/exporters/mongodb_exporter component=setup
      INFO[2021-11-16T19:14:26.900+05:30] Using /pmm2/exporters/postgres_exporter component=setup
      INFO[2021-11-16T19:14:26.900+05:30] Using /pmm2/exporters/proxysql_exporter component=setup
      INFO[2021-11-16T19:14:26.900+05:30] Using /pmm2/exporters/rds_exporter component=setup
      INFO[2021-11-16T19:14:26.900+05:30] Using /pmm2/exporters/azure_exporter component=setup
      INFO[2021-11-16T19:14:26.900+05:30] Using /pmm2/exporters/vmagent component=setup
      Checking local pmm-agent status...
      pmm-agent is not running.
      Registering pmm-agent on PMM Server...
      Registered.
      Configuration file /pmm2/config/pmm-agent.yaml updated.
      Please start pmm-agent: `pmm-agent --config-file=/pmm2/config/pmm-agent.yaml`.
      /pmm2# cat config/pmm-agent.yaml 
      # Updated by `pmm-agent setup`.
      ---
      ---
      id: /agent_id/4d98f900-ac92-454c-89f8-171da73c5ee6
       
      paths:
       paths_base: /pmm2
       exporters_base: /pmm2/exporters
       node_exporter: /pmm2/exporters/node_exporter
       mysqld_exporter: /pmm2/exporters/mysqld_exporter
       mongodb_exporter: /pmm2/exporters/mongodb_exporter
       postgres_exporter: /pmm2/exporters/postgres_exporter
       proxysql_exporter: /pmm2/exporters/proxysql_exporter
       rds_exporter: /pmm2/exporters/rds_exporter
       azure_exporter: /pmm2/exporters/azure_exporter
       vmagent: /pmm2/exporters/vmagent
       tempdir: /pmm2/tmp
       pt_summary: /pmm2/tools/pt-summary
       pt_pg_summary: /pmm2/tools/pt-pg-summary
       pt_mysql_summary: /pmm2/tools/pt-mysql-summary
       pt_mongodb_summary: /pmm2/tools/pt-mongodb-summary
      

       

       

      Error:

      /pmm2# ./bin/pmm-agent --config-file=${PMM_DIR}/config/pmm-agent.yaml
      INFO[2021-11-16T19:32:13.702+05:30] time="2021-11-16T19:32:13+05:30" level=error msg="Error reading textfile collector directory \"/usr/local/percona/pmm2/collectors/textfile-collector/high-resolution\": open /usr/local/percona/pmm2/collectors/textfile-collector/high-resolution: no such file or directory" source="textfile.go:225" agentID=/agent_id/a8593487-917d-45f3-929d-86513e38227f component=agent-process type=node_exporter
      INFO[2021-11-16T19:32:16.852+05:30] time="2021-11-16T19:32:16+05:30" level=error msg="Error reading textfile collector directory \"/usr/local/percona/pmm2/collectors/textfile-collector/medium-resolution\": open /usr/local/percona/pmm2/collectors/textfile-collector/medium-resolution: no such file or directory" source="textfile.go:225" agentID=/agent_id/a8593487-917d-45f3-929d-86513e38227f component=agent-process type=node_exporter
      INFO[2021-11-16T19:32:18.693+05:30] time="2021-11-16T19:32:18+05:30" level=error msg="Error reading textfile collector directory \"/usr/local/percona/pmm2/collectors/textfile-collector/high-resolution\": open /usr/local/percona/pmm2/collectors/textfile-collector/high-resolution: no such file or directory" source="textfile.go:225" agentID=/agent_id/a8593487-917d-45f3-929d-86513e38227f component=agent-process type=node_exporter
      

       

       

      fmbiete Identified this issue and also found the following finding with workaround, Thank you.

      In pmm-managed DB agent table, pmm version is null for some reason and that causing this issue.

      pmm-managed=# select version from agents where agent_id = '/agent_id/a8593487-917d-45f3-929d-86513e38227f';
       version 
      ---------
       
      (1 row)
      pmm-managed=# select count(*), agent_type from agents where version is null group by agent_type;
       count | agent_type 
      -------+-----------------------------------
       2 | node_exporter
       1 | postgres_exporter
       1 | qan-postgresql-pgstatements-agent
       1 | vmagent
      (4 rows)

       

      Workaround: Updating version in In pmm-managed DB agent table fixs the pmm-agent startup issue on pmm client side.

      ON pmm-server:

      pmm-managed=# update agents set version = '2.23.0' where agent_id = '/agent_id/a8593487-917d-45f3-929d-86513e38227f';
      UPDATE 1
      pmm-managed=# select version from agents where agent_id = '/agent_id/a8593487-917d-45f3-929d-86513e38227f';
       version 
      ---------
       2.23.0
      (1 row)
      

      pmm-client:

      /pmm2# ./bin/pmm-agent --config-file=${PMM_DIR}/config/pmm-agent.yaml
      
      INFO[2021-11-16T19:43:33.159+05:30] 2021-11-16T14:13:33.157Z	info	VictoriaMetrics/lib/promscrape/scraper.go:359	static_configs: added targets: 4, removed targets: 0; total targets: 4  agentID=/agent_id/7e7ff8d2-220d-4fee-9116-ca7b90d12a02 component=agent-process type=vm_agent
      INFO[2021-11-16T19:43:34.129+05:30] Sending status: RUNNING (port 42000).         agentID=/agent_id/7e7ff8d2-220d-4fee-9116-ca7b90d12a02 component=agent-process type=vm_agent
      INFO[2021-11-16T19:43:34.131+05:30] Sending status: RUNNING (port 42001).         agentID=/agent_id/a8593487-917d-45f3-929d-86513e38227f component=agent-process type=node_exporter
      

       

      Attachments

        Issue Links

          Activity

            People

              shashank.sinha Shashank Sinha (Inactive)
              lalit.choudhary Lalit Choudhary
              Votes:
              1 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Smart Checklist