Uploaded image for project: 'Percona Monitoring and Management'
  1. Percona Monitoring and Management
  2. PMM-6153

PMM 2.7.0 inoperable when no Internet connectivity

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Priority: High
    • Resolution: Fixed
    • Affects Version/s: 2.7.0
    • Fix Version/s: 2.8.0
    • Component/s: None
    • Labels:
    • Story Points:
      2
    • Sprint:
      Platform Sprint 19
    • Needs Review:
      Yes
    • Needs QA:
      Yes
    • Needs Packaging:
      No
    • Needs Doc:
      No

      Description

      Issue: If there is no external internet access on pmm-server docker container, then the Nginx process will keep restarting and pmm-client can not communicate to pmm-server.

       

      2020-06-18 08:46:00,251 INFO exited: nginx (exit status 1; not expected)
      2020-06-18 08:46:00,253 INFO spawned: 'nginx' with pid 254
      2020-06-18 08:46:01,738 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

       

      As per the  /srv/logs/nginx.startup.log  following is the error.

       

      nginx: [emerg] host not found in upstream "www.percona.com" in /etc/nginx/conf.d/pmm.conf:179
      nginx: [emerg] host not found in upstream "www.percona.com" in /etc/nginx/conf.d/pmm.conf:179
      nginx: [emerg] host not found in upstream "www.percona.com" in /etc/nginx/conf.d/pmm.conf:179
      nginx: [emerg] host not found in upstream "www.percona.com" in /etc/nginx/conf.d/pmm.conf:179

       

      [root@6c48334b2a23 opt]# curl -I https://www.percona.com/blog/feed/
      curl: (6) Could not resolve host: www.percona.com; Unknown error 

      This is related to change in pmm 2.7.0 for Newsfeed to Home dashboard

      https://github.com/percona/pmm-server/blob/PMM-2.0/nginx.conf#L179

       

      # proxy requests to the Percona news rss feed
       location = /percona-blog/feed {
       auth_request off;
      proxy_pass https://www.percona.com/blog/feed/;
       proxy_set_header User-Agent "$http_user_agent pmm-server/2.x";
       } 
       }
      

       

       

      This bug is a blocker if there is no internet access on a docker container, as workarounds,

      we must enable external internet connectivity. 

      OR

      Use pmm 2.6 until we have fix for this.

       

      Repoduciable test case:

      • Disble external internet on docker.
      • installed pmm-server on docker

       

      # docker create -v /srv --name pmm-data percona/pmm-server:2.7.0 /bin/true
      # docker run -d -p 80:80 -p 443:443 --volumes-from pmm-data --name pmm-server --restart always percona/pmm-server:2.7.0
      # docker inspect pmm-server | grep "IPAddress"
       "SecondaryIPAddresses": null,
       "IPAddress": "172.17.0.2",
       "IPAddress": "172.17.0.2",
      

       

      • you will notice Nginx process keep restarting and error will in nginx.startup.log
      • due to this pmm-client will unable to communicate over port 443
      # pmm-admin config --server-insecure-tls --server-url=https://admin:admin@172.17.0.2:443 
      Checking local pmm-agent status...
      pmm-agent is running.
      Registering pmm-agent on PMM Server...
      Failed to register pmm-agent on PMM Server: Post https://172.17.0.2:443/v1/management/Node/Register: dial tcp 172.17.0.2:443: connect: connection refused.

      Issue not reproduciable with previous pmm2 version , pmm 2.6.0

        Smart Checklist

          Attachments

            Issue Links

              Activity

                People

                Assignee:
                Unassigned
                Reporter:
                lalit.choudhary Lalit Choudhary
                Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                  Dates

                  Created:
                  Updated:
                  Resolved:

                    Time Tracking

                    Estimated:
                    Original Estimate - Not Specified
                    Not Specified
                    Remaining:
                    Remaining Estimate - Not Specified
                    Not Specified
                    Logged:
                    Time Spent - 2 days, 20 minutes
                    2d 20m