Details

    • 3

    Description

      Originally reported on percona forum,

      https://www.percona.com/forums/questions-discussions/percona-monitoring-and-management/55004-pm2-client-tries-443-instead-of-80

      Even after configuring pmm-server to run on port 80 and also configuring pmm client to use port 80, In /usr/local/percona/pmm2/config/pmm-agent.yaml  and pmm-admin status output it shows port 443 for PMM server.

      Test Example:

      # docker run -d -p 80:80 --volumes-from pmm-data --name pmm-server --restart always percona/pmm-server:2.1.0
      9a88bdb76fdc61407536c0306aec08e9c4567147709cae7f7cd5e8cd8e2d8aab
      
      # pmm-admin config  --server-insecure-tls --server-url=https://admin:[email protected]:80
      Checking local pmm-agent status...
      pmm-agent is running.
      Registering pmm-agent on PMM Server...
      Failed to register pmm-agent on PMM Server: Post https://172.17.0.2:80/v1/management/Node/Register: http: server gave HTTP response to HTTPS client.
      
      # pmm-admin config   --server-url=http://admin:[email protected]:80
      Warning: PMM Server requires TLS communications with client.
      Checking local pmm-agent status...
      pmm-agent is running.
      Registering pmm-agent on PMM Server...
      Registered.
      Configuration file /usr/local/percona/pmm2/config/pmm-agent.yaml updated.
      Reloading pmm-agent configuration...
      Configuration reloaded.
      Checking local pmm-agent status...
      pmm-agent is running.
      

      As we can see *Warning: PMM Server requires TLS communications with the client. * Not an error.

      But the fact it is still using 443 port and not port 80, I confirm the same by looking at the network traffic for PMM host and port via tcpdump

      $ sudo tcpdump -i any  -nn host 172.17.0.2
      tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
      listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
      10:58:36.756367 IP 172.17.0.2.58118 > 192.168.0.128.42000: Flags [P.], seq 2917560316:2917560680, ack 1255062762, win 471, options [nop,nop,TS val 1774791959 ecr 1170434366], length 364
      10:58:36.756398 IP 172.17.0.2.58118 > 192.168.0.128.42000: Flags [P.], seq 0:364, ack 1, win 471, options [nop,nop,TS val 1774791959 ecr 1170434366], length 364
      10:58:36.762740 IP 192.168.0.128.42000 > 172.17.0.2.58118: Flags [P.], seq 1:575, ack 364, win 461, options [nop,nop,TS val 1170439366 ecr 1774791959], length 574
      10:58:36.762753 IP 192.168.0.128.42000 > 172.17.0.2.58118: Flags [P.], seq 1:575, ack 364, win 461, options [nop,nop,TS val 1170439366 ecr 1774791959], length 574
      10:58:36.762797 IP 172.17.0.2.58118 > 192.168.0.128.42000: Flags [.], ack 575, win 480, options [nop,nop,TS val 1774791965 ecr 1170439366], length 0
      10:58:36.762801 IP 172.17.0.2.58118 > 192.168.0.128.42000: Flags [.], ack 575, win 480, options [nop,nop,TS val 1774791965 ecr 1170439366], length 0
      10:58:37.543502 IP 172.17.0.2.443 > 172.17.0.1.51956: Flags [P.], seq 3023383317:3023383364, ack 1569860736, win 243, options [nop,nop,TS val 3833582058 ecr 2608746619], length 47
      10:58:37.543523 IP 172.17.0.2.443 > 172.17.0.1.51956: Flags [P.], seq 0:47, ack 1, win 243, options [nop,nop,TS val 3833582058 ecr 2608746619], length 47
      10:58:37.543607 IP 172.17.0.1.51956 > 172.17.0.2.443: Flags [.], ack 47, win 307, options [nop,nop,TS val 2608756571 ecr 3833582058], length 0
      10:58:37.543615 IP 172.17.0.1.51956 > 172.17.0.2.443: Flags [.], ack 47, win 307, options [nop,nop,TS val 2608756571 ecr 3833582058], length 0
      10:58:37.544000 IP 172.17.0.1.51956 > 172.17.0.2.443: Flags [P.], seq 1:60, ack 47, win 307, options [nop,nop,TS val 2608756572 ecr 3833582058], length 59
      
      
      $ sudo tcpdump -i any -nn port 443
      tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
      listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
      10:57:37.543032 IP 172.17.0.2.443 > 172.17.0.1.51956: Flags [P.], seq 3023382507:3023382554, ack 1569860045, win 243, options [nop,nop,TS val 3833522058 ecr 2608686573], length 47
      10:57:37.543053 IP 172.17.0.2.443 > 172.17.0.1.51956: Flags [P.], seq 0:47, ack 1, win 243, options [nop,nop,TS val 3833522058 ecr 2608686573], length 47
      10:57:37.543569 IP 172.17.0.1.51956 > 172.17.0.2.443: Flags [P.], seq 1:92, ack 47, win 307, options [nop,nop,TS val 2608696572 ecr 3833522058], length 91
      10:57:37.543580 IP 172.17.0.1.51956 > 172.17.0.2.443: Flags [P.], seq 1:92, ack 47, win 307, options [nop,nop,TS val 2608696572 ecr 3833522058], length 91
      10:57:37.543824 IP 172.17.0.2.443 > 172.17.0.1.51956: Flags [P.], seq 47:93, ack 92, win 243, options [nop,nop,TS val 3833522059 ecr 2608696572], length 46
      10:57:37.543839 IP 172.17.0.2.443 > 172.17.0.1.51956: Flags [P.], seq 47:93, ack 92, win 243, options [nop,nop,TS val 3833522059 ecr 2608696572], length 46
      10:57:37.544052 IP 172.17.0.2.443 > 172.17.0.1.51956: Flags [P.], seq 93:135, ack 92, win 243, options [nop,nop,TS val 3833522059 ecr 2608696572], length 42
      10:57:37.544058 IP 172.17.0.2.443 > 172.17.0.1.51956: Flags [P.], seq 93:135, ack 92, win 243, options [nop,nop,TS val 3833522059 ecr 2608696572], length 42
      10:57:37.544283 IP 172.17.0.1.51956 > 172.17.0.2.443: Flags [.], ack 135, win 307, options [nop,nop,TS val 2608696572 ecr 3833522059], length 0
      10:57:37.544308 IP 172.17.0.1.51956 > 172.17.0.2.443: Flags [.], ack 135, win 307, options [nop,nop,TS val 2608696572 ecr 3833522059], length 0
      
      # cat /usr/local/percona/pmm2/config/pmm-agent.yaml
      # Updated by `pmm-agent setup`.
      ---
      id: /agent_id/033d7bc4-dfa1-4200-b3e9-1319ca3c1262
      listen-port: 7777
      server:
        address: 172.17.0.2:443
        username: admin
        password: admin
        insecure-tls: true
      paths:
        exporters_base: /usr/local/percona/pmm2/exporters
        node_exporter: /usr/local/percona/pmm2/exporters/node_exporter
        mysqld_exporter: /usr/local/percona/pmm2/exporters/mysqld_exporter
        mongodb_exporter: /usr/local/percona/pmm2/exporters/mongodb_exporter
        postgres_exporter: /usr/local/percona/pmm2/exporters/postgres_exporter
        proxysql_exporter: /usr/local/percona/pmm2/exporters/proxysql_exporter
        tempdir: /tmp
      ports:
        min: 42000
        max: 51999
      debug: false
      trace: false
      
      # pmm-admin status
      Agent ID: /agent_id/033d7bc4-dfa1-4200-b3e9-1319ca3c1262
      Node ID : /node_id/6b6a765b-d244-4cc4-8e9f-f24de08440f1
      
      PMM Server:
      	URL    : https://172.17.0.2:443/
      	Version: 2.1.0
      
      PMM-agent:
      	Connected : true
      	Time drift: 65.458µs
      	Latency   : 180.601µs
      
      Agents:
      	/agent_id/414dae03-5c6d-4397-9174-ed5c35a7b576 MYSQLD_EXPORTER RUNNING
      	/agent_id/7febe587-5c9f-449c-8a16-aa905dcaa188 NODE_EXPORTER RUNNING
      	/agent_id/efff6f0d-9ee0-49a7-adeb-42d527963630 QAN_MYSQL_SLOWLOG_AGENT WAITING
      

       
      2nd Example: with port 555

      # pmm-admin config --server-insecure-tls --server-url=https://admin:[email protected]:555
      Checking local pmm-agent status...
      pmm-agent is running.
      Registering pmm-agent on PMM Server...
      Failed to register pmm-agent on PMM Server: Post https://172.17.0.2:555/v1/management/Node/Register: dial tcp 172.17.0.2:555: connect: connection refused.
      
      # docker run -d -p 555:555 --volumes-from pmm-data --name pmm-server --restart always percona/pmm-server:2.1.0
      92bf21a2387fd5a8bda70c66f1c59f6c8f7e62a63a616c1a3b5ba7101bfa9510
      
      # pmm-admin config --server-insecure-tls --server-url=http://admin:[email protected]:555
      Warning: PMM Server requires TLS communications with client.
      Checking local pmm-agent status...
      pmm-agent is running.
      Registering pmm-agent on PMM Server...
      Registered.
      Configuration file /usr/local/percona/pmm2/config/pmm-agent.yaml updated.
      Reloading pmm-agent configuration...
      Configuration reloaded.
      Checking local pmm-agent status...
      pmm-agent is running.
      

      Same issue in this case, pmm still using 443 port for communication.

      Since pmm-client configured sucessfully user may under impression that it's using the custom-configured port and not 443.  If PMM switching to HTTPS connection over 443 port then the Warning message should be more clear or it should not configure pmm client and should throw an error with a valid message.

      As the above test shows, no matter whichever port we configured to use with DOCKER pmm-server or in pmm-admin config --server-url, pmm will always use port 443 for data transfer between pmm server and client.

       

      Expected Behaviour:

      Here the expectation is to have consistent info in configuration and clarity if it's using configured port by a user with HTTP or port 443 with HHTPS. If PMM switching to TLS connection over 443 port then the Warning message should be more clear or it should not configure pmm client and should throw an error with a valid message.

      And valid behavior should be documented. 

      Currently as per PMM 2 documentation
      https://www.percona.com/doc/percona-monitoring-and-management/2.x/concepts/architecture.html

      To make data transfer from PMM Client to PMM Server secure, all exporters are able to use SSL/TLS encrypted connections, and their communication with the PMM server is protected by the HTTP basic authentication.

      Attachments

        Activity

          People

            denys.kondratenko Denys Kondratenko
            lalit.choudhary Lalit Choudhary
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:

              Smart Checklist