Details
-
Bug
-
Status: Open
-
Medium
-
Resolution: Unresolved
-
2.1.0
-
None
-
3
Description
Originally reported on percona forum,
Even after configuring pmm-server to run on port 80 and also configuring pmm client to use port 80, In /usr/local/percona/pmm2/config/pmm-agent.yaml and pmm-admin status output it shows port 443 for PMM server.
Test Example:
# docker run -d -p 80:80 --volumes-from pmm-data --name pmm-server --restart always percona/pmm-server:2.1.0 9a88bdb76fdc61407536c0306aec08e9c4567147709cae7f7cd5e8cd8e2d8aab # pmm-admin config --server-insecure-tls --server-url=https://admin:[email protected]:80 Checking local pmm-agent status... pmm-agent is running. Registering pmm-agent on PMM Server... Failed to register pmm-agent on PMM Server: Post https://172.17.0.2:80/v1/management/Node/Register: http: server gave HTTP response to HTTPS client. # pmm-admin config --server-url=http://admin:[email protected]:80 Warning: PMM Server requires TLS communications with client. Checking local pmm-agent status... pmm-agent is running. Registering pmm-agent on PMM Server... Registered. Configuration file /usr/local/percona/pmm2/config/pmm-agent.yaml updated. Reloading pmm-agent configuration... Configuration reloaded. Checking local pmm-agent status... pmm-agent is running.
As we can see *Warning: PMM Server requires TLS communications with the client. * Not an error.
But the fact it is still using 443 port and not port 80, I confirm the same by looking at the network traffic for PMM host and port via tcpdump
$ sudo tcpdump -i any -nn host 172.17.0.2
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
10:58:36.756367 IP 172.17.0.2.58118 > 192.168.0.128.42000: Flags [P.], seq 2917560316:2917560680, ack 1255062762, win 471, options [nop,nop,TS val 1774791959 ecr 1170434366], length 364
10:58:36.756398 IP 172.17.0.2.58118 > 192.168.0.128.42000: Flags [P.], seq 0:364, ack 1, win 471, options [nop,nop,TS val 1774791959 ecr 1170434366], length 364
10:58:36.762740 IP 192.168.0.128.42000 > 172.17.0.2.58118: Flags [P.], seq 1:575, ack 364, win 461, options [nop,nop,TS val 1170439366 ecr 1774791959], length 574
10:58:36.762753 IP 192.168.0.128.42000 > 172.17.0.2.58118: Flags [P.], seq 1:575, ack 364, win 461, options [nop,nop,TS val 1170439366 ecr 1774791959], length 574
10:58:36.762797 IP 172.17.0.2.58118 > 192.168.0.128.42000: Flags [.], ack 575, win 480, options [nop,nop,TS val 1774791965 ecr 1170439366], length 0
10:58:36.762801 IP 172.17.0.2.58118 > 192.168.0.128.42000: Flags [.], ack 575, win 480, options [nop,nop,TS val 1774791965 ecr 1170439366], length 0
10:58:37.543502 IP 172.17.0.2.443 > 172.17.0.1.51956: Flags [P.], seq 3023383317:3023383364, ack 1569860736, win 243, options [nop,nop,TS val 3833582058 ecr 2608746619], length 47
10:58:37.543523 IP 172.17.0.2.443 > 172.17.0.1.51956: Flags [P.], seq 0:47, ack 1, win 243, options [nop,nop,TS val 3833582058 ecr 2608746619], length 47
10:58:37.543607 IP 172.17.0.1.51956 > 172.17.0.2.443: Flags [.], ack 47, win 307, options [nop,nop,TS val 2608756571 ecr 3833582058], length 0
10:58:37.543615 IP 172.17.0.1.51956 > 172.17.0.2.443: Flags [.], ack 47, win 307, options [nop,nop,TS val 2608756571 ecr 3833582058], length 0
10:58:37.544000 IP 172.17.0.1.51956 > 172.17.0.2.443: Flags [P.], seq 1:60, ack 47, win 307, options [nop,nop,TS val 2608756572 ecr 3833582058], length 59
$ sudo tcpdump -i any -nn port 443
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
10:57:37.543032 IP 172.17.0.2.443 > 172.17.0.1.51956: Flags [P.], seq 3023382507:3023382554, ack 1569860045, win 243, options [nop,nop,TS val 3833522058 ecr 2608686573], length 47
10:57:37.543053 IP 172.17.0.2.443 > 172.17.0.1.51956: Flags [P.], seq 0:47, ack 1, win 243, options [nop,nop,TS val 3833522058 ecr 2608686573], length 47
10:57:37.543569 IP 172.17.0.1.51956 > 172.17.0.2.443: Flags [P.], seq 1:92, ack 47, win 307, options [nop,nop,TS val 2608696572 ecr 3833522058], length 91
10:57:37.543580 IP 172.17.0.1.51956 > 172.17.0.2.443: Flags [P.], seq 1:92, ack 47, win 307, options [nop,nop,TS val 2608696572 ecr 3833522058], length 91
10:57:37.543824 IP 172.17.0.2.443 > 172.17.0.1.51956: Flags [P.], seq 47:93, ack 92, win 243, options [nop,nop,TS val 3833522059 ecr 2608696572], length 46
10:57:37.543839 IP 172.17.0.2.443 > 172.17.0.1.51956: Flags [P.], seq 47:93, ack 92, win 243, options [nop,nop,TS val 3833522059 ecr 2608696572], length 46
10:57:37.544052 IP 172.17.0.2.443 > 172.17.0.1.51956: Flags [P.], seq 93:135, ack 92, win 243, options [nop,nop,TS val 3833522059 ecr 2608696572], length 42
10:57:37.544058 IP 172.17.0.2.443 > 172.17.0.1.51956: Flags [P.], seq 93:135, ack 92, win 243, options [nop,nop,TS val 3833522059 ecr 2608696572], length 42
10:57:37.544283 IP 172.17.0.1.51956 > 172.17.0.2.443: Flags [.], ack 135, win 307, options [nop,nop,TS val 2608696572 ecr 3833522059], length 0
10:57:37.544308 IP 172.17.0.1.51956 > 172.17.0.2.443: Flags [.], ack 135, win 307, options [nop,nop,TS val 2608696572 ecr 3833522059], length 0
# cat /usr/local/percona/pmm2/config/pmm-agent.yaml # Updated by `pmm-agent setup`. --- id: /agent_id/033d7bc4-dfa1-4200-b3e9-1319ca3c1262 listen-port: 7777 server: address: 172.17.0.2:443 username: admin password: admin insecure-tls: true paths: exporters_base: /usr/local/percona/pmm2/exporters node_exporter: /usr/local/percona/pmm2/exporters/node_exporter mysqld_exporter: /usr/local/percona/pmm2/exporters/mysqld_exporter mongodb_exporter: /usr/local/percona/pmm2/exporters/mongodb_exporter postgres_exporter: /usr/local/percona/pmm2/exporters/postgres_exporter proxysql_exporter: /usr/local/percona/pmm2/exporters/proxysql_exporter tempdir: /tmp ports: min: 42000 max: 51999 debug: false trace: false
# pmm-admin status Agent ID: /agent_id/033d7bc4-dfa1-4200-b3e9-1319ca3c1262 Node ID : /node_id/6b6a765b-d244-4cc4-8e9f-f24de08440f1 PMM Server: URL : https://172.17.0.2:443/ Version: 2.1.0 PMM-agent: Connected : true Time drift: 65.458µs Latency : 180.601µs Agents: /agent_id/414dae03-5c6d-4397-9174-ed5c35a7b576 MYSQLD_EXPORTER RUNNING /agent_id/7febe587-5c9f-449c-8a16-aa905dcaa188 NODE_EXPORTER RUNNING /agent_id/efff6f0d-9ee0-49a7-adeb-42d527963630 QAN_MYSQL_SLOWLOG_AGENT WAITING
2nd Example: with port 555
# pmm-admin config --server-insecure-tls --server-url=https://admin:[email protected]:555 Checking local pmm-agent status... pmm-agent is running. Registering pmm-agent on PMM Server... Failed to register pmm-agent on PMM Server: Post https://172.17.0.2:555/v1/management/Node/Register: dial tcp 172.17.0.2:555: connect: connection refused. # docker run -d -p 555:555 --volumes-from pmm-data --name pmm-server --restart always percona/pmm-server:2.1.0 92bf21a2387fd5a8bda70c66f1c59f6c8f7e62a63a616c1a3b5ba7101bfa9510 # pmm-admin config --server-insecure-tls --server-url=http://admin:[email protected]:555 Warning: PMM Server requires TLS communications with client. Checking local pmm-agent status... pmm-agent is running. Registering pmm-agent on PMM Server... Registered. Configuration file /usr/local/percona/pmm2/config/pmm-agent.yaml updated. Reloading pmm-agent configuration... Configuration reloaded. Checking local pmm-agent status... pmm-agent is running.
Same issue in this case, pmm still using 443 port for communication.
Since pmm-client configured sucessfully user may under impression that it's using the custom-configured port and not 443. If PMM switching to HTTPS connection over 443 port then the Warning message should be more clear or it should not configure pmm client and should throw an error with a valid message.
As the above test shows, no matter whichever port we configured to use with DOCKER pmm-server or in pmm-admin config --server-url, pmm will always use port 443 for data transfer between pmm server and client.
Expected Behaviour:
Here the expectation is to have consistent info in configuration and clarity if it's using configured port by a user with HTTP or port 443 with HHTPS. If PMM switching to TLS connection over 443 port then the Warning message should be more clear or it should not configure pmm client and should throw an error with a valid message.
And valid behavior should be documented.
Currently as per PMM 2 documentation
https://www.percona.com/doc/percona-monitoring-and-management/2.x/concepts/architecture.html
To make data transfer from PMM Client to PMM Server secure, all exporters are able to use SSL/TLS encrypted connections, and their communication with the PMM server is protected by the HTTP basic authentication.