Uploaded image for project: 'Percona Backup for MongoDB'
  1. Percona Backup for MongoDB
  2. PBM-369

Replica sets can't connect to config server when TLS is used

    Details

    • Type: Bug
    • Status: Done
    • Priority: Medium
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: 1.1.0
    • Component/s: Documentation
    • Labels:
      None
    • Environment:

      CentOS Linux release 7.6.1810 (Core) (docker container)

      percona-server-mongodb-server-4.0.12-6.el7.x86_64

      percona-backup-mongodb-1.0.0-1.el7.x86_64

    • Needs QA:
      Yes

      Description

      I've got a MongoDB 4.0.11 cluster set up with routers, a config server, and two replicasets. Strict SSL checking is used, but I've got the same set of certs on each node. pbm-agent is configured using the URI

      PBM_MONGODB_URI=mongodb://pbmAgent:pass@mongo1-cmgo-replica-0.mongo1-cmgo.default.svc.cluster.local:27017/?authSource=admin&replicaSet=mongo1&tls=true&tlsCertificateKeyFile=/data/db/private/client.pem&tlsCAFile=/data/db/private/cacert.pem

      and can talk to the local mongod process. pbm is also talking to the config servers. When I execute the pbm backup command, the backup hangs. Looking at the logs, pdb-agent is failing to connect to the config server:

      2019/11/22 23:37:20 connect to mongodb: create mongo connection to configsvr: mongo ping: server selection error: server selection timeout

      info current topology: Type: Unknown
      info Servers:
      Addr: mongocnf-cmgo-replica-0.mongocnf-cmgo.default.svc.cluster.local:27017, Type: Unknown, State: Connected, Average RTT: 0, Last error: connection() : connection(mongocnf-cmgo-replica-0.mongocnf-cmgo.default.svc.cluster.local:27017[-187]) unable to decode message length: EOF
      23:37:20 docker-entrypoint.sh: LOG:info Addr: mongocnf-cmgo-replica-1.mongocnf-cmgo.default.svc.cluster.local:27017, Type: Unknown, State: Connected, Average RTT: 0, Last error: connection() : connection(mongocnf-cmgo-replica-1.mongocnf-cmgo.default.svc.cluster.local:27017[-186]) unable to decode message length: EOF
      23:37:20 docker-entrypoint.sh: LOG:info Addr: mongocnf-cmgo-replica-2.mongocnf-cmgo.default.svc.cluster.local:27017, Type: Unknown, State: Connected, Average RTT: 0, Last error: connection() : connection(mongocnf-cmgo-replica-2.mongocnf-cmgo.default.svc.cluster.local:27017[-185]) unable to decode message length: EOF

      On the Mongod process on the client is complaining that the connections are not SSL:

      I NETWORK [conn9279] Error receiving request from client: SSLHandshakeFailed: The server is configured to only allow SSL connections. Ending connection from 192.168.1.198:60382 (connection id: 9279)

      192.168.1.198 is the address of the mongo replica that is doing part of the backup. The other replica set has a node going crazy this way as well.

      I don't see anything in the documentation about how to configure the SSL from the replicas to the config server. If they just used the URI that the PBM backup command was given, it would work.

        Smart Checklist

          Attachments

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                mrnorton Mike Norton
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - Not Specified
                  Not Specified
                  Logged:
                  Time Spent - 7 hours
                  7h