Uploaded image for project: 'Cloud Dev'
  1. Cloud Dev
  2. CLOUD-89

[PSMDB] k8s-mongodb-initiator should exit 0 if replset is already initiated

    Details

    • Type: Bug
    • Status: Pending Release
    • Priority: High
    • Resolution: Fixed
    • Affects Version/s: PSMDB 0.1.0
    • Fix Version/s: PSMDB 0.2.0
    • Component/s: recreate
    • Labels:
      None

      Description

      Problem: the k8s-mongodb-initiator should only run if the replset has not been initiated before.

      Currently when a CR is deleted and recreated without deleting PVC data, k8s-mongodb-initiator tries to run and exits with exit-code 1 instead of 0 because the replset initiation has already happened:

      $ k8s-mongodb-initiator 
      2019-01-15 12:25:20.884 initiator.go:219 INFO   Mongod replset initiator started  service=my-cluster-name
      2019-01-15 12:25:20.884 initiator.go:223 INFO   Waiting to start initiation  sleep=0s
      2019-01-15 12:25:20.886 initiator.go:154 INFO   Connected to MongoDB  auth=false host=localhost:27017 replset= ssl=false ssl_secure=false
      2019-01-15 12:25:20.887 initiator.go:72  INFO   Initiating replset 
      {
      	"_id": "rs0",
      	"version": 1,
      	"members": [
      		{
      			"_id": 0,
      			"host": "my-cluster-name-rs0-0.my-cluster-name-rs0.default.svc.cluster.local:27017",
      			"arbiterOnly": false,
      			"buildIndexes": true,
      			"hidden": false,
      			"priority": 1,
      			"tags": {
      				"serviceName": "my-cluster-name"
      			},
      			"slaveDelay": 0,
      			"votes": 1
      		}
      	]
      }
      2019-01-15 12:25:20.888 initiator.go:83  INFO   Got mongodb auth error, server appears to be initiated already. Skipping 
      2019-01-15 12:25:20.888 initiator.go:201 INFO   Waiting for host to become primary
      2019-01-15 12:26:54.089 initiator.go:204 ERROR  Error getting waiting for primary session  error=timed out waiting for host to become primary
      2019-01-15 12:26:54.089 initiator.go:238 ERROR  Error preparing replset  error=timed out waiting for host to become primary
      2019-01-15 12:26:54.089 main.go:136      FATAL  Error initiating replset: timed out waiting for host to become primary 

      Fix needed: cleanly exit the initiator at line 83 of initiator.go ("Got mongodb auth error, server appears to be initiated already. Skipping") instead of continuing initiation.

      Reproduce:

      1. Start up latest 'master' with clean PVCs. It should initiate and start successfully with 3 x nodes in the replset.
      2. Delete the CR but don't delete PVCs.
      3. Create the CR again. The k8s-mongodb-initiator should fail with exit code 1.

       

        Smart Checklist

          Attachments

            Issue Links

              Activity

                People

                • Assignee:
                  mykola.marzhan Mykola Marzhan
                  Reporter:
                  tim.vaillancourt Tim Vaillancourt (Inactive)
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  1 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Time Tracking

                    Estimated:
                    Original Estimate - Not Specified
                    Not Specified
                    Remaining:
                    Remaining Estimate - Not Specified
                    Not Specified
                    Logged:
                    Time Spent - 1 day, 30 minutes
                    1d 30m