Details
-
Bug
-
Status: Done
-
Medium
-
Resolution: Fixed
-
1.11.0
-
None
-
Yes
-
Yes
Description
When starting a backup it can happen that the deadline to start a backup can be exceeded especially in the sharding environment because of many nodes.
We have set some deadline of 40sec and if the backup is not started by then it will be marked as failed (although it can actually succeed).
This deadline is set here: https://github.com/percona/percona-server-mongodb-operator/blob/main/pkg/controller/perconaservermongodbbackup/backup.go#L15
The error can look like this:
apiVersion: psmdb.percona.com/v1 kind: PerconaServerMongoDBBackup metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"psmdb.percona.com/v1","kind":"PerconaServerMongoDBBackup","metadata":{"annotations":{},"name":"backup1","namespace":"mongo"},"spec":{"psmdbCluster":"my-cluster-name","storageName":"s3-eu-west-2"}} creationTimestamp: "2022-03-21T09:23:12Z" generation: 1 name: backup1 namespace: mongo resourceVersion: "2189522" uid: 89007408-3439-4b52-a406-af71643f7b40 spec: psmdbCluster: my-cluster-name storageName: s3-eu-west-2 status: azure: credentialsSecret: "" destination: "2022-03-21T09:23:33Z" error: starting deadline exceeded lastTransition: "2022-03-21T09:23:34Z" pbmName: "2022-03-21T09:23:33Z" s3: bucket: <bucket> credentialsSecret: my-cluster-name-backup-s3 region: eu-west-2 start: "2022-03-21T09:23:34Z" state: error storageName: s3-eu-west-2
Attachments
Issue Links
- relates to
-
K8SPSMDB-638 backup can fail with "starting deadline exceeded" even if it finishes in PBM
-
- Open
-