Details
-
Bug
-
Status: Done
-
Medium
-
Resolution: Fixed
-
1.11.0
-
None
-
Yes
-
Yes
Description
When starting a backup it can happen that the deadline to start a backup can be exceeded especially in the sharding environment because of many nodes.
We have set some deadline of 40sec and if the backup is not started by then it will be marked as failed (although it can actually succeed).
This deadline is set here: https://github.com/percona/percona-server-mongodb-operator/blob/main/pkg/controller/perconaservermongodbbackup/backup.go#L15
The error can look like this:
apiVersion: psmdb.percona.com/v1
kind: PerconaServerMongoDBBackup
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"psmdb.percona.com/v1","kind":"PerconaServerMongoDBBackup","metadata":{"annotations":{},"name":"backup1","namespace":"mongo"},"spec":{"psmdbCluster":"my-cluster-name","storageName":"s3-eu-west-2"}}
creationTimestamp: "2022-03-21T09:23:12Z"
generation: 1
name: backup1
namespace: mongo
resourceVersion: "2189522"
uid: 89007408-3439-4b52-a406-af71643f7b40
spec:
psmdbCluster: my-cluster-name
storageName: s3-eu-west-2
status:
azure:
credentialsSecret: ""
destination: "2022-03-21T09:23:33Z"
error: starting deadline exceeded
lastTransition: "2022-03-21T09:23:34Z"
pbmName: "2022-03-21T09:23:33Z"
s3:
bucket: <bucket>
credentialsSecret: my-cluster-name-backup-s3
region: eu-west-2
start: "2022-03-21T09:23:34Z"
state: error
storageName: s3-eu-west-2