Uploaded image for project: 'Percona XtraBackup'
  1. Percona XtraBackup
  2. PXB-2165

Modify xbcloud to store backups using s3 access key parameters if AWS access key env variables are set

Details

    • Bug
    • Status: Done
    • Medium
    • Resolution: Fixed
    • None
    • 2.4.21, 8.0.13
    • None

    Description

      Hi,

      According to our documentation xbcloud supports the following env variables:

      • AWS_ACCESS_KEY_ID (or ACCESS_KEY_ID)
      • AWS_SECRET_ACCESS_KEY (or SECRET_ACCESS_KEY)
      • AWS_DEFAULT_REGION (or DEFAULT_REGION)
      • AWS_ENDPOINT (or ENDPOINT)
      • AWS_CA_BUNDLE

      If we set ACCESS_KEY_ID, SECRET_ACCESS_KEY and ENDPOINT  and the environment already has 'AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS' variables, the latter are used by xbcloud as default ones. In case we need to have the access to e.g. 'storage.googleapis.com' and we set ACCESS_KEY_ID and SECRET_ACCESS_KEY we can not reach the bucket because wrong variables (AWS_*) are used.

      As for me we can not use 'AWS_*' variables by default because they are used for access to aws and force the user to unset or rewrite them, it is bad idea.
      for example, user can have AWS CLI installed/configured (which require AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS vars) and want to do backups to GCP.

      STR:
      0) xbcloud Ver 8.0.11 for Linux (x86_64)
      1) set 'AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS' variables with valid credentials;
      2) set ACCESS_KEY_ID, SECRET_ACCESS_KEY and ENDPOINT(storage.googleapis.com) variables with valid values for 'storage.googleapis.com' ;
      3) try to get file from the existing backup:

      xbcloud get  s3://operator-testing/some-name-2020-29-04-12:33:44-full xtrabackup_info
      200429 13:23:07 xbcloud: Probe failed. Please check your credentials and endpoint settings.
      

      The verbose output you can find in the attachment.

      3) unset AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS' variables (after that everything works fine):

      xbcloud get  s3://operator-testing/some-name-2020-29-04-12:33:44-full xtrabackup_info
      ....
      200429 13:18:41 xbcloud: Downloading some-name-2020-29-04-12:33:44-full/xtrabackup_info.00000000000000000001.
      200429 13:18:41 xbcloud: Download successfull some-name-2020-29-04-12:33:44-full/xtrabackup_info.00000000000000000001, size 29
      

      I have tried to use '--storage=google' option but result was the same.

      Attachments

        Activity

          People

            rahul.malik Rahul Malik (Inactive)
            slava.sarzhan Slava Sarzhan
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - Not Specified
                Not Specified
                Logged:
                Time Spent - 6 hours
                6h

                Smart Checklist