Uploaded image for project: 'Percona Monitoring and Management'
  1. Percona Monitoring and Management
  2. PMM-8548

AMI upgrade from 2.16.0 and 2.17.0 to 2.20.0 failed


    • Bug
    • Status: Done
    • High
    • Resolution: Fixed
    • 2.20.0
    • None
    • AMI
    • None
    • [obsolete] C/S Core


      User impact: User can't upgrade PMM


      1. Run 2.16.0 AMI PMM
      2. Open Home dashboard and start upgrade to 2.20.0

      Actual Result: Upgrade stuck in loop

       TASK [Cleanup yum metadata] ****************************************************
      skipping: [localhost]
      TASK [Download pmm2 packages] **************************************************
      fatal: [localhost]: FAILED! => {"changed": true, "changes": {"installed": [], "updated": [["pmm-server", "2.20.0-23.2107271100.e68715f.el7.noarch from pmm2-server"], ["percona-victoriametrics", "1.60.0-1.el7.x86_64 from pmm2-server"], ["percona-qan-api2", "2.20.0-15.2107120744.af14526.el7.x86_64 from pmm2-server"], ["dbaas-controller", "2.20.0-1.2106281154.316d3d2.el7.x86_64 from pmm2-server"], ["pmm-managed", "2.20.0-17.2107220951.3c660c3.el7.x86_64 from pmm2-server"], ["percona-grafana", "7.5.7-93.2107220953.908ff0c.el7.x86_64 from pmm2-server"], ["percona-dashboards", "2.20.0-15.2107131605.75d67a4.el7.noarch from pmm2-server"], ["percona-alertmanager", "0.21.0-3.el7.x86_64 from pmm2-server"], ["pmm2-client", "2.20.0-6.el7.x86_64 from percona-release-x86_64"]]}, "msg": "https://repo.percona.com/pmm2-components/yum/release/7/RPMS/x86_64/percona-victoriametrics-1.60.0-1.el7.x86_64.rpm: [Errno -1] Package does not match intended download. Suggestion: run yum --enablerepo=pmm2-server clean metadata\nTrying other mirror.\n\n\nError downloading packages:\n  percona-victoriametrics-1.60.0-1.el7.x86_64: [Errno 256] No more mirrors to try.\n\n", "rc": 1, "results": ["All packages providing pmm-update are up to date", "All packages providing percona-clickhouse-client are up to date", "All packages providing percona-clickhouse-server are up to date", "All packages providing dbaas-tools are up to date", "Loaded plugins: changelog, fastestmirror\nLoading mirror speeds from cached hostfile\n * base: d36uatko69830t.cloudfront.net\n * epel: dl.fedoraproject.org\n * extras: d36uatko69830t.cloudfront.net\n * updates: d36uatko69830t.cloudfront.net\nPackage pmm-update-2.20.0-60.2107271101.d72f0db.el7.noarch already installed and latest version\nPackage percona-clickhouse-client- already installed and latest version\nPackage percona-clickhouse-server- already installed and latest version\nPackage dbaas-tools-0.5.1-4.2009041503.el7.x86_64 already installed and latest version\nResolving Dependencies\n--> Running transaction check\n---> Package dbaas-controller.x86_64 0:2.17.0-1.2105070916.4fd8e4d.el7 will be updated\n---> Package dbaas-controller.x86_64 0:2.20.0-1.2106281154.316d3d2.el7 will be an update\n---> Package percona-alertmanager.x86_64 0:0.20.0-3.el7 will be updated\n---> Package percona-alertmanager.x86_64 0:0.21.0-3.el7 will be an update\n---> Package percona-dashboards.noarch 0:2.17.0-15.2105041522.8f48f1e.el7 will be updated\n---> Package percona-dashboards.noarch 0:2.20.0-15.2107131605.75d67a4.el7 will be an update\n---> Package percona-grafana.x86_64 0:7.3.7-92.2105051427.9d10fa2.el7 will be updated\n---> Package percona-grafana.x86_64 0:7.5.7-93.2107220953.908ff0c.el7 will be an update\n---> Package percona-qan-api2.x86_64 0:2.17.0-15.2104262202.f8aa0a4.el7 will be updated\n---> Package percona-qan-api2.x86_64 0:2.20.0-15.2107120744.af14526.el7 will be an update\n---> Package percona-victoriametrics.x86_64 0:1.53.1-1.el7 will be updated\n---> Package percona-victoriametrics.x86_64 0:1.60.0-1.el7 will be an update\n---> Package pmm-managed.x86_64 0:2.17.0-17.2105070914.e66288f.el7 will be updated\n---> Package pmm-managed.x86_64 0:2.20.0-17.2107220951.3c660c3.el7 will be an update\n---> Package pmm-server.noarch 0:2.17.0-22.2105110659.89e2758.el7 will be updated\n---> Package pmm-server.noarch 0:2.20.0-23.2107271100.e68715f.el7 will be an update\n---> Package pmm2-client.x86_64 0:2.17.0-6.el7 will be updated\n---> Package pmm2-client.x86_64 0:2.20.0-6.el7 will be an update\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package              Arch   Version                          Repository   Size\n================================================================================\nUpdating:\n dbaas-controller     x86_64 2.20.0-1.2106281154.316d3d2.el7  pmm2-server 4.3 M\n percona-alertmanager x86_64 0.21.0-3.el7                     pmm2-server  12 M\n percona-dashboards   noarch 2.20.0-15.2107131605.75d67a4.el7 pmm2-server  32 M\n percona-grafana      x86_64 7.5.7-93.2107220953.908ff0c.el7  pmm2-server  41 M\n percona-qan-api2     x86_64 2.20.0-15.2107120744.af14526.el7 pmm2-server 4.4 M\n percona-victoriametrics\n                      x86_64 1.60.0-1.el7                     pmm2-server 7.9 M\n pmm-managed          x86_64 2.20.0-17.2107220951.3c660c3.el7 pmm2-server  16 M\n pmm-server           noarch 2.20.0-23.2107271100.e68715f.el7 pmm2-server 2.0 M\n pmm2-client          x86_64 2.20.0-6.el7                     percona-release-x86_64\n                                                                           43 M\n\nTransaction Summary\n================================================================================\nUpgrade  9 Packages\n\nTotal size: 161 M\nTotal download size: 7.9 M\nBackground downloading packages, then exiting:\nDelta RPMs disabled because /usr/bin/applydeltarpm not installed.\n"]}
      PLAY RECAP *********************************************************************
      localhost                  : ok=10   changed=0    unreachable=0    failed=1    skipped=2    rescued=0    ignored=0   
      time="2021-08-04T07:57:47Z" level=fatal msg="RunPlaybook failed: exit status 2"
      ProjectName: pmm-update
      Version: 2.20.0
      PMMVersion: 2.20.0

      How to test:

      1. Run 2.16.0 AMI PMM
      2. Open Home dashboard and start an upgrade to 2.20.0

      Problem and solution:

      We have the victoriametrics package:
      We upload this package on each release. Even if the version didn’t change.
      It works good with testing and experimental repo but for some unknown reason, this breaks the production repo. We’ll stop upload the same package in the next release.

      Expected Result:

      Upgrades from 2.18.0 and 2.19.0 look ok

      I checked upgrade with enabled testing repositories and it was ok


        Issue Links



              nikita.beletskii Nikita Beletskii (Inactive)
              nailya.kutlubaeva Nailya Kutlubaeva
              0 Vote for this issue
              4 Start watching this issue



                Time Tracking

                  Original Estimate - Not Specified
                  Not Specified
                  Remaining Estimate - Not Specified
                  Not Specified
                  Time Spent - 1 hour

                  Smart Checklist