Uploaded image for project: 'Percona Monitoring and Management'
  1. Percona Monitoring and Management
  2. PMM-1661

pmm-managed not started after whole server restart

    Details

    • Type: Bug
    • Status: Done
    • Priority: Medium
    • Resolution: Fixed
    • Affects Version/s: 1.4.0
    • Fix Version/s: 1.5.0
    • Component/s: PMM Manage
    • Labels:
      None

      Description

      Prometheus start took just 14 seconds:
      time="2017-10-30T09:57:00Z" level=info msg="Starting prometheus (version=1.7.2, branch=v1.7.2, revision=22eadbe635528fa17b99a7635fed6b6018103042)" source="main.go:88"
      time="2017-10-30T09:57:10Z" level=info msg="Scanning files." source="crashrecovery.go:55"
      time="2017-10-30T09:57:10Z" level=info msg="10000 files scanned." source="crashrecovery.go:77"
      time="2017-10-30T09:57:14Z" level=info msg="Listening on :9090" source="web.go:259"

      But it's not fast enough for current pmm-managed timeouts in supervisord:
      first time started:
      time="2017-10-30T09:56:58Z" level=info msg="Starting server on http://127.0.0.1:7771/ ..." component=gRPC

      time="2017-10-30T09:55:59Z" level=warning msg="Received SIGTERM, exiting gracefully..." source="main.go:234"

      Stopped by supervised:
      2017-10-30 09:57:06,328 INFO gave up: pmm-managed entered FATAL state, too many start retries too quickly

      Original error visible with pmm-admin list:
      Error getting a list of external metrics: status code 502 (text/html).
      Please check versions of your PMM Server and PMM Client.

      Fixed by:
      docker exec -it pmm-server supervisorctl start pmm-managed

      Expected behaviour:

      • wait more for prometheus
      • or have longer timeout in supervisord

        Smart Checklist

          Attachments

            Issue Links

              Activity

                People

                • Assignee:
                  borys.belinsky Borys Belinsky
                  Reporter:
                  nickolay.ihalainen Nickolay Ihalainen
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  5 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved: