Uploaded image for project: 'Percona Monitoring and Management'
  1. Percona Monitoring and Management
  2. PMM-2969

rds_exporter becomes throttled by CloudWatch API

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Priority: High
    • Resolution: Fixed
    • Affects Version/s: 1.14.1
    • Fix Version/s: 1.15.0
    • Component/s: RDS_exporter
    • Labels:
      None
    • Environment:

      The version of pmm is 1.14.0, monitor to AWS RDS(MySQL 5.7.21), set by add access key and securet key, add monitor account for RDS.

      Description

       There are some error in pmm-rds_exporter-9042.log

      time="2018-09-06T18:54:19-07:00" level=error msg="unable to get logs for instance ********: ThrottlingException: Rate exceeded\n\tstatus code: 400, request id: ef0e8f72-b240-11e8-bf8d-375517b4e803" source="enhanced.go:110" 

      and some error in prometheus.log

      level=warn ts=2018-09-07T01:55:42.097865018Z caller=scrape.go:713 component="scrape manager" scrape_pool=rds-mysql-mr target=http://127.0.0.1:10000/metrics-mr msg="append failed" err="out of order sample"
       level=warn ts=2018-09-07T01:55:47.140712396Z caller=scrape.go:942 component="scrape manager" scrape_pool=rds-mysql-mr target=http://127.0.0.1:10000/metrics-mr msg="Error on ingesting out-of-order samples" num_dropped=2 

      aaapmm.com/prometheus/targets

      has Endporint 's state is DOWN and Error is [out of order sample] and [context deadline exceeded]

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                chenxinglong Allen.Chan
              • Votes:
                3 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: