Uploaded image for project: 'Percona Monitoring and Management'
  1. Percona Monitoring and Management
  2. PMM-3298

Panic error in rds_exporter log after adding instance from second AWS account

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Priority: Medium
    • Resolution: Fixed
    • Affects Version/s: 1.17.0
    • Fix Version/s: 2.2.0
    • Component/s: RDS_exporter
    • Labels:
      None
    • Story Points:
      2
    • Sprint:
      Platform Sprint 4, Platform Sprint 5
    • Needs Review:
      Yes
    • Needs QA:
      Yes
    • Needs Doc:
      Yes

      Description

      User impact: will be updated

      STR:

      1. Add a couple of RDS instances (via UI) form the first AWS account
      2. Add RDS instance from second AWS account

      *Given result: * in rds_exporter.logs:

      time="2018-11-21T11:26:55Z" level=info msg="Using 1 sessions." component=sessions source="sessions.go:149"
      time="2018-11-21T11:26:55Z" level=info msg="Updating enhanced metrics every 10s." component=enhanced source="collector.go:49"
      time="2018-11-21T11:26:55Z" level=info msg="Basic metrics   : http://127.0.0.1:9042/basic" source="main.go:65"
      time="2018-11-21T11:26:55Z" level=info msg="Enhanced metrics: http://127.0.0.1:9042/enhanced" source="main.go:66"
      panic: runtime error: invalid memory address or nil pointer dereference
      [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x89f36a]
      
      goroutine 27 [running]:
      github.com/percona/rds_exporter/vendor/github.com/aws/aws-sdk-go/aws/session.(*Session).Copy(0x0, 0x0, 0x0, 0x0, 0xc4203ea400)
      	/home/builder/rpm/BUILD/rds_exporter-e71e0f3b4f101cf84191052834e3af3d774a0aa9/src/github.com/percona/rds_exporter/vendor/github.com/aws/aws-sdk-go/aws/session/session.go:550 +0x3a
      github.com/percona/rds_exporter/vendor/github.com/aws/aws-sdk-go/aws/session.(*Session).clientConfigWithErr(0x0, 0xb0c872, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
      	/home/builder/rpm/BUILD/rds_exporter-e71e0f3b4f101cf84191052834e3af3d774a0aa9/src/github.com/percona/rds_exporter/vendor/github.com/aws/aws-sdk-go/aws/session/session.go:571 +0x88
      github.com/percona/rds_exporter/vendor/github.com/aws/aws-sdk-go/aws/session.(*Session).ClientConfig(0x0, 0xb0c872, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
      	/home/builder/rpm/BUILD/rds_exporter-e71e0f3b4f101cf84191052834e3af3d774a0aa9/src/github.com/percona/rds_exporter/vendor/github.com/aws/aws-sdk-go/aws/session/session.go:565 +0xbd
      github.com/percona/rds_exporter/vendor/github.com/aws/aws-sdk-go/service/cloudwatch.New(0xbb5b20, 0x0, 0x0, 0x0, 0x0, 0x0)
      	/home/builder/rpm/BUILD/rds_exporter-e71e0f3b4f101cf84191052834e3af3d774a0aa9/src/github.com/percona/rds_exporter/vendor/github.com/aws/aws-sdk-go/service/cloudwatch/service.go:48 +0x88
      github.com/percona/rds_exporter/basic.NewScraper(0xc42006e600, 0xc4200b4340, 0xc4202ce360, 0x1)
      	/home/builder/rpm/BUILD/rds_exporter-e71e0f3b4f101cf84191052834e3af3d774a0aa9/src/github.com/percona/rds_exporter/basic/scraper.go:34 +0x86
      github.com/percona/rds_exporter/basic.(*Exporter).collect.func1(0xc42006e600, 0xc4200b4340, 0xc4202ce360, 0xc4204940d0)
      	/home/builder/rpm/BUILD/rds_exporter-e71e0f3b4f101cf84191052834e3af3d774a0aa9/src/github.com/percona/rds_exporter/basic/basic.go:64 +0x3f
      created by github.com/percona/rds_exporter/basic.(*Exporter).collect
      	/home/builder/rpm/BUILD/rds_exporter-e71e0f3b4f101cf84191052834e3af3d774a0aa9/src/github.com/percona/rds_exporter/basic/basic.go:63 +0xec
      time="2018-11-21T11:27:00Z" level=info msg="Starting RDS exporter (version=, branch=, revision=)" source="main.go:30"
      time="2018-11-21T11:27:00Z" level=info msg="Build context (go=go1.10.1, user=, date=)" source="main.go:31"
      time="2018-11-21T11:27:00Z" level=info msg="Creating sessions..." component=sessions source="sessions.go:35"
      time="2018-11-21T11:27:00Z" level=error msg="Skipping instance us-east-1/rds-aurora1 - can't determine resourceID." component=sessions source="sessions.go:125"
      time="2018-11-21T11:27:00Z" level=error msg="Skipping instance us-east-1/rds-mysql56 - can't determine resourceID." component=sessions source="sessions.go:125"
      Region     Instance     Resource ID                    Interval
      us-east-1  aurora57pmm  db-GR4HWW3VHLL3KZBTJ7EDUU2A4A  1m0s
      time="2018-11-21T11:27:00Z" level=info msg="Using 1 sessions." component=sessions source="sessions.go:149"
      time="2018-11-21T11:27:00Z" level=info msg="Updating enhanced metrics every 10s." component=enhanced source="collector.go:49"
      time="2018-11-21T11:27:00Z" level=info msg="Basic metrics   : http://127.0.0.1:9042/basic" source="main.go:65"
      time="2018-11-21T11:27:00Z" level=info msg="Enhanced metrics: http://127.0.0.1:9042/enhanced" source="main.go:66"
      [root@a2ae9b3fd430 opt]# tail -50 /var/log/pmm-rds_exporter-9042.log 
      github.com/percona/rds_exporter/vendor/github.com/aws/aws-sdk-go/aws/session.(*Session).ClientConfig(0x0, 0xb0c872, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
      	/home/builder/rpm/BUILD/rds_exporter-e71e0f3b4f101cf84191052834e3af3d774a0aa9/src/github.com/percona/rds_exporter/vendor/github.com/aws/aws-sdk-go/aws/session/session.go:565 +0xbd
      github.com/percona/rds_exporter/vendor/github.com/aws/aws-sdk-go/service/cloudwatch.New(0xbb5b20, 0x0, 0x0, 0x0, 0x0, 0x0)
      	/home/builder/rpm/BUILD/rds_exporter-e71e0f3b4f101cf84191052834e3af3d774a0aa9/src/github.com/percona/rds_exporter/vendor/github.com/aws/aws-sdk-go/service/cloudwatch/service.go:48 +0x88
      github.com/percona/rds_exporter/basic.NewScraper(0xc42019c180, 0xc420159900, 0xc420166000, 0xf551a0)
      	/home/builder/rpm/BUILD/rds_exporter-e71e0f3b4f101cf84191052834e3af3d774a0aa9/src/github.com/percona/rds_exporter/basic/scraper.go:34 +0x86
      github.com/percona/rds_exporter/basic.(*Exporter).collect.func1(0xc42019c180, 0xc420159900, 0xc420166000, 0xc4204342d0)
      	/home/builder/rpm/BUILD/rds_exporter-e71e0f3b4f101cf84191052834e3af3d774a0aa9/src/github.com/percona/rds_exporter/basic/basic.go:64 +0x3f
      created by github.com/percona/rds_exporter/basic.(*Exporter).collect
      	/home/builder/rpm/BUILD/rds_exporter-e71e0f3b4f101cf84191052834e3af3d774a0aa9/src/github.com/percona/rds_exporter/basic/basic.go:63 +0xec
      time="2018-11-21T12:55:12Z" level=info msg="Starting RDS exporter (version=, branch=, revision=)" source="main.go:30"
      time="2018-11-21T12:55:12Z" level=info msg="Build context (go=go1.10.1, user=, date=)" source="main.go:31"
      time="2018-11-21T12:55:12Z" level=info msg="Creating sessions..." component=sessions source="sessions.go:35"
      time="2018-11-21T12:55:12Z" level=error msg="Skipping instance us-east-1/rds-aurora1 - can't determine resourceID." component=sessions source="sessions.go:125"
      time="2018-11-21T12:55:12Z" level=error msg="Skipping instance us-east-1/rds-mysql56 - can't determine resourceID." component=sessions source="sessions.go:125"
      

      Expected result: No errors in logs

        Smart Checklist

          Attachments

            Activity

              People

              Assignee:
              Unassigned
              Reporter:
              nailya.kutlubaeva Nailya Kutlubaeva
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - Not Specified
                  Not Specified
                  Logged:
                  Time Spent - 2 hours, 23 minutes
                  2h 23m