Details

    Description

      User story:

      • When I need to add Azure related DB for the monitoring I need to be able to enter Azure related credentials in PMM UI (add Instance) So I can see the list of all DB existed in the Azure account and processed with Adding them to PMM for monitoring

      UI/UX:
      TBD
      something related to :

      Acceptance criteria

      • User can enter credentials: AZURE_CLIENT_ID, AZURE_CLIENT_SECRET, AZURE_TENANT_ID, AZURE_SUBSCRIPTION_ID click discovery and see the list of DB services possible monitor by PMM
      • List of DBs should contain: Name, DomainName, resourceGroup, Type, Version, Location (port is unavailable - check if standard port works)
      • The user is able to select a DB from the list and proceed to a screen to fill more details to start to monitor it by PMM.
      • On the screen of Add remote MySQL (PostgreSQL/MariaDB) Instance, the user can see and edit the next fields (filled in or empty):
        • DomainName - filled in
        • Service Name - default DomainName
        • Port - default for DB type: 3306, 5432 - check if it works - ports does not return with selected resource discovery
        • user - filled in by concatenating "<administratorLogin>@<name>"
        • Password - empty
        • Labels
          • Region - repopulated by location
          • Environment - filled in by tags.environment
          • Do we need fill in any other labels?
          • Custom labels empty
        • Additional option - need additional investigation 
      • User able to do all operations using Azure and PMM UI only
      • The Discovery process works for:
        • Azure Database for PostgreSQL
        • Azure Database for MySQL
        • Azure Database for MariaDB  - need an additional investigation - as it stays separate from "standard" Azure DB - and its cost does not encourage to do many experiments. 

      Here is a screenshot of adding Azure Monitor as a datasource we may consider/use it as an example.

      Also Resource group is required.
      It's not clear if it has to be discovered automatically or added as new instance each time with whole list of *ids.

      Out of scope:

      • any Azure SQL related discovery
      • Start monitoring Azure Node resource

      Suggested implementation:

      •  Implement two endpoints in PMM management API:
        • DiscoverAzureDBs: accepts AZURE_CLIENT_ID, AZURE_CLIENT_SECRET, AZURE_TENANT_ID, AZURE_SUBSCRIPTION_ID  and return list of database instances
        • AddedAzireDB: accept details return if successfully added.
      • Implement Azure service in pmm-managed to discover DBs (how to discover in details)
      • Add the ability in pmm-managed to save service details
      • Start agents to collect metrics and query analytics.
      • Cover with API tests

      Risks:

      • Discovery and monitoring of MariaDB
      • How to form DSN
      • TLS
      • **

      How to test:

      1. Create DB on Azure
      2. Genarates credentials.
      3. Discover from grafana
      4. Add to monitor
      5. See matrics on garafana dashboard.

      More detailed TBD

       

      Details:
      Example how to discover 

       

      package main
      
      
      // https://github.com/Azure/azure-sdk-for-go/blob/master/services/resourcegraph/mgmt/2019-04-01/resourcegraph/client.go
      // https://docs.microsoft.com/en-us/azure/governance/resource-graph/first-query-go
      // https://docs.microsoft.com/en-us/azure/governance/resource-graph/concepts/explore-resources
      // ./argQuery "Resources | where type =~ 'Microsoft.DBforMySQL/servers' | limit 5" "<TENANT_ID>"
      // ./argQuery "Resources | where type =~ 'Microsoft.DBforPostgreSQL/servers' | limit 5" "<TENANT_ID>"
      
      
      import (
              "context"
              "encoding/json"
              "fmt"
              "os"
              "strconv"
      
      
              arg "github.com/Azure/azure-sdk-for-go/services/resourcegraph/mgmt/2019-04-01/resourcegraph"
              "github.com/Azure/go-autorest/autorest/azure/auth"
      )
      
      
      func main() {
              // Get variables from command line arguments
              var query = os.Args[1]
              var subList = os.Args[2:]
      
      
              // Create and authorize a ResourceGraph client
              argClient := arg.New()
              // authorizer, err := auth.NewAuthorizerFromCLI()
              authorizer, err := auth.NewAuthorizerFromEnvironment()
              if err == nil {
                      argClient.Authorizer = authorizer
              } else {
                      fmt.Printf(err.Error())
              }
      
      
              // Set options
              RequestOptions := arg.QueryRequestOptions{
                      ResultFormat: "objectArray",
              }
      
      
              // Create the query request
              Request := arg.QueryRequest{
                      Subscriptions: &subList,
                      Query:         &query,
                      Options:       &RequestOptions,
              }
      
      
              // Run the query and get the results
              var results, queryErr = argClient.Resources(context.Background(), Request)
              if queryErr == nil {
                      fmt.Printf("Resources found: " + strconv.FormatInt(*results.TotalRecords, 10) + "\n")
                      res, err := json.MarshalIndent(results.Data, "", "    ")
                      if err != nil {
                              panic(err)
                      }
                      fmt.Printf("Results: \n %s \n", res)
              } else {
                      fmt.Printf(queryErr.Error())
              }
      }
      
      

       

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              roma.novikov Roma Novikov
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - Not Specified
                  Not Specified
                  Logged:
                  Time Spent - 3 hours
                  3h

                  Smart Checklist