Uploaded image for project: 'Percona Monitoring and Management'
  1. Percona Monitoring and Management
  2. PMM-5688

Queries inserted with the wrong timestamp after a network outage

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open
    • Priority: Medium
    • Resolution: Unresolved
    • Affects Version/s: 2.4.0
    • Fix Version/s: None
    • Component/s: MySQLd_Exporter, QAN Agent
    • Labels:
      None
    • Environment:

      CentOS Linux release 7.3.1611 (Core)

      Description

      When running a simple workload like: 

       while sleep 0.1; do mysql -e 'select now()'; done

      If the network connectivity between pmm-client and pmm-server is interrupted (e.g. by blocking port 443 on pmm-server) for a few minutes, when it is re-established, the queries are inserted but with the wrong timestamp. 

      That is, all queries seem to be inserted, but they show up as having been executed at ingest time, not at the time registered on the slow query log. 

      This can be seen in the "load" column on the query ranking, as a spike in QPS right after the network connection is established (the spike doesn't really happen, on mysqld's side, the simple select loop has been running constantly). Please refer to the attachment to see what I mean. 

      I can further verify this by feeding the agent a pre-existing slow query log. 

      That is, use the agent as normal, but on the fly, replace mysql's slow query log for a file with older queries. These queries will show up in QAN but will be timestamped as having executed at ingest time, not at the time on the slow query log entries. 

        Smart Checklist

          Attachments

            Activity

              People

              Assignee:
              Unassigned
              Reporter:
              lefipar Fernando Ipar
              Votes:
              2 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated: