Uploaded image for project: 'Percona Monitoring and Management'
  1. Percona Monitoring and Management
  2. PMM-7627

Consistent PMM Server log management

    XMLWordPrintable

Details

    Description

      As a result of issues such as PMM-7464 the log files for NGINX can grow out of control, e.g.

      $ du -sh /srv/logs/nginx.error.log 
13G     /srv/logs/nginx.error.log

      The log managed by supervisord is already controlled, but the ones left to NGINX are not.

      Suggested fix:

      • add log rotation to allow log control

      Implementation:

      • add log rotation
      • delegate log rotation to exclusively to supervisord
      • redirect all nginx logs to /stdout and /stderr vs to log files

      How to test:

      • there are automated tests, which should be green:
        https://github.com/Percona-Lab/pmm-api-tests
        https://github.com/percona/pmm-managed
      • get inside of the PMM docker container and verify that the `/srv/logs` file list is limited to the following:
        • a fresh install wouldn't have the files, deleted by this PR
        • a PMM Server upgraded from an older version to this one would still have those files, but this is impossible to test

      Example of a PMM Server latest version:

      Attachments

        Issue Links

          blocks

          Bug - A problem which impairs or prevents the functions of the product. PMM-8307 Default config limits for allowed connections prevents monitoring large numbers (500+) of DB servers

          • Medium - Minor loss of function, or other problem where easy workaround is present.
          • Done
          duplicates

          Bug - A problem which impairs or prevents the functions of the product. PMM-5845 Logrotate issues with Nginx/Supervisord logs

          • Medium - Minor loss of function, or other problem where easy workaround is present.
          • Done

          Admin & Maintenance Task - For Percona Internal usage only PMM-5385 Manage all PMM Server logs with supervisord

          • Medium - Minor loss of function, or other problem where easy workaround is present.
          • Done
          is duplicated by

          Improvement - An improvement or enhancement to an existing feature or task. PMM-5292 Reduce amount and size of ClickHouse logs

          • Medium - Minor loss of function, or other problem where easy workaround is present.
          • Done
          relates to

          Bug - A problem which impairs or prevents the functions of the product. PMM-7464 NGINX misconfiguration leads to log storm in push mode

          • High - Crashes, loss of data, severe memory leak.
          • Done
          mentioned in

          Page Loading...

          Page Loading...

          Activity

            People

              alexander.tymchuk Alexander Tymchuk
              ceri.williams Ceri Williams
              Votes:
              1 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - Not Specified
                  Not Specified
                  Logged:
                  Time Spent - 1 day, 4 hours
                  1d 4h

                  Smart Checklist