Uploaded image for project: 'Percona Server for MongoDB'
  1. Percona Server for MongoDB
  2. PSMDB-245

Need to ensure KeyDB's WiredTiger logs are properly rotated without restart

Details

    • Bug
    • Status: Done
    • Medium
    • Resolution: Fixed
    • 3.6.8-2.0
    • 4.0.4-1, 3.6.10-3.0
    • None
    • None

    Description

      This was observed by Tomislav:
      when I add some number of databases and remove then at the same time I don't see significant key.wt size increase, but when I add some number of databases and then remove them all later the file size is not decreased in that moment, but only when I do the server restart - this might be related to how WT is handling data/files in general. I'll upload an example file here.

      ### INITIAL STATUS: ###
      ls -alh
      total 28M
      drwx------ 2 plavi plavi 4,0K lis  20 11:43 .
      drwxrwxr-x 5 plavi plavi  13M lis  20 11:43 ..
      -rw------- 1 plavi plavi 4,0K lis  20 11:43 key.wt
      -rw------- 1 plavi plavi 4,0K lis  20 11:43 parameters.wt
      -rw------- 1 plavi plavi   46 lis  20 11:43 WiredTiger
      -rw------- 1 plavi plavi 4,0K lis  20 11:43 WiredTigerLAS.wt
      -rw------- 1 plavi plavi   21 lis  20 11:43 WiredTiger.lock
      -rw------- 1 plavi plavi 5,0M lis  20 11:43 WiredTigerLog.0000000001
      -rw------- 1 plavi plavi 5,0M lis  20 11:43 WiredTigerPreplog.0000000001
      -rw------- 1 plavi plavi 5,0M lis  20 11:43 WiredTigerPreplog.0000000002
      -rw------- 1 plavi plavi  890 lis  20 11:43 WiredTiger.turtle
      -rw------- 1 plavi plavi 4,0K lis  20 11:43 WiredTiger.wt
      ### AFTER ADDING 100000 dbs: ###
      ls -alh
      total 42M
      drwx------ 2 plavi plavi 4,0K lis  19 22:54 .
      drwxrwxr-x 5 plavi plavi  13M lis  19 23:22 ..
      -rw------- 1 plavi plavi 4,6M lis  20 07:22 key.wt
      -rw------- 1 plavi plavi  32K lis  19 21:33 parameters.wt
      -rw------- 1 plavi plavi   46 lis  19 16:35 WiredTiger
      -rw------- 1 plavi plavi 4,0K lis  19 21:33 WiredTigerLAS.wt
      -rw------- 1 plavi plavi   21 lis  19 16:35 WiredTiger.lock
      -rw------- 1 plavi plavi 4,8M lis  19 22:13 WiredTigerLog.0000000006
      -rw------- 1 plavi plavi 4,8M lis  19 22:54 WiredTigerLog.0000000007
      -rw------- 1 plavi plavi 5,0M lis  19 23:21 WiredTigerLog.0000000008
      -rw------- 1 plavi plavi 5,0M lis  19 22:13 WiredTigerPreplog.0000000003
      -rw------- 1 plavi plavi 5,0M lis  19 22:54 WiredTigerPreplog.0000000004
      -rw------- 1 plavi plavi 1,1K lis  19 21:33 WiredTiger.turtle
      -rw------- 1 plavi plavi  32K lis  19 21:33 WiredTiger.wt
      ### AFTER REMOVING 100000 DBS: ###
      plavi@bender  ~/lab/psmdb/bin/percona-server-mongodb-3.6.8-2.0/nodes/db/keydb  ls -alh
      total 58M
      drwx------ 2 plavi plavi 4,0K lis  20 11:28 .
      drwxrwxr-x 5 plavi plavi  13M lis  20 11:30 ..
      -rw------- 1 plavi plavi 5,9M lis  20 11:22 key.wt
      -rw------- 1 plavi plavi  32K lis  19 21:33 parameters.wt
      -rw------- 1 plavi plavi   46 lis  19 16:35 WiredTiger
      -rw------- 1 plavi plavi 4,0K lis  19 21:33 WiredTigerLAS.wt
      -rw------- 1 plavi plavi   21 lis  19 16:35 WiredTiger.lock
      -rw------- 1 plavi plavi 4,8M lis  19 22:13 WiredTigerLog.0000000006
      -rw------- 1 plavi plavi 4,8M lis  19 22:54 WiredTigerLog.0000000007
      -rw------- 1 plavi plavi 4,8M lis  20 11:15 WiredTigerLog.0000000008
      -rw------- 1 plavi plavi 4,8M lis  20 11:22 WiredTigerLog.0000000009
      -rw------- 1 plavi plavi 4,8M lis  20 11:28 WiredTigerLog.0000000010
      -rw------- 1 plavi plavi 5,0M lis  20 11:29 WiredTigerLog.0000000011
      -rw------- 1 plavi plavi 5,0M lis  20 11:15 WiredTigerPreplog.0000000005
      -rw------- 1 plavi plavi 5,0M lis  20 11:28 WiredTigerPreplog.0000000007
      -rw------- 1 plavi plavi 1,1K lis  19 21:33 WiredTiger.turtle
      -rw------- 1 plavi plavi  32K lis  19 21:33 WiredTiger.wt
      > show dbs;
      admin   0.000GB
      config  0.000GB
      local   0.000GB
      ### AFTER RESTART: ###
      plavi@bender  ~/lab/psmdb/bin/percona-server-mongodb-3.6.8-2.0/nodes/db/keydb  ls -alh
      total 28M
      drwx------ 2 plavi plavi 4,0K lis  20 11:40 .
      drwxrwxr-x 5 plavi plavi  13M lis  20 11:40 ..
      -rw------- 1 plavi plavi 176K lis  20 11:40 key.wt
      -rw------- 1 plavi plavi  36K lis  20 11:40 parameters.wt
      -rw------- 1 plavi plavi   46 lis  19 16:35 WiredTiger
      -rw------- 1 plavi plavi 4,0K lis  20 11:40 WiredTigerLAS.wt
      -rw------- 1 plavi plavi   21 lis  19 16:35 WiredTiger.lock
      -rw------- 1 plavi plavi 5,0M lis  20 11:40 WiredTigerLog.0000000011
      -rw------- 1 plavi plavi 5,0M lis  20 11:15 WiredTigerPreplog.0000000005
      -rw------- 1 plavi plavi 5,0M lis  20 11:28 WiredTigerPreplog.0000000007
      -rw------- 1 plavi plavi 1,1K lis  20 11:40 WiredTiger.turtle
      -rw------- 1 plavi plavi  36K lis  20 11:40 WiredTiger.wt
      ### AFTER ADDING AND REMOVING 100000 DATABASES ###
      plavi@bender  ~/lab/psmdb/bin/percona-server-mongodb-3.6.8-2.0/nodes/db/keydb  ls -alh
      total 52M
      drwx------ 2 plavi plavi 4,0K lis  20 13:39 .
      drwxrwxr-x 5 plavi plavi  13M lis  20 13:43 ..
      -rw------- 1 plavi plavi  12K lis  20 12:48 key.wt
      -rw------- 1 plavi plavi 4,0K lis  20 11:43 parameters.wt
      -rw------- 1 plavi plavi   46 lis  20 11:43 WiredTiger
      -rw------- 1 plavi plavi 4,0K lis  20 11:43 WiredTigerLAS.wt
      -rw------- 1 plavi plavi   21 lis  20 11:43 WiredTiger.lock
      -rw------- 1 plavi plavi 4,8M lis  20 12:17 WiredTigerLog.0000000001
      -rw------- 1 plavi plavi 4,8M lis  20 12:37 WiredTigerLog.0000000002
      -rw------- 1 plavi plavi 4,8M lis  20 12:57 WiredTigerLog.0000000003
      -rw------- 1 plavi plavi 4,8M lis  20 13:18 WiredTigerLog.0000000004
      -rw------- 1 plavi plavi 4,8M lis  20 13:39 WiredTigerLog.0000000005
      -rw------- 1 plavi plavi 5,0M lis  20 13:42 WiredTigerLog.0000000006
      -rw------- 1 plavi plavi 5,0M lis  20 12:57 WiredTigerPreplog.0000000005
      -rw------- 1 plavi plavi 5,0M lis  20 13:39 WiredTigerPreplog.0000000007
      -rw------- 1 plavi plavi  890 lis  20 11:43 WiredTiger.turtle
      -rw------- 1 plavi plavi 4,0K lis  20 11:43 WiredTiger.wt
      ### AFTER RESTART ###
      ✘ plavi@bender  ~/lab/psmdb/bin/percona-server-mongodb-3.6.8-2.0/nodes/db/keydb  ls -alh
      total 28M
      drwx------ 2 plavi plavi 4,0K lis  20 19:53 .
      drwxrwxr-x 5 plavi plavi  13M lis  20 19:53 ..
      -rw------- 1 plavi plavi  16K lis  20 19:53 key.wt
      -rw------- 1 plavi plavi  16K lis  20 19:53 parameters.wt
      -rw------- 1 plavi plavi   46 lis  20 11:43 WiredTiger
      -rw------- 1 plavi plavi 4,0K lis  20 19:53 WiredTigerLAS.wt
      -rw------- 1 plavi plavi   21 lis  20 11:43 WiredTiger.lock
      -rw------- 1 plavi plavi 5,0M lis  20 19:53 WiredTigerLog.0000000007
      -rw------- 1 plavi plavi 5,0M lis  20 19:53 WiredTigerPreplog.0000000001
      -rw------- 1 plavi plavi 5,0M lis  20 19:53 WiredTigerPreplog.0000000002
      -rw------- 1 plavi plavi 1,1K lis  20 19:53 WiredTiger.turtle
      -rw------- 1 plavi plavi  32K lis  20 19:53 WiredTiger.wt
      

      Attachments

        Issue Links

          Activity

            People

              igor.solodovnikov Igor Solodovnikov
              igor.solodovnikov Igor Solodovnikov
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 1 day, 5 hours, 30 minutes
                  1d 5h 30m

                  Smart Checklist