Uploaded image for project: 'Percona Server for MongoDB'
  1. Percona Server for MongoDB
  2. PSMDB-245

Need to ensure KeyDB's WiredTiger logs are properly rotated without restart

    Details

    • Type: Bug
    • Status: Done
    • Priority: Medium
    • Resolution: Fixed
    • Affects Version/s: 3.6.8-2.0
    • Fix Version/s: 3.6.10-3.0, 4.0.4-1
    • Component/s: None
    • Labels:
      None

      Description

      This was observed by Tomislav:
      when I add some number of databases and remove then at the same time I don't see significant key.wt size increase, but when I add some number of databases and then remove them all later the file size is not decreased in that moment, but only when I do the server restart - this might be related to how WT is handling data/files in general. I'll upload an example file here.

      ### INITIAL STATUS: ###
      ls -alh
      total 28M
      drwx------ 2 plavi plavi 4,0K lis  20 11:43 .
      drwxrwxr-x 5 plavi plavi  13M lis  20 11:43 ..
      -rw------- 1 plavi plavi 4,0K lis  20 11:43 key.wt
      -rw------- 1 plavi plavi 4,0K lis  20 11:43 parameters.wt
      -rw------- 1 plavi plavi   46 lis  20 11:43 WiredTiger
      -rw------- 1 plavi plavi 4,0K lis  20 11:43 WiredTigerLAS.wt
      -rw------- 1 plavi plavi   21 lis  20 11:43 WiredTiger.lock
      -rw------- 1 plavi plavi 5,0M lis  20 11:43 WiredTigerLog.0000000001
      -rw------- 1 plavi plavi 5,0M lis  20 11:43 WiredTigerPreplog.0000000001
      -rw------- 1 plavi plavi 5,0M lis  20 11:43 WiredTigerPreplog.0000000002
      -rw------- 1 plavi plavi  890 lis  20 11:43 WiredTiger.turtle
      -rw------- 1 plavi plavi 4,0K lis  20 11:43 WiredTiger.wt
      ### AFTER ADDING 100000 dbs: ###
      ls -alh
      total 42M
      drwx------ 2 plavi plavi 4,0K lis  19 22:54 .
      drwxrwxr-x 5 plavi plavi  13M lis  19 23:22 ..
      -rw------- 1 plavi plavi 4,6M lis  20 07:22 key.wt
      -rw------- 1 plavi plavi  32K lis  19 21:33 parameters.wt
      -rw------- 1 plavi plavi   46 lis  19 16:35 WiredTiger
      -rw------- 1 plavi plavi 4,0K lis  19 21:33 WiredTigerLAS.wt
      -rw------- 1 plavi plavi   21 lis  19 16:35 WiredTiger.lock
      -rw------- 1 plavi plavi 4,8M lis  19 22:13 WiredTigerLog.0000000006
      -rw------- 1 plavi plavi 4,8M lis  19 22:54 WiredTigerLog.0000000007
      -rw------- 1 plavi plavi 5,0M lis  19 23:21 WiredTigerLog.0000000008
      -rw------- 1 plavi plavi 5,0M lis  19 22:13 WiredTigerPreplog.0000000003
      -rw------- 1 plavi plavi 5,0M lis  19 22:54 WiredTigerPreplog.0000000004
      -rw------- 1 plavi plavi 1,1K lis  19 21:33 WiredTiger.turtle
      -rw------- 1 plavi plavi  32K lis  19 21:33 WiredTiger.wt
      ### AFTER REMOVING 100000 DBS: ###
      plavi@bender  ~/lab/psmdb/bin/percona-server-mongodb-3.6.8-2.0/nodes/db/keydb  ls -alh
      total 58M
      drwx------ 2 plavi plavi 4,0K lis  20 11:28 .
      drwxrwxr-x 5 plavi plavi  13M lis  20 11:30 ..
      -rw------- 1 plavi plavi 5,9M lis  20 11:22 key.wt
      -rw------- 1 plavi plavi  32K lis  19 21:33 parameters.wt
      -rw------- 1 plavi plavi   46 lis  19 16:35 WiredTiger
      -rw------- 1 plavi plavi 4,0K lis  19 21:33 WiredTigerLAS.wt
      -rw------- 1 plavi plavi   21 lis  19 16:35 WiredTiger.lock
      -rw------- 1 plavi plavi 4,8M lis  19 22:13 WiredTigerLog.0000000006
      -rw------- 1 plavi plavi 4,8M lis  19 22:54 WiredTigerLog.0000000007
      -rw------- 1 plavi plavi 4,8M lis  20 11:15 WiredTigerLog.0000000008
      -rw------- 1 plavi plavi 4,8M lis  20 11:22 WiredTigerLog.0000000009
      -rw------- 1 plavi plavi 4,8M lis  20 11:28 WiredTigerLog.0000000010
      -rw------- 1 plavi plavi 5,0M lis  20 11:29 WiredTigerLog.0000000011
      -rw------- 1 plavi plavi 5,0M lis  20 11:15 WiredTigerPreplog.0000000005
      -rw------- 1 plavi plavi 5,0M lis  20 11:28 WiredTigerPreplog.0000000007
      -rw------- 1 plavi plavi 1,1K lis  19 21:33 WiredTiger.turtle
      -rw------- 1 plavi plavi  32K lis  19 21:33 WiredTiger.wt
      > show dbs;
      admin   0.000GB
      config  0.000GB
      local   0.000GB
      ### AFTER RESTART: ###
      plavi@bender  ~/lab/psmdb/bin/percona-server-mongodb-3.6.8-2.0/nodes/db/keydb  ls -alh
      total 28M
      drwx------ 2 plavi plavi 4,0K lis  20 11:40 .
      drwxrwxr-x 5 plavi plavi  13M lis  20 11:40 ..
      -rw------- 1 plavi plavi 176K lis  20 11:40 key.wt
      -rw------- 1 plavi plavi  36K lis  20 11:40 parameters.wt
      -rw------- 1 plavi plavi   46 lis  19 16:35 WiredTiger
      -rw------- 1 plavi plavi 4,0K lis  20 11:40 WiredTigerLAS.wt
      -rw------- 1 plavi plavi   21 lis  19 16:35 WiredTiger.lock
      -rw------- 1 plavi plavi 5,0M lis  20 11:40 WiredTigerLog.0000000011
      -rw------- 1 plavi plavi 5,0M lis  20 11:15 WiredTigerPreplog.0000000005
      -rw------- 1 plavi plavi 5,0M lis  20 11:28 WiredTigerPreplog.0000000007
      -rw------- 1 plavi plavi 1,1K lis  20 11:40 WiredTiger.turtle
      -rw------- 1 plavi plavi  36K lis  20 11:40 WiredTiger.wt
      ### AFTER ADDING AND REMOVING 100000 DATABASES ###
      plavi@bender  ~/lab/psmdb/bin/percona-server-mongodb-3.6.8-2.0/nodes/db/keydb  ls -alh
      total 52M
      drwx------ 2 plavi plavi 4,0K lis  20 13:39 .
      drwxrwxr-x 5 plavi plavi  13M lis  20 13:43 ..
      -rw------- 1 plavi plavi  12K lis  20 12:48 key.wt
      -rw------- 1 plavi plavi 4,0K lis  20 11:43 parameters.wt
      -rw------- 1 plavi plavi   46 lis  20 11:43 WiredTiger
      -rw------- 1 plavi plavi 4,0K lis  20 11:43 WiredTigerLAS.wt
      -rw------- 1 plavi plavi   21 lis  20 11:43 WiredTiger.lock
      -rw------- 1 plavi plavi 4,8M lis  20 12:17 WiredTigerLog.0000000001
      -rw------- 1 plavi plavi 4,8M lis  20 12:37 WiredTigerLog.0000000002
      -rw------- 1 plavi plavi 4,8M lis  20 12:57 WiredTigerLog.0000000003
      -rw------- 1 plavi plavi 4,8M lis  20 13:18 WiredTigerLog.0000000004
      -rw------- 1 plavi plavi 4,8M lis  20 13:39 WiredTigerLog.0000000005
      -rw------- 1 plavi plavi 5,0M lis  20 13:42 WiredTigerLog.0000000006
      -rw------- 1 plavi plavi 5,0M lis  20 12:57 WiredTigerPreplog.0000000005
      -rw------- 1 plavi plavi 5,0M lis  20 13:39 WiredTigerPreplog.0000000007
      -rw------- 1 plavi plavi  890 lis  20 11:43 WiredTiger.turtle
      -rw------- 1 plavi plavi 4,0K lis  20 11:43 WiredTiger.wt
      ### AFTER RESTART ###
      ✘ plavi@bender  ~/lab/psmdb/bin/percona-server-mongodb-3.6.8-2.0/nodes/db/keydb  ls -alh
      total 28M
      drwx------ 2 plavi plavi 4,0K lis  20 19:53 .
      drwxrwxr-x 5 plavi plavi  13M lis  20 19:53 ..
      -rw------- 1 plavi plavi  16K lis  20 19:53 key.wt
      -rw------- 1 plavi plavi  16K lis  20 19:53 parameters.wt
      -rw------- 1 plavi plavi   46 lis  20 11:43 WiredTiger
      -rw------- 1 plavi plavi 4,0K lis  20 19:53 WiredTigerLAS.wt
      -rw------- 1 plavi plavi   21 lis  20 11:43 WiredTiger.lock
      -rw------- 1 plavi plavi 5,0M lis  20 19:53 WiredTigerLog.0000000007
      -rw------- 1 plavi plavi 5,0M lis  20 19:53 WiredTigerPreplog.0000000001
      -rw------- 1 plavi plavi 5,0M lis  20 19:53 WiredTigerPreplog.0000000002
      -rw------- 1 plavi plavi 1,1K lis  20 19:53 WiredTiger.turtle
      -rw------- 1 plavi plavi  32K lis  20 19:53 WiredTiger.wt
      

        Smart Checklist

          Attachments

            Issue Links

              Activity

                People

                • Assignee:
                  igor.solodovnikov Igor Solodovnikov
                  Reporter:
                  igor.solodovnikov Igor Solodovnikov
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  2 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Time Tracking

                    Estimated:
                    Original Estimate - Not Specified
                    Not Specified
                    Remaining:
                    Remaining Estimate - 0 minutes
                    0m
                    Logged:
                    Time Spent - 1 day, 5 hours, 30 minutes
                    1d 5h 30m