Details

    • Type: Bug
    • Status: Done
    • Priority: Medium
    • Resolution: Done
    • Affects Version/s: 5.7.17-13, 5.6.31-77.0
    • Fix Version/s: 5.7.18-16, 5.6.36-82.1
    • Component/s: TokuDB
    • Labels:
      None

      Description

      Contribution from Rich Prohaska and Kristian Nielsen @ MariaDB

      Rich Prohaska
      Sep 14
      
      to me, Kristian 
      Hello George,
      
      Kristian Nielsen and I have been working on getting optimistic parallel replication working effectively with TokuDB.  We needed to fix a couple of bogus stalls in the fractal tree's lock manager that occur due to critical section races.  The stall manifest as a thread (transaction) waiting for an available lock until its lock timer pops.  In addition, the optimistic parallel replication feature needs a new mechanism to terminate any client threads that are waiting on a lock. 
      
      Please see this page for details:
      http://prohaska7.blogspot.com/2016/09/races-in-fractal-tree-lock-manager.html
      
      The code is here:
      https://github.com/percona/PerconaFT/pull/360
      
      Thanks
      Rich Prohask
      
      Rich Prohaska
      Sep 14
      
      to me, Kristian 
      Hello George,
      
      MariaDB's optimistic parallel replication uses a new MySQL API (something like THD_report_wait_for) to notify the optimistic scheduler that a lock conflict was encountered.  In addition, it uses a handlerton api (something like kill_query) to cause the storage engine to wakeup any waiting client threads.  These changes are not included in the push request as I thought that they were specific to MariaDB.  However, I have not been following the algorithms being used in MySQL replication (especially 5.7 and 8.0), so maybe they have something similar.  
      
      
      Kristian Nielsen <knielsen@knielsen-hq.org>
      Sep 14
      
      to Rich, me 
      Rich Prohaska <prohaska7@gmail.com> writes:
      
      > MariaDB's optimistic parallel replication uses a new MySQL API (something
      > like THD_report_wait_for) to notify the optimistic scheduler that a lock
      > conflict was encountered.  In addition, it uses a handlerton api (something
      > like kill_query) to cause the storage engine to wakeup any waiting client
      > threads.  These changes are not included in the push request as I thought
      > that they were specific to MariaDB.  However, I have not been following the
      > algorithms being used in MySQL replication (especially 5.7 and 8.0), so
      > maybe they have something similar.
      
      The kill_query handlerton API is standard and has been in both MySQL and
      MariaDB for a long time. It would make sense to include it in TokuDB for
      Percona Server, it should work the same as in MariaDB without modifications.
      This allows a user to use KILL QUERY to interrupt a query that is waiting
      for a row lock.
      
      The thd_report_wait_for API is specific to MariaDB and optimistic parallel
      replication. So it will go in under #ifdef MARIADB_BASE_VERSION, and will
      not directly benefit Percona Server.
      
      MySQL 5.7 does have something similar to thd_report_wait_for(). It is called
      thd_report_row_lock_wait(), and actually is an identical API (except for the
      function name) to the MariaDB thd_report_wait_for(), as seen from the
      storage engine.
      
      However, MySQL 5.7 does not have optimistic replication, nor the ability to
      break engine row locks in case of conflict. The thd_report_row_lock_wait()
      only sets a flag that is used by the MySQL replication manager at the end of
      commit, to handle a few InnoDB corner cases. From my limited knowledge of
      the MySQL replication code, there would not be a benefit for TokuDB to
      implement the MySQL 5.7 thd_report_row_lock_wait() API currently.
      
      I do not know what the policy is for TokuDB to upstream MariaDB-specific
      code like what Rich mentions. From my side, we can upstream it, or I can
      maintain it separately in MariaDB, either way works fine for me. The changes
      are minor, anyway, after Rich's fixes to the locktree code.
      
      Hope this helps,
      
       - Kristian.
      
      George O. Lorch III <george.lorch@percona.com>
      Sep 15
      
      to Rich, Kristian, bcc: me 
      Hey Rich,
      
      I found this old issue in the Tokutek JIRA that seems to describe the same issue you guys just fixed https://tokutek.atlassian.net/browse/FT-332
      
      Rich Prohaska
      Sep 15
      
      to me 
      Yes, the first paragraph describes one of the problems that I fixed (the release before wait problem). The second paragraph describes a solution to a scaling and fairness problem with the lock retry algorithm.  I would consider a different change that attaches pending lock requests to the lock that causes the conflict.  When the lock is release, O(1) time is spent to reassign the lock.
      

        Smart Checklist

          Attachments

            Activity

              People

              • Assignee:
                vladislav.lesin Vladislav Lesin (Inactive)
                Reporter:
                george.lorch George Lorch
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: