Uploaded image for project: 'ProxySQL Admin Scripts'
  1. ProxySQL Admin Scripts
  2. PSQLADM-6

Improve adding/removing PXC nodes

    XMLWordPrintable

    Details

    • Type: New Feature
    • Status: Done
    • Priority: Medium
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      The proxysql_node_monitor adds and removes PXC nodes based on their status in the cluster. For example a node that is shutdown, first switches into OFFLINE_SOFT and later is removed entirely, like in the example log here:

      Mon Nov 27 11:20:16 UTC 2017 ###### Loading mysql_servers config into runtime ######
      Mon Nov 27 11:20:19 UTC 2017 Cluster node (10:172.17.0.2:3306) current status 'ONLINE' in ProxySQL database!
      Mon Nov 27 11:20:19 UTC 2017 Cluster node (10:172.17.0.4:3306) current status 'ONLINE' in ProxySQL database!
      Mon Nov 27 11:20:19 UTC 2017 Cluster node (10:172.17.0.3:3306) current status 'ONLINE' in ProxySQL database!
      Mon Nov 27 11:20:20 UTC 2017 ###### Loading mysql_servers config into runtime ######
      Mon Nov 27 11:20:23 UTC 2017 Cluster node (10:172.17.0.2:3306) current status 'ONLINE' in ProxySQL database!
      Mon Nov 27 11:20:23 UTC 2017 Cluster node (10:172.17.0.4:3306) current status 'ONLINE' in ProxySQL database!
      Mon Nov 27 11:20:23 UTC 2017 Cluster node (10:172.17.0.3:3306) current status 'OFFLINE_SOFT' in ProxySQL database!
      Mon Nov 27 11:20:23 UTC 2017 Percona XtraDB Cluster membership looks good
      Mon Nov 27 11:20:26 UTC 2017 Cluster node (10:172.17.0.2:3306) current status 'ONLINE' in ProxySQL database!
      Mon Nov 27 11:20:26 UTC 2017 Cluster node (10:172.17.0.4:3306) current status 'ONLINE' in ProxySQL database!
      Mon Nov 27 11:20:26 UTC 2017 Cluster node (10:172.17.0.3:3306) current status 'OFFLINE_SOFT' in ProxySQL database!
      Mon Nov 27 11:20:26 UTC 2017 Percona XtraDB Cluster membership looks good
      Mon Nov 27 11:20:29 UTC 2017 Cluster node (10:172.17.0.2:3306) current status 'ONLINE' in ProxySQL database!
      Mon Nov 27 11:20:29 UTC 2017 Cluster node (10:172.17.0.4:3306) current status 'ONLINE' in ProxySQL database!
      Mon Nov 27 11:20:29 UTC 2017 Cluster node (10:172.17.0.3:3306) current status 'OFFLINE_SOFT' in ProxySQL database!
      Mon Nov 27 11:20:29 UTC 2017 Percona XtraDB Cluster membership looks good
      Mon Nov 27 11:20:32 UTC 2017 Node 10:172.17.0.3:3306 does not exists in cluster membership!
      Mon Nov 27 11:20:32 UTC 2017 Removing Node 10:172.17.0.3:3306 from ProxySQL database!
      Mon Nov 27 11:20:32 UTC 2017 Cluster node (10:172.17.0.2:3306) current status 'ONLINE' in ProxySQL database!
      Mon Nov 27 11:20:32 UTC 2017 Cluster node (10:172.17.0.4:3306) current status 'ONLINE' in ProxySQL database!

      However, in many cases, we may actually wish to see how many nodes are/were in total in the cluster and what is their current state, including if they are completely unavailable.

      So I think it should be configurable whether nodes are removed completely or just switched to OFFLINE state. And maybe even the non-removing behavior should be default.

       

      Moreover, when there is only one node left in the cluster, logging for some reason stops printing remaining node details:

      Mon Nov 27 11:26:06 UTC 2017 Cluster node (10:172.17.0.2:3306) current status 'ONLINE' in ProxySQL database!
      Mon Nov 27 11:26:06 UTC 2017 Cluster node (10:172.17.0.4:3306) current status 'OFFLINE_SOFT' in ProxySQL database!
      Mon Nov 27 11:26:06 UTC 2017 Percona XtraDB Cluster membership looks good
      Mon Nov 27 11:26:10 UTC 2017 Cluster node (10:172.17.0.2:3306) current status 'ONLINE' in ProxySQL database!
      Mon Nov 27 11:26:10 UTC 2017 Cluster node (10:172.17.0.4:3306) current status 'OFFLINE_SOFT' in ProxySQL database!
      Mon Nov 27 11:26:10 UTC 2017 Percona XtraDB Cluster membership looks good
      Mon Nov 27 11:26:13 UTC 2017 Node 10:172.17.0.4:3306 does not exists in cluster membership!
      Mon Nov 27 11:26:13 UTC 2017 Removing Node 10:172.17.0.4:3306 from ProxySQL database!
      Mon Nov 27 11:26:13 UTC 2017 Cluster node (10:172.17.0.2:3306) current status 'ONLINE' in ProxySQL database!
      Mon Nov 27 11:26:13 UTC 2017 ###### Loading mysql_servers config into runtime ######
      Mon Nov 27 11:26:17 UTC 2017 ###### Loading mysql_servers config into runtime ######
      Mon Nov 27 11:26:20 UTC 2017 ###### Loading mysql_servers config into runtime ######

      mysql> select hostgroup_id,hostname,status,weight,comment from mysql_servers;
      ---------------------------------------------
      | hostgroup_id | hostname   | status | weight | comment   |
      ---------------------------------------------
      | 10           | 172.17.0.2 | ONLINE | 1000   | READWRITE |
      ---------------------------------------------
      1 row in set (0.00 sec)

      Which is just confusing. I think logging should be consistent regardless of how many nodes are still online.

        Smart Checklist

          Attachments

            Issue Links

              Activity

                People

                • Assignee:
                  ramesh.sivaraman Ramesh Sivaraman
                  Reporter:
                  przemyslaw.malkowski@percona.com Przemyslaw Malkowski
                • Votes:
                  1 Vote for this issue
                  Watchers:
                  3 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved: