New connections going to pxc-0, but existing connections still able to execute DML/DDL queries on pxc-2
The issue could be fixed by adding hard-stop-after 10s
to the global haproxy configuration
Environment
None
Activity
Show:
Nickolay Ihalainen December 22, 2023 at 12:43 PM
correct, after 10 seconds connection will be closed and primary role fully switches to pxc-0. Existing queries are rolled back and application could retry the transaction. PXC is not supporting single primary mode and such urgent shutdown prevents performance drops/transaction rollbacks (due to optimistic locking).
ege.gunes December 22, 2023 at 11:38 AM
I confirm that the problem happens and hard-stop-after 10s fixes the issue. But we still have a 10 second time window that the query can still end up in pod-2 right?
Create the cluster (I’m using ./anydbver deploy k8s-pxc:1.13.0 , default cr.yaml with RollingUpdate)
Put pxc-0 down:
kubectl -n pxc exec -it cluster1-pxc-0 -c pxc -- bash -c 'touch /var/lib/mysql/sleep-forever;mysql -uroot -p"$MYSQL_ROOT_PASSWORD" -e shutdown'Connect via haproxy:
kubectl -n pxc exec -it cluster1-pxc-2 -c pxc -- bash ; mysql -uroot -p$MYSQL_ROOT_PASSWORD -h cluster1-haproxySELECT @hostname returns pxc-2
Fix pxc-0: kubectl -n pxc exec -it cluster1-pxc-0 -c pxc -- rm -f /var/lib/mysql/sleep-forever
New connections going to pxc-0, but existing connections still able to execute DML/DDL queries on pxc-2
The issue could be fixed by adding
hard-stop-after 10sto the global haproxy configuration