Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

ProxySQL Active Frontend Connections graph is unusable with +10k clients

Description

When thousands of Client connections are established against ProxySQL, the performance of Active Frontend Connections leads to a graph that doesn't render. Further this can make the rendering and general usage of ProxySQL Instance Summary dashboard feel very sluggish.
We should find a way to use topk or other limiter in order to reduce the impact of analyzing +10k metric series, specifically series proxysql_processlist_client_connection_list
This is expected to be a somewhat common occurrence as one of the value propositions of ProxySQL is the connection pooling feature, so thousands of client connections is expected.

Steps to Reproduce

  1. Launch PMM Server against ProxySQL v2.4 + PXC or Percona Server for MySQL

  2. Launch 10,000 client connections against ProxySQL

  3. Load ProxySQL Instance Summary dashboard

How to test

Check Active Frontend Connection panel with 10000 client connections.

How to document

None

AFFECTED CS IDs

https://percona.service-now.com/nav_to.do?uri=u_ps_project.do?sys_id=50233173878ed1904035b8c9cebb354f

Attachments

4

Activity

Show:

Ihor Cherkasov March 22, 2023 at 12:40 AM
Edited

Verified on FB: https://github.com/Percona-Lab/pmm-submodules/pull/3104#issuecomment-1478004547

Now graph shows Top 30 IP addresses with the highest amount of connections:

Ihor Cherkasov March 22, 2023 at 12:38 AM

Hello! It happened because the developer, who was responsible for the fix was on PTO and I needed to discuss my concerns about possible issue before reopening it. Yesterday we had a discussion and my concerns were not in vain. Fix is ready now and it looks good for me now.

Ihor Cherkasov March 2, 2023 at 8:55 PM
Edited

Tested on FB: https://github.com/Percona-Lab/pmm-submodules/pull/3104#issuecomment-1451927140

Now dashboard's name is Top 30 Active Frontend Connections

Performance of MySQL / ProxySQL Instance Summary looks good to me

But on hovering the graph, it shows accumulated sum, comparing to the previous value, but should show the actual connections amount for each IP (without accumulating):

Please fix it and return back for testing. Thanks!

Aaditya Dubey February 13, 2023 at 11:59 AM

Hi  , ,

Thank you for the report.
Will check and get back to you.

Michael Coburn February 7, 2023 at 6:38 PM

I wasn't able to solve it during my engagement with the customer, instead I basically "turned off" the graph by disabling the query from running (when I made a copy of ProxySQL Instance Summary dashboard)

We probably need some iteration of using topk() formula-fu to make it work performantly , what do you think

Done

Details

Assignee

Reporter

Priority

Components

Labels

Needs QA

Yes

Needs Doc

No

Planned Version/s

Fix versions

Story Points

Affects versions

Parent

Smart Checklist

Created February 6, 2023 at 6:06 PM
Updated October 9, 2024 at 8:36 AM
Resolved March 29, 2023 at 7:11 AM