Details
-
Bug
-
Status: Done
-
Critical
-
Resolution: Fixed
-
None
-
Yes
-
1
-
Yes
-
Yes
-
No
-
Server Features
-
Description
User impact:
No QAN data for mongodb with auth enabled after restore
Steps to reproduce:
- Start PMM Server Backup GA
- setup mongo replica set with auth enabled + pbm 2 and add it to monitoring with pmm-admin
- setup s3 location and start on demand backup
- trigger restore and explore pmm-agent logs
Actual result:
- restore succeeded, but a lot of errors are coming from qan_profiler_agent
- No QAN data
Expected result:
- Restore should be successful and QAN should still get data
Note:
pmm-agent log:
INFO[2022-10-25T12:58:00.354+00:00] Job started. component=runner id=/job_id/7663749d-4d21-4143-96e0-5c626cb500a9 type=mongodb_restore INFO[2022-10-25T12:58:00.355+00:00] Configuring S3 location. id=/job_id/7663749d-4d21-4143-96e0-5c626cb500a9 name="b1_2022-10-25T12:57:19Z" type=mongodb_restore INFO[2022-10-25T12:58:00.473+00:00] Waiting for no running pbm operations. id=/job_id/7663749d-4d21-4143-96e0-5c626cb500a9 name="b1_2022-10-25T12:57:19Z" type=mongodb_restore INFO[2022-10-25T12:58:06.154+00:00] Finding backup entity name. id=/job_id/7663749d-4d21-4143-96e0-5c626cb500a9 name="b1_2022-10-25T12:57:19Z" type=mongodb_restore INFO[2022-10-25T12:58:07.258+00:00] starting backup restore for: 2022-10-25T12:57:26Z. id=/job_id/7663749d-4d21-4143-96e0-5c626cb500a9 name="b1_2022-10-25T12:57:19Z" type=mongodb_restore INFO[2022-10-25T12:58:07.341+00:00] waiting for pbm restore id=/job_id/7663749d-4d21-4143-96e0-5c626cb500a9 name="b1_2022-10-25T12:57:19Z" type=mongodb_restore WARN[2022-10-25T12:58:15.117+00:00] couldn't retrieve data from cursor (Unauthorized) command getMore requires authentication agentID=/agent_id/de57be6e-6172-430e-a747-e702d394436f component=agent-builtin db=admin type=qan_mongodb_profiler_agent WARN[2022-10-25T12:58:15.264+00:00] couldn't retrieve data from cursor (Unauthorized) command getMore requires authentication agentID=/agent_id/de57be6e-6172-430e-a747-e702d394436f component=agent-builtin db=config type=qan_mongodb_profiler_agent WARN[2022-10-25T12:58:15.717+00:00] couldn't retrieve data from cursor (Unauthorized) command getMore requires authentication agentID=/agent_id/de57be6e-6172-430e-a747-e702d394436f component=agent-builtin db=test type=qan_mongodb_profiler_agent WARN[2022-10-25T12:58:15.723+00:00] couldn't retrieve data from cursor (Unauthorized) command getMore requires authentication agentID=/agent_id/de57be6e-6172-430e-a747-e702d394436f component=agent-builtin db=local type=qan_mongodb_profiler_agent ERRO[2022-10-25T12:58:16.122+00:00] couldn't create system.profile iterator, reason: (Unauthorized) command find requires authentication agentID=/agent_id/de57be6e-6172-430e-a747-e702d394436f component=agent-builtin db=admin type=qan_mongodb_profiler_agent ERRO[2022-10-25T12:58:16.267+00:00] couldn't create system.profile iterator, reason: (Unauthorized) command find requires authentication agentID=/agent_id/de57be6e-6172-430e-a747-e702d394436f component=agent-builtin db=config type=qan_mongodb_profiler_agent ERRO[2022-10-25T12:58:16.722+00:00] couldn't create system.profile iterator, reason: (Unauthorized) command find requires authentication agentID=/agent_id/de57be6e-6172-430e-a747-e702d394436f component=agent-builtin db=test type=qan_mongodb_profiler_agent ERRO[2022-10-25T12:58:16.727+00:00] couldn't create system.profile iterator, reason: (Unauthorized) command find requires authentication agentID=/agent_id/de57be6e-6172-430e-a747-e702d394436f component=agent-builtin db=local type=qan_mongodb_profiler_agent ERRO[2022-10-25T12:58:17.127+00:00] couldn't create system.profile iterator, reason: (Unauthorized) command find requires authentication agentID=/agent_id/de57be6e-6172-430e-a747-e702d394436f component=agent-builtin db=admin type=qan_mongodb_profiler_agent ERRO[2022-10-25T12:58:17.269+00:00] couldn't create system.profile iterator, reason: (Unauthorized) command find requires authentication agentID=/agent_id/de57be6e-6172-430e-a747-e702d394436f component=agent-builtin db=config type=qan_mongodb_profiler_agent ERRO[2022-10-25T12:58:17.736+00:00] couldn't create system.profile iterator, reason: (Unauthorized) command find requires authentication agentID=/agent_id/de57be6e-6172-430e-a747-e702d394436f component=agent-builtin db=test type=qan_mongodb_profiler_agent ERRO[2022-10-25T12:58:18.129+00:00] couldn't create system.profile iterator, reason: (Unauthorized) command find requires authentication agentID=/agent_id/de57be6e-6172-430e-a747-e702d394436f component=agent-builtin db=admin type=qan_mongodb_profiler_agent ERRO[2022-10-25T12:58:18.271+00:00] couldn't create system.profile iterator, reason: (Unauthorized) command find requires authentication agentID=/agent_id/de57be6e-6172-430e-a747-e702d394436f component=agent-builtin db=config type=qan_mongodb_profiler_agent ERRO[2022-10-25T12:58:18.737+00:00] couldn't create system.profile iterator, reason: (Unauthorized) command find requires authentication agentID=/agent_id/de57be6e-6172-430e-a747-e702d394436f component=agent-builtin db=test type=qan_mongodb_profiler_agent ERRO[2022-10-25T12:58:19.131+00:00] couldn't create system.profile iterator, reason: (Unauthorized) command find requires authentication agentID=/agent_id/de57be6e-6172-430e-a747-e702d394436f component=agent-builtin db=admin type=qan_mongodb_profiler_agent ERRO[2022-10-25T12:58:19.276+00:00] couldn't create system.profile iterator, reason: (Unauthorized) command find requires authentication agentID=/agent_id/de57be6e-6172-430e-a747-e702d394436f component=agent-builtin db=config type=qan_mongodb_profiler_agent ERRO[2022-10-25T12:58:20.277+00:00] couldn't create system.profile iterator, reason: (Unauthorized) command find requires authentication agentID=/agent_id/de57be6e-6172-430e-a747-e702d394436f component=agent-builtin db=config type=qan_mongodb_profiler_agent WARN[2022-10-25T12:58:20.563+00:00] couldn't retrieve data from cursor (Unauthorized) command getMore requires authentication agentID=/agent_id/de57be6e-6172-430e-a747-e702d394436f component=agent-builtin db=local type=qan_mongodb_profiler_agent WARN[2022-10-25T12:58:20.743+00:00] couldn't retrieve data from cursor (Unauthorized) command getMore requires authentication agentID=/agent_id/de57be6e-6172-430e-a747-e702d394436f component=agent-builtin db=test type=qan_mongodb_profiler_agent ERRO[2022-10-25T12:58:21.281+00:00] couldn't create system.profile iterator, reason: (Unauthorized) command find requires authentication agentID=/agent_id/de57be6e-6172-430e-a747-e702d394436f component=agent-builtin db=config type=qan_mongodb_profiler_agent ERRO[2022-10-25T12:58:21.568+00:00] couldn't create system.profile iterator, reason: (Unauthorized) command find requires authentication agentID=/agent_id/de57be6e-6172-430e-a747-e702d394436f component=agent-builtin db=local type=qan_mongodb_profiler_agent ERRO[2022-10-25T12:58:21.747+00:00] couldn't create system.profile iterator, reason: (Unauthorized) command find requires authentication agentID=/agent_id/de57be6e-6172-430e-a747-e702d394436f component=agent-builtin db=test type=qan_mongodb_profiler_agent ERRO[2022-10-25T12:58:22.283+00:00] couldn't create system.profile iterator, reason: (Unauthorized) command find requires authentication agentID=/agent_id/de57be6e-6172-430e-a747-e702d394436f component=agent-builtin db=config type=qan_mongodb_profiler_agent INFO[2022-10-25T12:58:22.416+00:00] Job finished.