on large prometheus instances (more > 50 targets) prometheus shutdown can take a long time (several minutes).
but supervisord and systemd don't wait and kill prometheus after some period.
In this case during prometheus start, it detects non-clean shutdown and run crash recovery, which can take up to 20 minutes on large installations.
so it is needed to configure supervisord and systemd to wait for 5 minutes after SIGTERM and before SIGKILL.
initial reqest was on PL Dublin 2017, also see https://www.percona.com/forums/questions-discussions/percona-monitoring-and-management/49655-pmm-shutdown-flow