Details
-
New Feature
-
Status: Done
-
Medium
-
Resolution: Done
-
None
-
None
-
1
-
Platform Sprint 18, Platform Sprint 19
-
Yes
-
Yes
-
Yes
-
No
Description
The Goal: Let people mark some timestamps on the graphs so they will have some visual marker when they look at the graph.
Annotation - "sticky note on the Graphs". They are visual only, no analytics or anything else on top of them.
Use cases and user interactions:
Case 1: SysAdmin did an update of some system library on the Node and expect this might affect Node or Database performance. He/She want to have a note about this
As a PMM User, I need to be able to run pmm-admin command and add information about changes I've made as annotation, so that in PMM I'll be able to see this information when I will be looking to OS Dashboards for this Node
Example:
pmm-admin annotate [--node] "OS upgrade" [--tags "14.04-20.04"] [--node-name=some-other-node-name]
Note: By default, the annotation should be added to the current Node.
Case 2: SysAdmin did a restart of Database Server and expect this might affect Database performance. He/She want to have a note about this
Example:
pmm-admin annotate [--service] "Service Restart" [--tags "manual,second-tag"] [--service-name=mysql5.6]
Note: we need to pass the Service Name if there more then one Service running on the current node
Case 3: DevOps need to mark moments when new configuration applied on some Nodes or services, so he/she needs and API to specify Node Name, Service Name Annotation next and additional tags, if needed
DOD:
Ability to add annotation related to exact service or node. Annotations added via CLI or API should be visible in grafana.
Annotations added for certain service should have tag with service name and shouldn't have tag `pmm_annotations`. To the text we should add "(Service Name: Name_of_the _service )"
Annotations added for certain node should have tag with node name and shouldn't have tag `pmm_annotations`. To the text we should add "(Node Name: Node_name )"
New flags:
--service
--service-name
--node
--node-name
All cases examples:
annotate "global annotation"
annotate --service "annotation for current service"
annotate --service-name=mysql57 "annotation for mysql57"
annotate --node "annotation for current service"
annotate --node-name=mysql57 "annotation for mysql57"
If given name doesnt exist then no annotation will be added.
Cases for test:
--node = current node
--node-name = node with name
--node --node-name = node with name
--node --service-name = current node and service with name
--node --node-name --service-name = node with name and service with name
--node --service = current node and all services of current node
-node --node-name --service - -service-name = service with name and node with name
--service = all services of current node
--service-name = service with name
--service --service-name = service with name
--service --node-name = all services of current node and node with name
--service-name --node-name = service with name and node with name
--service --service-name -node-name = service with name and node with name
Out of scope:
- Removing and Updating existing annotations.
- We don't save any data about annotations in pmm-managed.
- Grafana UI part
- Implement any analytics on top of Annotations. For Structured objects, we'll do Events later. Events will be designed to have stronger relations to Inventory model and strict
- if we add annotations for node we don't have to add annotations for services running on that node
Suggested implementation:
- API
- New endpoint in pmm-managed API
- Validate if service or node is exists
- Create grafana annotation via grafana API with extra tag (service name or node name) (https://grafana.com/docs/grafana/latest/http_api/annotations/)
- New endpoint in pmm-managed API
- CLI
- Node
- If node-name is not set
- get current node id from pmm-agent
- request node-name from pmm-server
- if node-name is set just send request
- If node-name is not set
- Service
- If service name is not set
- get current node id from pmm-agent
- request all services for current node from pmm-serve
- if node-name is set just send request
- If service name is not set
- Node
Discussion:
- Ability to add annotation per service
- is it worth add annotations to All Services on the current node if Annotation to Node added? Looks like - yes, because Service "should inherit labels from Node "
- no
- Figure out how --service and --current-node will work.
How it can be tested directly through Grafana API:
- Add an annotation for a node
# curl 'http://admin:[email protected]/graph/api/annotations' -H 'Content-Type: application/json;charset=UTF-8' --data-binary '{"time":1585053244718,"tags":["pmm-server"],"text":"Outage (Node Name: pmm-server)"}' {"id":1,"message":"Annotation added"}
- Check the annotation on dashboards for node pmm-server
- Add an annotation for a service
# curl 'http://admin:[email protected]/graph/api/annotations' -H 'Content-Type: application/json;charset=UTF-8' --data-binary '{"time":1585053244718,"tags":["ps_5.7_0.0.0.0_1"],"text":"Alarm!!! (Service Name: ps_5.7_0.0.0_1)"}' {"id":2,"message":"Annotation added"}
Attachments
Issue Links
- causes
-
PMM-6904 'pmm-admin annotate' command crashes for non-generic node types
-
- Done
-
- clones
-
PMM-5559 Global annotations can now be set with the pmm-admin annotate command.
-
- Done
-
- relates to
-
PMM-2562 Add support for templated variables (host) in annotations
-
- Done
-
-
PMM-5264 Annotation filters - remove $service and use AND
-
- Open
-
-
PMM-6198 Annotation intended only for Service also appears on Node dashboards
-
- Done
-
-
PMM-6143 Add more user-friendly error messages
-
- Open
-
- links to