Quickly create an AlertManager silence through the API

This article was last updated on: February 7, 2024 pm

overview

Usually when we want to silence the alert of an AlertManager, we need to operate through the UI interface, as shown below:

AlertManager silence

It’s a bit inefficient and not automated enough, so is there a way to quickly create an AlertManager silence?

– Yes, via API.

API Payload

v1

As follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
{
"matchers": [
{
"name": "alername1",
"value": ".*",
"isRegex": true
}
],
"startsAt": "2022-04-29T22:12:33.533Z",
"endsAt": "2022-04-29T23:11:44.603Z",
"createdBy": "api",
"comment": "Silence",
"status": {
"state": "active"
}
}

v2

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
{
"matchers": [
{
"name": "service",
"value": "rancher",
"isRegex": false,
"isEqual": true
},
{
"name": "alertname",
"value": "TargetDown",
"isRegex": false,
"isEqual": true
}
],
"startsAt": "2022-04-29T10:11:35.656Z",
"endsAt": "2022-04-29T12:11:35.656Z",
"createdBy": "Casey Cui",
"comment": " 配置错误导致的误报 ",
"id": null
}

Concrete implementation

curl implementation

📝 Notes:

Take the v1 API as an example

As follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
curl https://alertmanager.e-whisper.com/api/v1/silences -d '{
"matchers": [
{
"name": "alername1",
"value": ".*",
"isRegex": true
}
],
"startsAt": "2022-04-29T22:12:33.533Z",
"endsAt": "2022-04-29T23:11:44.603Z",
"createdBy": "api",
"comment": "Silence",
"status": {
"state": "active"
}
}'

📝Notes:
In a K8S cluster, the address is typically:http://alertmanager:9093

Python implementation

As follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
#!/usr/bin/python3
import requests
import socket
import datetime
import time

res = requests.post("http://alertmanager:9093/api/v2/silences", json={
"matchers": [
{"name": "job", "value": "myjob", "isRegex": False},
{"name": "instance", "value": "{}:1234".format(socket.gethostname()), "isRegex": False},
],
"startsAt": datetime.datetime.utcfromtimestamp(time.time()).isoformat(),
"endsAt": datetime.datetime.utcfromtimestamp(time.time() + 4*3600).isoformat(),
"comment": "Backups on {}".format(socket.gethostname()),
"createdBy": "My backup script",
},
)
res.raise_for_status()
silenceId = res.json()["silenceID"]

Script to remove silence:

1
2
res = requests.delete("http://alertmanager:9093/api/v2/silence/{}".format(silenceId))
res.raise_for_status()

EOF


Quickly create an AlertManager silence through the API
https://e-whisper.com/posts/13347/
Author
east4ming
Posted on
April 29, 2022
Licensed under