Response time alerts
Spectate records the response time of every check. With response time alerts on, a monitor also fires an alert when it answers, but too slowly. There are two thresholds: a hard limit in milliseconds, and a rise in percent compared with the monitor's own recent response times.
Turn on response time alerts
- Open the monitor.
- Select the Settings tab.
- Under Alerting, turn on Slow responses.
- Set Hard threshold, Deviation, Baseline window and Compared against.
- Select Save changes.
Response time alerts are off for a new monitor. In the API, the switch is performance_check_enabled.
Hard threshold
Hard threshold is a response time in milliseconds, 1 to 30,000. The dashboard suggests 5,000.
Spectate compares each successful check with the hard threshold. If the response time is above it, Spectate fires the alert "Response time exceeded threshold for NAME" with the severity high. The alert shows the response time and the threshold.
The alert resolves at the first successful check that is at or below the threshold. Failed checks do not count. A failed check is a down problem, not a response time problem.
Deviation from the baseline
The deviation alert finds a monitor that gets slower than usual, also when it stays below the hard threshold. Every 5 minutes, Spectate compares two median response times:
- The median of the last Baseline window, for example the last 15 minutes.
- The median of the Compared against period before that window, for example the 2 hours before.
If the recent median is higher than the earlier median by Deviation percent or more, Spectate fires the alert "Response time deviated for NAME". The severity is medium. The alert resolves when a later comparison finds a rise below Deviation.
For example, the earlier median is 200 ms and Deviation is 100%. The alert fires when the recent median reaches 400 ms.
| Setting | Values | Default in the dashboard | API field |
|---|---|---|---|
| Deviation | 1 to 500 percent | 100 | performance_deviation_percentage |
| Baseline window | 15 minutes to 2 hours, in steps of 15 minutes | 15 minutes | performance_deviation_timeframe, in minutes |
| Compared against | 1 to 24 hours, in whole hours | 2 hours | performance_deviation_before_timeframe, in minutes |
Baseline window must be shorter than Compared against. Spectate needs checks in both periods. Each period must hold enough checks for a useful median. For example, with an interval of 5 minutes, a 15-minute window holds 3 checks.
How the two thresholds work together
- A monitor has one response time alert at a time. While one fires, Spectate does not fire a second.
- If you set only Hard threshold, a check at or below it resolves the alert.
- If you leave a threshold empty in the API, Spectate does not use it.
performance_hard_threshold: nullturns off the hard threshold.performance_deviation_percentage: nullturns off the deviation check. - Spectate does not compare response times of a paused monitor.
Response time alerts follow the same escalation paths as the other alerts of the monitor. Because they have a lower severity than a down alert, a path can branch on severity and send them to a quieter channel.
See the response times
The Performance tab of a monitor shows the response times for a period you select. It shows Average, p50, p95, Apdex and the number of Samples. For HTTP and HTTPS monitors, it also splits each request into phases: DNS lookup, connection, TLS handshake and time to first byte.
All response times come from the monitor's primary region. A monitor in Europe that checks a server in Asia has a longer response time than a monitor in Asia. See Locations and checks.