Skip to main content
Use alerting to notify your team when an Issue appears, recurs, or becomes frequent. An alert is a reaction to an Issue event. It never creates a second representation of the failure: the Issue and its occurrences remain the record of what happened.

What alerting does

Foveus alerting lets a workspace:
  • create alert channels
  • create Issue alert rules
  • notify a channel when an Issue event matches a rule
  • keep an alert history with the delivery result of every alert
Alerting is configured in the Foveus dashboard. It is not configured through the application SDK or FoveusOptions.

Alert channels

Webhook channels are not supported yet and cannot be created. Rules cannot target a webhook channel. A webhook channel created before this change still exists; alerts routed to it are recorded in alert history as not sent.

Slack channels

A Slack channel requires an incoming webhook URL that starts with https://hooks.slack.com/.

Email channels

An email channel requires a to list of up to 10 recipients. Foveus sends email through ZeptoMail. When email delivery is not configured or fails, the alert is still recorded in alert history, with the delivery error.

Alert rules

Each rule is evaluated on its own. A rule alerts at most once for each Issue event.

New issue alerts

A new_issue rule alerts once when an Issue is created. It alerts again each time a resolved Issue recurs. A reopen is never swallowed by an earlier alert: every reopen requires someone to resolve the Issue first, so each one is worth knowing about. The rule’s cooldown does not apply to new_issue rules. An occurrence that happened before the Issue was resolved, but arrived late, is counted but does not reopen the Issue.

Threshold alerts

A threshold_exceeded rule has a threshold (thresholdCount), a rolling detection window (windowMinutes: 1, 5, 15, 30, 60, 120, 360, 720 or 1440 minutes) and a cooldown (cooldownMinutes). The rolling count is the number of the Issue’s occurrences in the last windowMinutes, counted by when each occurrence happened, not when Foveus received it. An occurrence exactly windowMinutes older than the latest one is outside the window. The rule:
  • alerts once when the count reaches the threshold (the crossing);
  • does not alert again while the count stays at or above the threshold;
  • alerts again after the count has fallen below the threshold and crosses it again, unless that new crossing is within max(cooldownMinutes, windowMinutes) of the previous threshold alert for the same Issue.
For example, with a threshold of 3 in 5 minutes, occurrences 1 and 2 do nothing, occurrence 3 alerts, and occurrences 4, 5, 6 inside the window do not alert again. Because evaluation uses occurrence time, the result does not depend on how telemetry was batched or when the background worker ran. Telemetry that arrives late (for example after an outage) is evaluated at the time it happened. A late occurrence that belongs to a burst Foveus already alerted on does not alert again.

An Issue created above the threshold

When a single batch creates an Issue that already has enough occurrences to cross a threshold, each matching rule alerts once: new_issue rules report the new Issue, and threshold_exceeded rules report the crossing. If both kinds of rule route to the same channel, that channel receives two alerts, one per rule.

Rule scope

Failure types are exception, http_error, timeout, business_rule, execution_error, dlq, retry_exhausted and sla_breach.

Alert history

Every alert is recorded before Foveus tries to deliver it. Alert history shows:
  • the Issue and the rule
  • what the alert reacts to: a new Issue, a reopen, or a threshold crossing (with the time of the crossing)
  • the channel
  • whether delivery succeeded, and the delivery error when it did not
A failed or unconfigured channel never removes the alert from history. Start with two rules:

Alerting, Issues and executions

Alerts fire from Issue events, not from individual log lines or requests. An Issue describes an observed failure pattern; it is not a confirmed root cause.

Troubleshooting

An alert is in history but was not delivered

Open the alert in history and read its delivery error. Common causes: the Slack webhook URL is wrong, email delivery is not configured, or the channel is a webhook channel.

A threshold alert did not fire

Check that:
  • the Issue reached the threshold within the window, counted by occurrence time
  • the Issue matches the rule’s service and failure-type scope
  • the crossing was not within max(cooldownMinutes, windowMinutes) of the previous threshold alert for that Issue
  • the rule and its channel are enabled

A reopen alert did not fire

Check that the recurrence happened after the Issue was resolved, and that a new_issue rule matches the Issue’s service and failure type.

What to do next