Alerts Guide

Set up alerts to get notified when error rates spike in your logs. Krafter monitors your log entries and triggers notifications via email or webhook when a threshold is exceeded within a time window.

How Alerts Work

An alert defines a rule: "Notify me when N or more logs at a given level occur within M minutes."

For example, an alert configured with level: "error", threshold: 10, and window_minutes: 5 will trigger when 10 or more error-level log entries are recorded within any 5-minute window.

How Alerts Evaluate

1. Schedule. Krafter checks every enabled alert every minute in the background.
2. Window. For each alert, Krafter counts log entries at the configured level over the past window_minutes minutes (relative to the moment of evaluation — not a fixed wall-clock window).
3. Threshold. If the count is at or above threshold (i.e. count >= threshold), the alert fires and notifications are dispatched.
4. Deduplication. After firing, Krafter records last_triggered_at and skips re-firing until at least window_minutes minutes have passed since last_triggered_at. So with window_minutes: 5, an active alert can fire at most once every 5 minutes — even if the threshold is crossed continuously, and even though the check itself runs every minute.

Choosing window_minutes

The check runs every minute and each run counts the last window_minutes minutes, so consecutive runs cover the timeline without a gap for any window of 1 minute or more. A larger window overlaps — two runs may count the same entry — which is harmless: the deduplication below keeps one alert from firing twice.

window_minutes: 1 leaves no slack: it is exactly the check interval, so a run delayed by a few seconds leaves those seconds unexamined. For pager-grade alerting prefer 2, which absorbs a late run, and reach for a larger window when the point is a rate rather than a single entry.

Detection latency is up to 1 minute (worst case: the entry arrives just after a check) plus the delivery time of the email or webhook.

Creating an Alert

Create an alert on a log project using the Alerts API:

curl -X POST https://app.krafter.dev/api/v1/logs/projects/a1b2c3d4-5e6f-7a8b-9c0d-1e2f3a4b5c6d/alerts \
  -H "Authorization: Bearer kr_live_abc123def456" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "High Error Rate",
    "level": "error",
    "threshold": 10,
    "window_minutes": 5,
    "channels": ["email", "webhook"],
    "notify_email": "ops@example.com",
    "notify_webhook": "https://example.com/webhooks/alerts"
  }'

// 201 Created
{
  "data": {
    "id": "f1e2d3c4-b5a6-9788-7c6d-5e4f3a2b1c0d",
    "project_id": "a1b2c3d4-5e6f-7a8b-9c0d-1e2f3a4b5c6d",
    "name": "High Error Rate",
    "level": "error",
    "threshold": 10,
    "window_minutes": 5,
    "channels": ["email", "webhook"],
    "notify_email": "ops@example.com",
    "notify_webhook": "https://example.com/webhooks/alerts",
    "enabled": true,
    "last_triggered_at": null,
    "inserted_at": "2025-01-15T10:00:00Z",
    "updated_at": "2025-01-15T10:00:00Z"
  }
}

The alert is enabled by default and starts monitoring immediately.

Alert Channels

Alerts support two notification channels:

Email

Set notify_email to the address that should receive alert notifications. The email includes the alert name, project, log level, and the threshold that was exceeded.

Webhook

Set notify_webhook to an HTTPS endpoint that should receive a POST request when the alert triggers. The full payload schema, headers, and current security caveats are documented under Webhook Delivery below.

You can use one or both channels by including them in the channels array. Omit the array and Krafter fills it in from the notify_* fields you set, so an alert with only notify_email needs no channels at all:

{
  "channels": ["email"]
}

{
  "channels": ["webhook"]
}

{
  "channels": ["email", "webhook"]
}

Webhook Delivery

When an alert fires with webhook in channels, Krafter POSTs a JSON payload to notify_webhook.

Payload schema

{
  "alert_id":       "f1e2d3c4-b5a6-9788-7c6d-5e4f3a2b1c0d",
  "alert_name":     "High Error Rate",
  "level":          "error",
  "match_count":    42,
  "window_minutes": 5,
  "triggered_at":   "2026-04-26T12:00:00Z"
}

Field	Type	Description
alert_id	string	UUID of the alert that fired.
alert_name	string	Human-readable alert name (as configured on the alert).
level	string	The log level the alert is configured to monitor (debug, info, warn, error, fatal).
match_count	integer	Actual count of matching log entries observed in the evaluation window. Always >= threshold.
window_minutes	integer	The evaluation window the alert is configured with.
triggered_at	string	ISO 8601 UTC timestamp of when the alert evaluation fired.

The payload does not include the matching log entries themselves — use the Search & Export API with the same level and a time range around triggered_at to fetch them.

Headers

Header	Value
Content-Type	application/json
x-krafter-event	alert.triggered
x-krafter-webhook-id	UUID — fresh per delivery attempt; use it to deduplicate retries.
x-krafter-signature	t=<unix>,v1=<hex> — HMAC-SHA256 of <t>.<body>; see Verifying signatures, keyed with the alert's notify_webhook_secret.

Verifying the signature

When you create or update an alert with a notify_webhook, Krafter generates a 32-byte random secret (returned as a 64-char lowercase hex string in the notify_webhook_secret field of the create/rotate response).

x-krafter-signature carries t=<unix>,v1=<hex>: the HMAC-SHA256 covers <t>.<raw body>, keyed with that secret. Split the header, reject anything whose t is more than 300 seconds old, then compare HMAC-SHA256(secret, "<t>." + rawBody) to v1 in constant time. Node.js, Elixir and Python implementations are on Verifying signatures.

The signature is omitted only when the alert has no secret stored — typically alerts created before signing was rolled out. New alerts always have a secret.

Receiver requirements

• Respond 2xx within 10 seconds (Krafter's HTTP read timeout).
• Be idempotent — Oban retries up to 3 attempts on any non-2xx response or network error. Use x-krafter-webhook-id to deduplicate.

Auto-disable on repeated failure

After 10 consecutive delivery failures (non-2xx response or network error), Krafter sets the alert's notify_webhook_active field to false and stops attempting deliveries on that webhook. The alert itself remains enabled — emails (if configured) keep firing — only the webhook channel is suspended.

Re-enable the webhook by issuing a PUT /api/v1/logs/alerts/:id with {"notify_webhook_active": true}, or by rotating the URL via the same endpoint. The failure counter resets to 0 on the next successful delivery.

Rotating the webhook secret

Rotate the HMAC secret periodically (or immediately if it leaks). Until a dedicated rotation endpoint ships, update the alert via PUT /api/v1/logs/alerts/:id with a new notify_webhook_secret value, or delete and recreate the alert. Krafter emits the new secret only in the response of the create/update call — store it on receipt.

Managing Alerts

Disable an Alert

Temporarily stop an alert without deleting it by setting enabled to false:

curl -X PUT https://app.krafter.dev/api/v1/logs/alerts/f1e2d3c4-b5a6-9788-7c6d-5e4f3a2b1c0d \
  -H "Authorization: Bearer kr_live_abc123def456" \
  -H "Content-Type: application/json" \
  -d '{"enabled": false}'

Update Thresholds

Adjust the threshold or time window as you tune your alerting:

curl -X PUT https://app.krafter.dev/api/v1/logs/alerts/f1e2d3c4-b5a6-9788-7c6d-5e4f3a2b1c0d \
  -H "Authorization: Bearer kr_live_abc123def456" \
  -H "Content-Type: application/json" \
  -d '{
    "threshold": 25,
    "window_minutes": 10
  }'

Delete an Alert

Permanently remove an alert rule:

curl -X DELETE https://app.krafter.dev/api/v1/logs/alerts/f1e2d3c4-b5a6-9788-7c6d-5e4f3a2b1c0d \
  -H "Authorization: Bearer kr_live_abc123def456"

Example: Monitor Fatal Errors

Alert on any fatal-level log entry, over a 2-minute window — one minute of coverage plus a minute of slack for a delayed run (see the window note above):

curl -X POST https://app.krafter.dev/api/v1/logs/projects/a1b2c3d4-5e6f-7a8b-9c0d-1e2f3a4b5c6d/alerts \
  -H "Authorization: Bearer kr_live_abc123def456" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Fatal Error Alert",
    "level": "fatal",
    "threshold": 1,
    "window_minutes": 2,
    "channels": ["email", "webhook"],
    "notify_email": "oncall@example.com",
    "notify_webhook": "https://example.com/webhooks/pagerduty"
  }'

Next Steps

• Alerts API -- Full API reference for alert CRUD operations
• Search & Export API -- Investigate the logs that triggered an alert
• Ingestion Guide -- Set appropriate log levels so alerts fire correctly