Alerts Guide

Set up alerts to get notified when error rates spike in your logs. Krafter monitors your log entries and triggers notifications via email or webhook when a threshold is exceeded within a time window.

How Alerts Work

An alert defines a rule: "Notify me when N or more logs at a given level occur within M minutes."

For example, an alert configured with level: "error", threshold: 10, and window_minutes: 5 will trigger when 10 or more error-level log entries are recorded within any 5-minute window.

How Alerts Evaluate

1. Schedule. Krafter checks every enabled alert every 5 minutes in the background.
2. Window. For each alert, Krafter counts log entries at the configured level over the past window_minutes minutes (relative to the moment of evaluation — not a fixed wall-clock window).
3. Threshold. If the count is at or above threshold (i.e. count >= threshold), the alert fires and notifications are dispatched.
4. Deduplication. After firing, Krafter records last_triggered_at and skips re-firing until at least window_minutes minutes have passed since last_triggered_at. So with window_minutes: 5, an active alert can fire at most once every 5 minutes — even if the threshold is crossed continuously.

Latency budget

Detection latency is up to 5 minutes (worst case: an event happens just after a check, the next check is 5 minutes later) plus the delivery time of the email or webhook. For pager-grade alerting on critical errors, set threshold: 1 and window_minutes: 1 and budget for roughly 1 minute end-to-end.

Creating an Alert

Create an alert on a log project using the Alerts API:

curl -X POST https://app.krafter.dev/api/v1/logs/projects/a1b2c3d4-5e6f-7a8b-9c0d-1e2f3a4b5c6d/alerts \
  -H "Authorization: Bearer kr_live_abc123def456" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "High Error Rate",
    "level": "error",
    "threshold": 10,
    "window_minutes": 5,
    "channels": ["email", "webhook"],
    "notify_email": "ops@example.com",
    "notify_webhook": "https://example.com/webhooks/alerts"
  }'

// 201 Created
{
  "data": {
    "id": "f1e2d3c4-b5a6-9788-7c6d-5e4f3a2b1c0d",
    "project_id": "a1b2c3d4-5e6f-7a8b-9c0d-1e2f3a4b5c6d",
    "name": "High Error Rate",
    "level": "error",
    "threshold": 10,
    "window_minutes": 5,
    "channels": ["email", "webhook"],
    "notify_email": "ops@example.com",
    "notify_webhook": "https://example.com/webhooks/alerts",
    "enabled": true,
    "last_triggered_at": null,
    "inserted_at": "2025-01-15T10:00:00Z",
    "updated_at": "2025-01-15T10:00:00Z"
  }
}

The alert is enabled by default and starts monitoring immediately.

Alert Channels

Alerts support two notification channels:

Email

Set notify_email to the address that should receive alert notifications. The email includes the alert name, project, log level, and the threshold that was exceeded.

Webhook

Set notify_webhook to an HTTPS endpoint that should receive a POST request when the alert triggers. The full payload schema, headers, and current security caveats are documented under Webhook Delivery below.

You can use one or both channels by including them in the channels array:

{
  "channels": ["email"]
}

{
  "channels": ["webhook"]
}

{
  "channels": ["email", "webhook"]
}

Webhook Delivery

When an alert fires with webhook in channels, Krafter POSTs a JSON payload to notify_webhook.

Payload schema

{
  "alert_id":       "f1e2d3c4-b5a6-9788-7c6d-5e4f3a2b1c0d",
  "alert_name":     "High Error Rate",
  "level":          "error",
  "match_count":    42,
  "window_minutes": 5,
  "triggered_at":   "2026-04-26T12:00:00Z"
}

Field	Type	Description
alert_id	string	UUID of the alert that fired.
alert_name	string	Human-readable alert name (as configured on the alert).
level	string	The log level the alert is configured to monitor (debug, info, warn, error, fatal).
match_count	integer	Actual count of matching log entries observed in the evaluation window. Always >= threshold.
window_minutes	integer	The evaluation window the alert is configured with.
triggered_at	string	ISO 8601 UTC timestamp of when the alert evaluation fired.

The payload does not include the matching log entries themselves — use the Search & Export API with the same level and a time range around triggered_at to fetch them.

Headers

Header	Value
Content-Type	application/json
x-krafter-event	alert.triggered
x-krafter-webhook-id	UUID — fresh per delivery attempt; use it to deduplicate retries.
x-krafter-signature	sha256=<hex> — HMAC-SHA256 of the raw request body using the alert's notify_webhook_secret.

Verifying the signature

When you create or update an alert with a notify_webhook, Krafter generates a 32-byte random secret (returned as a 64-char lowercase hex string in the notify_webhook_secret field of the create/rotate response). Recompute the HMAC on your side and compare it constant-time:

const crypto = require("crypto");

function verify(req, secret) {
  const raw = req.rawBody; // the exact bytes of the request body, before JSON parsing
  const expected = "sha256=" + crypto.createHmac("sha256", secret).update(raw).digest("hex");
  const got = req.headers["x-krafter-signature"];
  return crypto.timingSafeEqual(Buffer.from(expected), Buffer.from(got));
}

The signature is omitted only when the alert has no secret stored — typically alerts created before signing was rolled out. New alerts always have a secret.

Receiver requirements

• Respond 2xx within 10 seconds (Krafter's HTTP read timeout).
• Be idempotent — Oban retries up to 3 attempts on any non-2xx response or network error. Use x-krafter-webhook-id to deduplicate.

Auto-disable on repeated failure

After 10 consecutive delivery failures (non-2xx response or network error), Krafter sets the alert's notify_webhook_active field to false and stops attempting deliveries on that webhook. The alert itself remains enabled — emails (if configured) keep firing — only the webhook channel is suspended.

Re-enable the webhook by issuing a PUT /api/v1/logs/alerts/:id with {"notify_webhook_active": true}, or by rotating the URL via the same endpoint. The failure counter resets to 0 on the next successful delivery.

Rotating the webhook secret

Rotate the HMAC secret periodically (or immediately if it leaks). Until a dedicated rotation endpoint ships, update the alert via PUT /api/v1/logs/alerts/:id with a new notify_webhook_secret value, or delete and recreate the alert. Krafter emits the new secret only in the response of the create/update call — store it on receipt.

Managing Alerts

Disable an Alert

Temporarily stop an alert without deleting it by setting enabled to false:

curl -X PUT https://app.krafter.dev/api/v1/logs/alerts/f1e2d3c4-b5a6-9788-7c6d-5e4f3a2b1c0d \
  -H "Authorization: Bearer kr_live_abc123def456" \
  -H "Content-Type: application/json" \
  -d '{"enabled": false}'

Update Thresholds

Adjust the threshold or time window as you tune your alerting:

curl -X PUT https://app.krafter.dev/api/v1/logs/alerts/f1e2d3c4-b5a6-9788-7c6d-5e4f3a2b1c0d \
  -H "Authorization: Bearer kr_live_abc123def456" \
  -H "Content-Type: application/json" \
  -d '{
    "threshold": 25,
    "window_minutes": 10
  }'

Delete an Alert

Permanently remove an alert rule:

curl -X DELETE https://app.krafter.dev/api/v1/logs/alerts/f1e2d3c4-b5a6-9788-7c6d-5e4f3a2b1c0d \
  -H "Authorization: Bearer kr_live_abc123def456"

Example: Monitor Fatal Errors

Alert immediately on any fatal-level log entry (threshold of 1 within 1 minute):

curl -X POST https://app.krafter.dev/api/v1/logs/projects/a1b2c3d4-5e6f-7a8b-9c0d-1e2f3a4b5c6d/alerts \
  -H "Authorization: Bearer kr_live_abc123def456" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Fatal Error Alert",
    "level": "fatal",
    "threshold": 1,
    "window_minutes": 1,
    "channels": ["email", "webhook"],
    "notify_email": "oncall@example.com",
    "notify_webhook": "https://example.com/webhooks/pagerduty"
  }'

Next Steps

• Alerts API -- Full API reference for alert CRUD operations
• Search & Export API -- Investigate the logs that triggered an alert
• Ingestion Guide -- Set appropriate log levels so alerts fire correctly