Experiments

Run A/B tests by attaching an experiment to a string-typed flag. Define variants with traffic allocation, start and stop the experiment, track conversions, and read the results -- all through the API.

How Experiments Work

1. Create a string-typed flag whose default_value is the control variant value
2. Attach an experiment with two or more variants and percentage splits
3. Start the experiment -- the evaluate endpoint now returns variant values based on allocation
4. Track conversions from your app when users complete a goal
5. Stop the experiment and read the results

Variant assignment is consistent per user_id so the same user always sees the same variant.

Step 1: Create a String Flag for the Experiment

Experiments target a flag whose value is the variant key. Use flag_type: "string" and set default_value to the control variant -- this is the fallback returned when the experiment is not running.

curl -X POST https://app.krafter.dev/api/v1/flags \
  -H "Authorization: Bearer kr_live_abc123def456" \
  -H "Content-Type: application/json" \
  -d '{
    "project_id": "PROJECT_ID",
    "key": "checkout_flow",
    "name": "Checkout Flow",
    "flag_type": "string",
    "default_value": "control",
    "description": "Checkout flow A/B test"
  }'

{
  "data": {
    "id": "f1a2b3c4-5d6e-7f8a-9b0c-1d2e3f4a5b6c",
    "project_id": "PROJECT_ID",
    "key": "checkout_flow",
    "name": "Checkout Flow",
    "flag_type": "string",
    "default_value": "control",
    "enabled": false,
    "description": "Checkout flow A/B test",
    "created_at": "2025-01-15T14:00:00Z"
  }
}

Step 2: Create an Experiment

Attach an experiment to the flag with two or more variants. Percentages must add up to 100.

curl -X POST https://app.krafter.dev/api/v1/flags/f1a2b3c4-5d6e-7f8a-9b0c-1d2e3f4a5b6c/experiments \
  -H "Authorization: Bearer kr_live_abc123def456" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Checkout Flow Test",
    "environment": "production",
    "variants": [
      {"key": "control", "value": "old_checkout", "percentage": 50},
      {"key": "treatment", "value": "new_checkout", "percentage": 50}
    ]
  }'

{
  "data": {
    "id": "e1a2b3c4-5d6e-7f8a-9b0c-1d2e3f4a5b6c",
    "flag_id": "f1a2b3c4-5d6e-7f8a-9b0c-1d2e3f4a5b6c",
    "name": "Checkout Flow Test",
    "environment": "production",
    "status": "draft",
    "variants": [
      {"key": "control", "value": "old_checkout", "percentage": 50},
      {"key": "treatment", "value": "new_checkout", "percentage": 50}
    ],
    "conversions": 0,
    "created_at": "2025-01-15T14:05:00Z"
  }
}

The experiment starts in draft status. Flag evaluation still returns the default_value until you start it.

INFO

You can define more than two variants for multivariate tests. Just make sure the percentages add up to 100.

Step 3: Start the Experiment

When you are ready to begin collecting data, start the experiment:

curl -X POST https://app.krafter.dev/api/v1/flags/f1a2b3c4-5d6e-7f8a-9b0c-1d2e3f4a5b6c/experiments/e1a2b3c4-5d6e-7f8a-9b0c-1d2e3f4a5b6c/start \
  -H "Authorization: Bearer kr_live_abc123def456"

{
  "data": {
    "id": "e1a2b3c4-5d6e-7f8a-9b0c-1d2e3f4a5b6c",
    "status": "running"
  }
}

Now the evaluate endpoint returns variant values based on the user's allocation:

curl -X POST https://app.krafter.dev/api/v1/flags/evaluate \
  -H "Authorization: Bearer kr_live_abc123def456" \
  -H "Content-Type: application/json" \
  -d '{
    "project_id": "PROJECT_ID",
    "environment": "production",
    "context": {
      "user_id": "usr_123"
    }
  }'

{
  "flags": {
    "checkout_flow": {
      "value": "new_checkout",
      "rule_id": null
    }
  }
}

The same user_id always receives the same variant for the duration of the experiment.

Step 4: Track Conversions

When a user completes the goal you are measuring (e.g., completes a purchase), send a conversion event:

curl -X POST https://app.krafter.dev/api/v1/flags/track \
  -H "Authorization: Bearer kr_live_abc123def456" \
  -H "Content-Type: application/json" \
  -d '{
    "flag_id": "f1a2b3c4-5d6e-7f8a-9b0c-1d2e3f4a5b6c",
    "experiment_id": "e1a2b3c4-5d6e-7f8a-9b0c-1d2e3f4a5b6c",
    "user_id": "usr_123"
  }'

{
  "data": {
    "ok": true
  }
}

The system records which variant the user was assigned to and increments the conversion count.

WARNING

Call the track endpoint once per user per goal. Duplicate conversions for the same user are ignored.

Step 5: Read Results

Check the experiment results at any time while it is running (or after it stops):

curl https://app.krafter.dev/api/v1/flags/f1a2b3c4-5d6e-7f8a-9b0c-1d2e3f4a5b6c/experiments/e1a2b3c4-5d6e-7f8a-9b0c-1d2e3f4a5b6c \
  -H "Authorization: Bearer kr_live_abc123def456"

{
  "data": {
    "id": "e1a2b3c4-5d6e-7f8a-9b0c-1d2e3f4a5b6c",
    "name": "Checkout Flow Test",
    "environment": "production",
    "status": "running",
    "variants": [
      {"key": "control", "value": "old_checkout", "percentage": 50},
      {"key": "treatment", "value": "new_checkout", "percentage": 50}
    ],
    "conversions": 142,
    "created_at": "2025-01-15T14:05:00Z"
  }
}

Step 6: Stop the Experiment

When you have collected enough data, stop the experiment:

curl -X POST https://app.krafter.dev/api/v1/flags/f1a2b3c4-5d6e-7f8a-9b0c-1d2e3f4a5b6c/experiments/e1a2b3c4-5d6e-7f8a-9b0c-1d2e3f4a5b6c/stop \
  -H "Authorization: Bearer kr_live_abc123def456"

{
  "data": {
    "id": "e1a2b3c4-5d6e-7f8a-9b0c-1d2e3f4a5b6c",
    "status": "stopped"
  }
}

After stopping, the flag reverts to its default_value (or targeting rules, if any exist). You can then update the flag's default to the winning variant to ship the change permanently.

Experiment Lifecycle

Status	Evaluate behavior	Track behavior
draft	Returns flag's default_value	Rejected
running	Returns allocated variant	Accepted
stopped	Returns flag's default_value	Rejected

Next Steps

• Rules & Targeting -- Combine experiments with targeting rules
• Quickstart -- Create your first flag
• API Reference -- Full endpoint documentation