> ## Documentation Index
> Fetch the complete documentation index at: https://developers.scrunch.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Send Custom Agent Traffic Events

> POST AI bot and agent traffic events from any web host, CDN, or edge function to Scrunch using the custom web-traffic ingestion endpoint.

Send bot and AI agent traffic data from any platform to Scrunch AI. This endpoint accepts traffic events in JSON or NDJSON format and automatically classifies bots based on user agent strings.

## Endpoint

```
POST https://webhooks.scrunchai.com/v1/sites/{site_id}/platforms/custom/web-traffic
```

## Authentication

Authenticate using the `X-Api-Key` header with a JWT token provided in the Scrunch AI dashboard when creating a site with the "API" platform.

To generate the `X-Api-Key`, add a new website on the Agent Traffic page and select `API` as your platform.

```bash theme={null}
X-Api-Key: <JWT_TOKEN>
```

## Path parameters

| Parameter | Type   | Required | Description                                          |
| --------- | ------ | -------- | ---------------------------------------------------- |
| `site_id` | string | Yes      | The Site ID (ULID format) assigned in the dashboard. |

The `platform_id` is always `custom` for this endpoint.

## Request headers

| Header         | Value                                        | Required | Description                                                                                                          |
| -------------- | -------------------------------------------- | -------- | -------------------------------------------------------------------------------------------------------------------- |
| `X-Api-Key`    | `<JWT_TOKEN>`                                | Yes      | Authentication token from the dashboard.                                                                             |
| `Content-Type` | `application/json` or `application/x-ndjson` | Yes      | Use `application/json` for a single event, or `application/x-ndjson` for multiple events (one JSON object per line). |

## Request body

The body can be either a **single JSON object** or **NDJSON** (newline-delimited JSON, one object per line).

### Fields

| Field           | Type            | Required | Description                                                        |
| --------------- | --------------- | -------- | ------------------------------------------------------------------ |
| `domain`        | string          | Yes      | The domain of the site (e.g. `example.com`).                       |
| `user_agent`    | string          | Yes      | The full User-Agent string of the request.                         |
| `url`           | string          | Yes      | The full URL that was requested (e.g. `https://example.com/page`). |
| `path`          | string          | Yes      | The URL path (e.g. `/page`).                                       |
| `method`        | string          | Yes      | The HTTP method (e.g. `GET`, `POST`).                              |
| `status_code`   | integer         | Yes      | The HTTP response status code (e.g. `200`, `404`).                 |
| `timestamp`     | integer / float | Yes      | Unix epoch timestamp in **seconds** (e.g. `1700000000`).           |
| `response_time` | integer         | No       | Response time in milliseconds.                                     |
| `ip`            | string          | No       | The IP address of the client making the request.                   |

## Example: single event (JSON)

```bash theme={null}
curl -X POST "https://webhooks.scrunchai.com/v1/sites/{site_id}/platforms/custom/web-traffic" \
  -H "Content-Type: application/json" \
  -H "X-Api-Key: YOUR_API_KEY" \
  -d '{
    "domain": "example.com",
    "user_agent": "Mozilla/5.0 (compatible; GPTBot/1.0; +https://openai.com/gptbot)",
    "url": "https://example.com/blog/post",
    "path": "/blog/post",
    "method": "GET",
    "status_code": 200,
    "timestamp": 1700000000,
    "response_time": 120,
    "ip": "203.0.113.1"
  }'
```

## Example: batch events (NDJSON)

Send multiple events in a single request using newline-delimited JSON. Each line is a complete JSON object.

```bash theme={null}
curl -X POST "https://webhooks.scrunchai.com/v1/sites/{site_id}/platforms/custom/web-traffic" \
  -H "Content-Type: application/x-ndjson" \
  -H "X-Api-Key: YOUR_API_KEY" \
  -d '{"domain":"example.com","user_agent":"Mozilla/5.0 (compatible; GPTBot/1.0)","url":"https://example.com/page-1","path":"/page-1","method":"GET","status_code":200,"timestamp":1700000000}
{"domain":"example.com","user_agent":"Mozilla/5.0 (compatible; ClaudeBot/1.0)","url":"https://example.com/page-2","path":"/page-2","method":"GET","status_code":200,"timestamp":1700000060,"response_time":95,"ip":"198.51.100.42"}'
```

## Responses

| Status Code | Description                                                                 |
| ----------- | --------------------------------------------------------------------------- |
| 200         | Events accepted and queued for processing.                                  |
| 401         | Unauthorized — invalid or missing API key.                                  |
| 422         | Validation error — check the request body against the schema above.         |
| 429         | Rate limited — wait and retry. Respect the `Retry-After` header if present. |
| 500         | Internal server error.                                                      |

### Success response (200)

```json theme={null}
{
  "status": "ok"
}
```

## Notes

* **Batch size**: For NDJSON batches, keep each request under \~1 MB uncompressed.
* **Timestamp**: Must be a Unix epoch in seconds. Both integers and floats are accepted. Invalid timestamps will fall back to the current server time.
* **Bot detection**: The `user_agent` field is used to automatically identify and classify the bot/AI agent. Pass the original User-Agent string from the incoming request.
* **Activation**: After creating a site with the "API" platform in the dashboard, it starts in "pending" status. Once the first valid request is received, the site automatically transitions to "active" within 5 minutes.

## Best practices

* Use NDJSON for batch uploads to reduce request overhead
* Keep batch sizes under 1 MB for optimal performance
* Always pass the original User-Agent string for accurate bot classification
* Monitor response status codes and implement retry logic for 429 and 500 errors
* Use the `response_time` field to track performance metrics alongside bot traffic