> ## Documentation Index > Fetch the complete documentation index at: https://developers.scrunch.com/llms.txt > Use this file to discover all available pages before exploring further. # Agent Pages Publish: Add Pages on Demand > Make pages Agent Pages now, including pages your sitemap doesn't list, without waiting for scheduled discovery. ## Overview Publish makes the pages you name Agent Pages now. Each path is fetched from your live site, optimized and served to AI agents, usually within minutes. It works for pages your sitemap doesn't list too. Use it from your CMS when a page goes live, or to bring back a page you took down with [Agent Pages Unpublish](https://developers.scrunch.com/api-reference/agent-pages/unpublish-overview.md). Publish acts on the Agent Page only. If the path also has curated (Adaptive) content, agents keep receiving that version, and the response doesn't report it. See [Which API do I need?](https://developers.scrunch.com/api-reference/agent-pages/force-refresh-overview.md#which-api-do-i-need). *** ## What happens to a path 1. On a page-by-page setup, the page is marked **Serve on Agent Pages**, exactly as if you selected it in the Site Optimization Pages table, and it counts against your Agent Pages budget. On a whole-site (legacy) setup, nothing is selected, the page isn't added to any list, and no budget applies: the site keeps serving every page, and the path joins the scheduled checks. 2. If [Agent Pages Unpublish](https://developers.scrunch.com/api-reference/agent-pages/unpublish-overview.md) put the path on the Ignore List, that entry is removed. An Ignore List entry or path rule you added in the dashboard is never overridden: the path returns `excluded`. 3. A background job fetches the page from your site, optimizes it and publishes it to the edge. The page is served once that job finishes, usually within minutes. On a page-by-page setup, the site serves only its Agent Pages set: the pages marked in the dashboard or published through this API. With Site Optimization, a site with an empty set serves no Agent Pages. Without it, a site serves every page until its first page is marked or published, and from then on only the set: the first publish stops serving every page outside the set, and Unpublish refuses to remove the last one (`conflict`). ### `queued` is not yet served `queued` means the job was sent, not that the page is live. If the send itself fails, the page is still picked up by the hourly scheduled check, usually within an hour. If your site doesn't serve the page (for example it answers `404`), the job fails. The page stays selected and counts against your Agent Pages budget, but it isn't served until a scheduled check finds it live (about every 30 days) or you send it again. The response can't tell you, because it returns before the fetch, and there is no status endpoint. Publish a page only once it is live on your site. If your site redirects the path to another path, the page isn't published either, and the redirect is recorded as a scheduled check records it; a redirect that only changes the host within the same domain (adding or dropping `www.`, or another subdomain) and keeps the path still publishes. To retry a failed publish, send the path again after its 5-minute cooldown. A `queued` path can also carry `warnings` about its history: | Warning | Meaning | | - | - | | `Retired (), last checked : it serves only once the origin answers 200.` | The page was retired earlier. It serves again only once your site answers `200` for it. `` is `http_gone` (your site answered `404` or `410`), `sitemap_absent` (your sitemap dropped it), `non_html` (it isn't an HTML page) or `admin_retired` (retired by Scrunch). `` is the page's last scheduled check, not the day it was retired. Either part is left out when unknown, down to `Retired: it serves only once the origin answers 200.` | | `Recent checks saw this path redirect to : it serves once a check sees the page answer for itself.` | Recent checks saw the path redirect. It serves once a check sees the page answer without redirecting. `` reads `another URL` when no target was recorded. | ### Pages your sitemap doesn't list Each published path reports `in_sitemap`. A `queued` page your sitemap doesn't list is published as a **manual** Agent Page and carries the warning `Not in the site's sitemap…`. An `already_published` page never carries it: there, `in_sitemap: false` is informational. The sitemap check never removes a manual Agent Page. It stays served until your site answers `404` or redirects it at a scheduled check, or until you call [Agent Pages Unpublish](https://developers.scrunch.com/api-reference/agent-pages/unpublish-overview.md). A `queued` page your sitemap lists is managed by the sitemap, as before: if the sitemap later drops it, the sitemap check can retire it. Publish doesn't change how an `already_published` page is managed. ### Paths your CDN doesn't route to the edge Agents only receive Agent Pages for paths your CDN sends to the Scrunch edge worker. If your CDN rules cover part of the site only (for example `/us-en/*`), a path outside them is published and stored, but never served. The API can't detect this, so check your CDN routing before publishing outside it. *** ## Endpoint | Method | Path | Purpose | | - | - | - | | POST | `/v1/{brand_id}/sites/{site_id}/agent-pages/publish` | Publish a list of paths on one site. | The endpoint requires a bearer token with the `optimize` scope. See [Authentication](https://developers.scrunch.com/getting-started/authentication.md). Use the site ID that [List Sites](https://developers.scrunch.com/api-reference/agent-pages/list-sites.md) reports with `agent_pages_active: true`; see [Find your site ID](https://developers.scrunch.com/api-reference/agent-pages/force-refresh-overview.md#find-your-site-id). *** ## Request | Field | Type | Required | Description | | - | - | - | - | | `paths` | `string[]` | Yes | Site-relative paths to publish, for example `/pricing`. **1 to 100 paths** per call. | Paths follow the same rules as Force Refresh: send the public URL path, site-relative, up to 2,048 characters. Paths are lowercased, the trailing slash and query string are dropped, and duplicates are merged. `/` is the homepage. See [Path rules](https://developers.scrunch.com/api-reference/agent-pages/force-refresh-overview.md#path-rules). One difference: `.model.json` and other non-HTML files return `invalid_path` here, not `not_found`. The page is fetched with the spelling you sent first, so a case-sensitive site gets the URL it expects. The query string is dropped from the fetched URL too, so a tracking or session parameter is never fetched. *** ## Response The endpoint returns `200 OK` with one result per requested path, in request order, and the brand's remaining publish quota. | Field | Type | Description | | - | - | - | | `results[].path` | `string` | The normalized path this outcome applies to. | | `results[].outcome` | `string` | What happened to this path. See the table below. | | `results[].retry_after` | `string \| null` | For `cooldown`, when this path can be published again (ISO 8601, UTC). | | `results[].in_sitemap` | `boolean \| null` | For `queued` and `already_published` paths, whether your sitemap lists the page. `null` for other outcomes. | | `results[].warnings` | `string[]` | Why the page may not be served soon, or may be removed later. Empty when there is nothing to report. See [`queued` is not yet served](#queued-is-not-yet-served) and [Pages your sitemap doesn't list](#pages-your-sitemap-doesnt-list). | | `quota.hourly` / `quota.daily` | `object` | The brand's two publish quota windows, each with `limit`, `remaining`, and `resets_at`. | ### Outcome values | Outcome | Meaning | What to do | | - | - | - | | `queued` | The page is an Agent Page now and its fetch was sent. It is served once the fetch and optimization finish. If the send fails, the hourly scheduled check picks the page up, usually within an hour. | Nothing. If it isn't served within the hour, check that the page is live and send it again. | | `already_published` | The page is already being served. Nothing was fetched. On a page-by-page setup, a served page that wasn't marked yet is added to the set, if your budget allows, and counts against your Agent Pages budget. | To pick up a change on your site, use [Force Refresh](https://developers.scrunch.com/api-reference/agent-pages/force-refresh-overview.md). | | `excluded` | An Ignore List entry or path rule from the dashboard blocks this path. On a whole-site (legacy) setup with Priority List entries, a path outside them is also `excluded`. The API never overrides the dashboard. | Remove the entry under **Configure → Universal Optimizations**, then retry. | | `over_budget` | Your organization's Agent Pages budget is full (page-by-page setups only). In a batch, the first paths that fit are queued. Nothing was written for this path. | Stop serving another page, or ask your account team for more, then retry. | | `invalid_path` | Not a web page (a file such as `.pdf`, `.json` or `.model.json`), or a path the page list can't store. | Send the public URL path of an HTML page. | | `cooldown` | This path was published in the last 5 minutes. | Retry after `retry_after`. | | `rate_limited` | The brand's hourly or daily quota ran out partway through this batch. | Retry after `quota.*.resets_at`. | Only `queued` paths count toward the quota. *** ## Limits | Limit | Value | When exceeded | | - | - | - | | Paths per request | 100 | `422`; nothing is published. | | Per path | 1 publish every 5 minutes | That path returns `cooldown`. | | Per brand, hourly | 500 paths per rolling hour | Extra paths return `rate_limited`; `429` if nothing could be published. | | Per brand, daily | 5,000 paths per rolling 24 hours | Extra paths return `rate_limited`; `429` if nothing could be published. | | Agent Pages budget | Your organization's plan (page-by-page setups only) | Extra paths return `over_budget`. | The quotas are shared by all of a brand's sites and are separate from the Force Refresh and Unpublish quotas. *** ## Example: publish new pages ```bash theme={null} curl -X POST \ "https://api.scrunchai.com/v1/1234/sites/01JEXAMPLE00000000000000/agent-pages/publish" \ -H "Authorization: Bearer $SCRUNCH_API_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "paths": [ "/pricing", "/launch/fall-event", "/about", "/brochure.pdf" ] }' ``` **Response:** ```json theme={null} { "results": [ { "path": "/pricing", "outcome": "queued", "retry_after": null, "in_sitemap": true, "warnings": [] }, { "path": "/launch/fall-event", "outcome": "queued", "retry_after": null, "in_sitemap": false, "warnings": [ "Retired (http_gone), last checked 2026-10-20: it serves only once the origin answers 200.", "Not in the site's sitemap. Published as a manual Agent Page: it is not removed by the sitemap check, only by an origin 404 or redirect, or the unpublish API." ] }, { "path": "/about", "outcome": "already_published", "retry_after": null, "in_sitemap": true, "warnings": [] }, { "path": "/brochure.pdf", "outcome": "invalid_path", "retry_after": null, "in_sitemap": null, "warnings": [] } ], "quota": { "hourly": { "limit": 500, "remaining": 498, "resets_at": "2026-10-28T15:02:41Z" }, "daily": { "limit": 5000, "remaining": 4987, "resets_at": "2026-10-29T09:15:03Z" } } } ``` *** ## Errors | Status | When | | - | - | | `401` | The API key is missing or invalid. | | `403` | The API key lacks the `optimize` scope or access to the brand. | | `404` | No site with the supplied ULID exists for the brand, or Agent Pages aren't enabled for the brand. | | `409` | Agent Pages aren't active on this site. Check that `site_id` is the site [List Sites](https://developers.scrunch.com/api-reference/agent-pages/list-sites.md) reports with `agent_pages_active: true`. Nothing is published and no quota is used. | | `422` | The request body failed validation (empty `paths`, more than 100 entries, an empty path, a path over 2,048 characters, or a full URL). A malformed or duplicated `Content-Type` header also returns `422`; send exactly `Content-Type: application/json`. | | `429` | The brand's hourly or daily publish quota is used up and nothing could be published, or another publish request for the same brand is still being processed. The `Retry-After` header gives the seconds to wait (`1` for a request in progress). | A path that is excluded, over budget, invalid or in cooldown doesn't cause an error: the request still returns `200` with those outcomes. *** ## Best practices * **Publish once the page is live.** The fetch runs right after the response; a page your site doesn't serve yet fails silently. Read `warnings` on `queued` paths: a retired or redirecting page waits for your site to answer for it. * **Retry safely.** Re-sending a served path returns `already_published` and fetches nothing. On a page-by-page setup it is also added to the set, if it wasn't already and your budget allows. Re-sending a `queued` path within 5 minutes returns `cooldown`, or `already_published` once it is served; after that it fetches the page again, which is how you retry a failed publish. * **Send one request at a time per brand.** A request sent while another publish for the brand is in progress gets `429` with `Retry-After: 1`. Batch paths into one request (up to 100). * **Publish after Unpublish.** A path taken down with [Agent Pages Unpublish](https://developers.scrunch.com/api-reference/agent-pages/unpublish-overview.md) can be published again with this API. A path you put on the Ignore List in the dashboard can't: remove it there first. * **Check curated content.** If the dashboard shows an edited or deployed version at the path, agents keep receiving that version instead of the published Agent Page. ## Related topics - [Agent Pages Force Refresh: Update Pages on Demand](https://developers.scrunch.com/api-reference/agent-pages/force-refresh-overview.md) - [Agent Pages Unpublish: Stop Serving Pages on Demand](https://developers.scrunch.com/api-reference/agent-pages/unpublish-overview.md) - [Publish](https://developers.scrunch.com/api-reference/agent-pages/publish.md) This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.