API abuse & scraping

APIs are different from form-submit surfaces. You don’t want a sealed handoff per request - you want one Foil session that covers a user’s browsing session, reused across many API calls. And you want to distinguish LLM scrapers (ai-agent) from legitimate crawlers (verified-bot, crawler), because the right answer for those is different.

The threat

Two problems get lumped together as “API abuse,” and they want different responses:

LLM scraping. Cloud agents like browser-use and computer-use, plus self-hosted equivalents, scrape the web on behalf of model training, retrieval pipelines, or end-user automation. They look like browsers because they are browsers, just driven by an agent loop. Foil attributes these as ai-agent.
Scripted scraping. Classic headless Chrome or HTTP-level clients iterating your listing endpoints, pricing APIs, or search results. automation attribution.

Against both, you want something stronger than a per-user-account rate limit - attackers rotate accounts - and stronger than a per-IP rate limit - attackers rotate residential proxies. The visitor fingerprint is the axis that holds. At the same time, you almost certainly do want some non-human traffic:

Search crawlers (Googlebot, Bingbot) - you want them indexing your public content.
Web-Bot-Auth–authenticated agents - a growing set of AI products sign their requests with HTTP message signatures declaring themselves. Foil surfaces this as the verified-bot category and carries the domain/key identity through to your backend.
Your own internal tooling - health checks, monitoring, analytics pipelines.

The integration on this page separates “block” from “allow” on the attribution category, not just the verdict.

The flow

Start Foil once at app boot

For authenticated SPAs, start the client when the shell mounts. For public APIs accessed directly (no browser), see the bottom of this page.

Acquire a session once and reuse it

Call getSession() at the start of an API-consuming flow - not on every request. Pass sessionId as a header on subsequent XHRs.

Verify on the first protected request

Either verify a fresh sealed token, or look up the stored session via GET /v1/sessions/:sessionId and cache the verdict.

Re-verify on high-value actions or periodically

For mutations or sensitive reads, request a fresh handoff. For long-lived sessions, refresh the cached verdict every few minutes.

Split policy by attribution category

Allow verified-bot and crawler; block automation and ai-agent; treat human normally.

Client integration

Request a sealed handoff at the start of a session, stash the session ID, and send it as a header on every API call.

<script type="module">
  const foilPromise = import("https://cdn.usefoil.com/t.js").then(
    (Foil) =>
      Foil.start({
        publishableKey: "pk_live_your_publishable_key",
      }),
  );

  let sessionHandoffPromise = null;

  async function getFoilHandoff() {
    if (!sessionHandoffPromise) {
      sessionHandoffPromise = foilPromise.then((t) => t.getSession());
    }
    return sessionHandoffPromise;
  }

  async function apiFetch(path, init = {}) {
    const { sessionId, sealedToken } = await getFoilHandoff();
    const headers = new Headers(init.headers);
    headers.set("X-Foil-Session", sessionId);
    headers.set("X-Foil-Token", sealedToken);
    return fetch(path, { ...init, headers });
  }

  // Refresh the handoff for high-value mutations
  async function refreshHandoff() {
    const foil = await foilPromise;
    sessionHandoffPromise = Promise.resolve(foil.getSession());
    return sessionHandoffPromise;
  }
</script>

Server verification: two patterns

For long-lived API sessions you have a choice. Either verify the sealed token on every request (cheap - it’s a local crypto operation) and cache the decoded result, or call GET /v1/sessions/:sessionId once and cache the verdict for a few minutes.

Pattern A: verify the sealed token per request

const { safeVerifyFoilToken } = require("@abxy/foil-server");

async function foilGuard(req, res, next) {
  const sealedToken = req.get("X-Foil-Token");
  if (!sealedToken) {
    return res.status(401).json({ error: "Missing Foil token" });
  }

  const result = safeVerifyFoilToken(sealedToken, process.env.FOIL_SECRET_KEY);
  if (!result.ok) {
    return res.status(401).json({ error: "Invalid Foil token" });
  }

  const { decision, attribution } = result.data;
  const category = attribution?.bot?.facets?.category?.value;

  // Allow verified bots and search crawlers on read endpoints
  if (req.method === "GET" && (category === "verified-bot" || category === "crawler")) {
    req.foilVerdict = { verdict: "allowed_bot", category };
    return next();
  }

  // Block automation and ai-agent regardless of method
  if (decision.verdict === "bot") {
    return res.status(403).json({ error: "Blocked" });
  }

  req.foilVerdict = { verdict: decision.verdict, category: category ?? null };
  next();
}

app.use("/api", foilGuard);

from foil_server import safe_verify_foil_token
import os

def foil_guard(request):
    sealed = request.headers.get("X-Foil-Token")
    if not sealed:
        return {"error": "Missing Foil token"}, 401

    result = safe_verify_foil_token(sealed, os.environ["FOIL_SECRET_KEY"])
    if not result.ok:
        return {"error": "Invalid Foil token"}, 401

    decision = result.data.decision
    attribution = result.data.attribution or {}
    category = (
        attribution.get("bot", {}).get("facets", {}).get("category", {}).get("value")
    )

    if request.method == "GET" and category in ("verified-bot", "crawler"):
        request.state.foil = {"verdict": "allowed_bot", "category": category}
        return None

    if decision.verdict == "bot":
        return {"error": "Blocked"}, 403

    request.state.foil = {"verdict": decision.verdict, "category": category}
    return None

import foil "github.com/abxy-labs/foil-server-go"

func FoilGuard(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        sealed := r.Header.Get("X-Foil-Token")
        if sealed == "" {
            http.Error(w, "Missing Foil token", 401)
            return
        }

        tr := foil.SafeVerifyFoilToken(sealed, os.Getenv("FOIL_SECRET_KEY"))
        if !tr.OK {
            http.Error(w, "Invalid Foil token", 401)
            return
        }

        // Read category from attribution.bot.facets.category.value
        category := botCategory(tr.Data)

        if r.Method == http.MethodGet && (category == "verified-bot" || category == "crawler") {
            next.ServeHTTP(w, r)
            return
        }

        if tr.Data.Decision.Verdict == "bot" {
            http.Error(w, "Blocked", 403)
            return
        }

        next.ServeHTTP(w, r)
    })
}

require "foil/server"

before "/api/*" do
  sealed = request.env["HTTP_X_FOIL_TOKEN"]
  halt 401, { error: "Missing Foil token" }.to_json unless sealed

  tr = Foil::Server::SealedToken.safe_verify_foil_token(sealed)
  halt 401, { error: "Invalid Foil token" }.to_json unless tr[:ok]

  decision  = tr[:data][:decision]
  category  = tr[:data].dig(:attribution, :bot, :facets, :category, :value)

  next if request.get? && %w[verified-bot crawler].include?(category)

  halt 403, { error: "Blocked" }.to_json if decision[:verdict] == "bot"
end

use Foil\Server\SealedToken;

function foilGuard(): void {
    $sealed = $_SERVER["HTTP_X_FOIL_TOKEN"] ?? null;
    if (!$sealed) { http_response_code(401); exit; }

    $tr = SealedToken::safeVerify($sealed, getenv("FOIL_SECRET_KEY"));
    if (!$tr->ok) { http_response_code(401); exit; }

    $category = $tr->data->attribution["bot"]["facets"]["category"]["value"] ?? null;

    if ($_SERVER["REQUEST_METHOD"] === "GET" &&
        in_array($category, ["verified-bot", "crawler"])) {
        return;
    }

    if ($tr->data->decision["verdict"] === "bot") {
        http_response_code(403);
        exit;
    }
}

# Session readback for cached-verdict pattern (see below)
curl https://api.usefoil.com/v1/sessions/sid_... \
  -H "Authorization: Bearer sk_live_..."

Pattern B: cache the verdict

For very hot endpoints, verify once per session and cache the result.

Node.js

const { Foil } = require("@abxy/foil-server");
const client = new Foil({ secretKey: process.env.FOIL_SECRET_KEY });
const verdictCache = new Map(); // sessionId → { verdict, category, expiresAt }

async function getCachedVerdict(sessionId) {
  const cached = verdictCache.get(sessionId);
  if (cached && cached.expiresAt > Date.now()) return cached;

  const session = await client.sessions.get(sessionId);
  const category = session.automation?.category ?? null;
  const entry = {
    verdict: session.decision.automation_status, // "automated" | "human" | "uncertain"
    category,
    expiresAt: Date.now() + 60_000, // 1 minute
  };
  verdictCache.set(sessionId, entry);
  return entry;
}

The session readback returns decision.automation_status ("automated" | "human" | "uncertain") rather than the sealed-token verdict field; see Server verification for the full shape.

When to re-verify

Session reuse is great for throughput, but a long-lived session becomes a stale verdict. Three refresh triggers worth coding:

High-value mutations - a user changing their email, deleting data, exporting an archive. Always ask for a fresh sealed handoff from the client.
Periodic refresh - every 5–15 minutes for long sessions, re-fetch GET /v1/sessions/:sessionId. Foil’s server-side behavioral scoring can move a verdict between snapshot and behavioral phases as evidence accumulates, and you want the latest.
Suspicious pattern observed - your own application logic (burst of identical queries, geographic jump) can trigger a client-side foil.getSession() that produces a new, freshly-attested handoff.

Splitting policy by attribution

For read APIs, the policy matrix that works on most sites:

Category	GET	POST / PUT / DELETE
`human`	Allow	Allow
`verified-bot`	Allow	Block (bots generally shouldn’t mutate)
`crawler`	Allow	Block
`automation`	Block or throttle	Block
`ai-agent`	Block or throttle	Block
`unknown` (with `bot` verdict)	Throttle	Block

“Allow” doesn’t mean unlimited. Keep a generous rate limit on verified bots - a badly-written crawler can still hurt you - but don’t return 403s. If you want LLM agents to read your content but not scrape it, set different caps for ai-agent: a low QPS limit that’s fine for an agent answering one user’s question and painful for a training-data crawler.

APIs called directly (no browser)

Some API consumers don’t run a browser at all - a mobile app, a server-to-server integration, a CLI tool. Foil’s browser SDK doesn’t apply there. Two options:

Require pre-issued API keys for non-browser traffic. Route traffic without a Foil session header down the API-key path, and apply Foil only to browser-origin requests.
Rely on network-level attribution. Even without the browser bundle, Foil’s HTTP edge sees JA4 TLS fingerprints, HTTP/2 SETTINGS, and Web-Bot-Auth signatures, and the sessions API can surface these for direct requests. See Detection categories for what’s available without the client SDK.

What’s next

User-generated content

The write-side counterpart: stop LLM posts at the composer.

Server verification

Reference for both sealed-token and session-readback paths.

Detection categories

What Foil detects, with and without the browser SDK.

Going to production

Rollout plan for API-wide enforcement.

Overview

Get started

Integrate Detection

Use cases

Learn

Gate: Agentic Signup

Integrate Gate

Resources

API abuse & scraping

The threat

The flow

Client integration

Server verification: two patterns

Pattern A: verify the sealed token per request

Pattern B: cache the verdict

When to re-verify

Splitting policy by attribution

APIs called directly (no browser)

What’s next

User-generated content

Server verification

Detection categories

Going to production

​The threat

​The flow

​Client integration

​Server verification: two patterns

​Pattern A: verify the sealed token per request

​Pattern B: cache the verdict

​When to re-verify

​Splitting policy by attribution

​APIs called directly (no browser)

​What’s next

User-generated content

Server verification

Detection categories

Going to production

The threat

The flow

Client integration

Server verification: two patterns

Pattern A: verify the sealed token per request

Pattern B: cache the verdict

When to re-verify

Splitting policy by attribution

APIs called directly (no browser)

What’s next