API abuse & scraping

APIs are different from form-submit surfaces. A single page can make many API calls, so you decide how often to ask Foil for a fresh handoff. And you want to distinguish LLM scrapers (ai-agent) from crawlers and signed agents that you allow, because the right answer for those is different.

The threat

Two problems get lumped together as “API abuse,” and they want different responses:

LLM scraping. Cloud-hosted browser agents and their self-hosted equivalents scrape the web on behalf of model training, retrieval pipelines, or end-user automation. They look like browsers because they are browsers, just driven by an agent loop. Foil attributes these as ai-agent.
Scripted scraping. Classic headless Chrome iterating your listing endpoints, pricing APIs, or search results. Foil attributes these as automation. HTTP-level clients that don’t run a browser never produce a Foil session, so the guard below rejects their requests because they carry no token.

Against both, you want something stronger than a per-user-account rate limit - attackers rotate accounts - and stronger than a per-IP rate limit - attackers rotate residential proxies. The visitor fingerprint is the axis that holds. At the same time, you almost certainly do want some non-human traffic:

Search crawlers (Googlebot, Bingbot) - you want them indexing your public content. Foil labels a session crawler from the request’s User-Agent header, which anyone can send, so verify a crawler before you trust it.
Signed agents - some browser agents sign their requests with HTTP message signatures (Web Bot Auth) to identify themselves. When Foil verifies a signature, it adds a trust label with the value verified to the session’s attribution, and it adds the signing domain as a product label. For signers that Foil recognizes, the actor label is ai-agent, so a policy that blocks every ai-agent session also blocks these agents.
Your own internal tooling - health checks, monitoring, analytics pipelines. These calls don’t come from a browser, so they have no Foil session. Authenticate them with API keys.

The integration on this page separates “block” from “allow” on the attribution, not just the verdict. In the sealed token, attribution.bot.labels lists labels that each have a kind, a value, and a confidence from 0 to 1. The actor label is the one whose kind is actor, and its value is automation, ai-agent, crawler, or unknown. A session can have no actor label, so treat a missing label as unknown.

The flow

Start Foil once at app boot

For authenticated SPAs, start the client when the shell mounts. For public APIs accessed directly (no browser), see the bottom of this page.

Request a handoff when the page makes a protected call

Call getSession() and send the sealedToken as a header. Every call returns a fresh token for the same sessionId. To avoid a Foil call for every request on a chatty page, you can reuse a handoff for a short time.

Verify on each protected request

Verify the token locally with safeVerifyFoilToken() and reject a token that is more than a couple of minutes old. For very hot endpoints, look up the verified session in the Sessions API instead and cache the verdict.

Re-verify on high-value actions or periodically

For mutations or sensitive reads, request a fresh handoff. For cached verdicts, refresh every few minutes.

Split policy by attribution

Allow signed agents (trust label verified) and verified crawlers on read requests; block automation and ai-agent; treat human normally.

Client integration

Request a sealed handoff when the page makes a protected API call, and send the token as a header. getSession() is a call to Foil, so it adds latency to the request that waits for it. The example reuses a handoff for up to one minute so that a burst of requests doesn’t make a Foil call each time, and it requests a fresh handoff for high-value actions. A token that the page reuses is a token that a script could capture from a real browser session and replay. Keep the reuse window short, and set the maximum token age on the server only slightly above it. The example reuses a handoff for one minute, and the server in the next section accepts tokens that are up to two minutes old.

<script type="module">
  const foilPromise = import("https://cdn.usefoil.com/t.js").then(
    (Foil) =>
      Foil.start({
        publishableKey: "pk_live_your_publishable_key",
      }),
  );

  const HANDOFF_REUSE_MS = 60 * 1000;
  let cached = null;

  async function getFoilHandoff({ fresh = false } = {}) {
    if (!fresh && cached && Date.now() - cached.at < HANDOFF_REUSE_MS) {
      return cached.handoff;
    }

    try {
      const foil = await foilPromise;
      const handoff = await foil.getSession();
      cached = { handoff, at: Date.now() };
      return handoff;
    } catch (err) {
      // Send the request without a handoff. The server rejects it.
      return null;
    }
  }

  async function apiFetch(path, init = {}, { fresh = false } = {}) {
    const handoff = await getFoilHandoff({ fresh });
    const headers = new Headers(init.headers);
    if (handoff) {
      headers.set("X-Foil-Token", handoff.sealedToken);
    }
    return fetch(path, { ...init, headers });
  }

  // Request a fresh handoff for high-value mutations
  function apiMutate(path, init) {
    return apiFetch(path, init, { fresh: true });
  }
</script>

If the script fails to load or getSession() fails, the request goes out without a token and the guard in the next section rejects it. If you also serve clients that don’t use your front end, route them by API key before the guard.

Server verification: two patterns

You have two ways to check a session on your server. You can verify the sealed token on each protected request, which is a local crypto operation with no network call, or you can bind the verified session to your own session and read its latest verdict from the Sessions API. The server SDKs verify a token’s contents but don’t check its age. Both patterns need your own age check on decision.evaluated_at, so that a token captured from one session can’t be replayed later.

Pattern A: verify the sealed token per request

The examples do the following:

Verify the sealed token and treat it as missing if verification fails or if the token is more than two minutes old.
Allow a signed agent (a session with a trust label of verified) on GET requests, and block it on everything else.
Allow a session that Foil labels crawler on GET requests only after your own check confirms the crawler. The verified-crawler helper in each example stands in for that check. Use a reverse DNS lookup of the request’s IP address that a forward lookup confirms, or compare the address with the IP ranges that the crawler’s operator publishes.
Block every other session with a bot verdict.

const { safeVerifyFoilToken } = require("@abxy/foil-server");

const MAX_TOKEN_AGE_MS = 2 * 60 * 1000;

function verifyFoil(sealedToken) {
  if (!sealedToken) return null;

  const result = safeVerifyFoilToken(sealedToken, process.env.FOIL_SECRET_KEY);
  if (!result.ok) return null;

  const age = Date.now() - Date.parse(result.data.decision.evaluated_at);
  if (age > MAX_TOKEN_AGE_MS) return null;

  return result.data;
}

async function foilGuard(req, res, next) {
  const foil = verifyFoil(req.get("X-Foil-Token"));
  if (!foil) {
    return res.status(401).json({ error: "Missing or invalid Foil token" });
  }

  const labels = foil.attribution?.bot?.labels ?? [];
  const actor = labels.find((label) => label.kind === "actor")?.value ?? null;
  const isSignedAgent = labels.some(
    (label) => label.kind === "trust" && label.value === "verified",
  );

  // A verified signature identifies the agent. Allow it on reads only.
  if (isSignedAgent) {
    if (req.method !== "GET") return res.status(403).json({ error: "Blocked" });
    req.foilVerdict = { verdict: "signed_agent", actor };
    return next();
  }

  // The crawler label comes from the User-Agent header, which anyone can send.
  // Allow it only after you verify the crawler.
  if (req.method === "GET" && actor === "crawler" && (await isVerifiedCrawler(req))) {
    req.foilVerdict = { verdict: "verified_crawler", actor };
    return next();
  }

  // Block automation, ai-agent, and other bots regardless of method
  if (foil.decision.verdict === "bot") {
    return res.status(403).json({ error: "Blocked" });
  }

  req.foilVerdict = { verdict: foil.decision.verdict, actor };
  next();
}

app.use("/api", foilGuard);

import os
from datetime import datetime, timezone

from foil_server import safe_verify_foil_token

MAX_TOKEN_AGE_SECONDS = 2 * 60


def verify_foil(sealed):
    if not sealed:
        return None

    result = safe_verify_foil_token(sealed, os.environ["FOIL_SECRET_KEY"])
    if not result.ok:
        return None

    evaluated_at = datetime.fromisoformat(
        result.data.decision.evaluated_at.replace("Z", "+00:00")
    )
    age = (datetime.now(timezone.utc) - evaluated_at).total_seconds()
    if age > MAX_TOKEN_AGE_SECONDS:
        return None

    return result.data


def foil_guard(request):
    foil = verify_foil(request.headers.get("X-Foil-Token"))
    if foil is None:
        return {"error": "Missing or invalid Foil token"}, 401

    labels = (foil.attribution.bot or {}).get("labels", [])
    actor = next((label["value"] for label in labels if label["kind"] == "actor"), None)
    is_signed_agent = any(
        label["kind"] == "trust" and label["value"] == "verified" for label in labels
    )

    # A verified signature identifies the agent. Allow it on reads only.
    if is_signed_agent:
        if request.method != "GET":
            return {"error": "Blocked"}, 403
        request.state.foil = {"verdict": "signed_agent", "actor": actor}
        return None

    # The crawler label comes from the User-Agent header, which anyone can send.
    # Allow it only after you verify the crawler.
    if request.method == "GET" and actor == "crawler" and is_verified_crawler(request):
        request.state.foil = {"verdict": "verified_crawler", "actor": actor}
        return None

    if foil.decision.verdict == "bot":
        return {"error": "Blocked"}, 403

    request.state.foil = {"verdict": foil.decision.verdict, "actor": actor}
    return None

import (
    "net/http"
    "os"
    "time"

    foil "github.com/abxy-labs/foil-server-go"
)

const maxTokenAge = 2 * time.Minute

// tokenActor returns the actor label, and whether the token carries a
// trust label with the value "verified".
func tokenActor(token *foil.VerifiedFoilToken) (actor string, signed bool) {
    labels, _ := token.Attribution.Bot["labels"].([]any)
    for _, item := range labels {
        label, ok := item.(map[string]any)
        if !ok {
            continue
        }
        kind, _ := label["kind"].(string)
        value, _ := label["value"].(string)
        switch {
        case kind == "actor":
            actor = value
        case kind == "trust" && value == "verified":
            signed = true
        }
    }
    return actor, signed
}

func FoilGuard(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        tr := foil.SafeVerifyFoilToken(
            r.Header.Get("X-Foil-Token"),
            os.Getenv("FOIL_SECRET_KEY"),
        )
        if !tr.OK {
            http.Error(w, "Missing or invalid Foil token", 401)
            return
        }

        evaluatedAt, err := time.Parse(time.RFC3339, tr.Data.Decision.EvaluatedAt)
        if err != nil || time.Since(evaluatedAt) > maxTokenAge {
            http.Error(w, "Missing or invalid Foil token", 401)
            return
        }

        actor, signed := tokenActor(tr.Data)

        // A verified signature identifies the agent. Allow it on reads only.
        if signed {
            if r.Method != http.MethodGet {
                http.Error(w, "Blocked", 403)
                return
            }
            next.ServeHTTP(w, r)
            return
        }

        // The crawler label comes from the User-Agent header, which anyone
        // can send. Allow it only after you verify the crawler.
        if r.Method == http.MethodGet && actor == "crawler" && isVerifiedCrawler(r) {
            next.ServeHTTP(w, r)
            return
        }

        if tr.Data.Decision.Verdict == "bot" {
            http.Error(w, "Blocked", 403)
            return
        }

        next.ServeHTTP(w, r)
    })
}

require "foil/server"
require "time"

MAX_TOKEN_AGE_SECONDS = 2 * 60

before "/api/*" do
  sealed = request.env["HTTP_X_FOIL_TOKEN"]
  halt 401, { error: "Missing Foil token" }.to_json unless sealed

  tr = Foil::Server::SealedToken.safe_verify_foil_token(sealed, ENV.fetch("FOIL_SECRET_KEY"))
  halt 401, { error: "Invalid Foil token" }.to_json unless tr[:ok]

  token = tr[:data]
  age = Time.now - Time.iso8601(token[:decision][:evaluated_at])
  halt 401, { error: "Invalid Foil token" }.to_json if age > MAX_TOKEN_AGE_SECONDS

  labels = token.dig(:attribution, :bot, :labels) || []
  actor = labels.find { |label| label[:kind] == "actor" }&.dig(:value)
  signed_agent = labels.any? { |label| label[:kind] == "trust" && label[:value] == "verified" }

  # A verified signature identifies the agent. Allow it on reads only.
  if signed_agent
    halt 403, { error: "Blocked" }.to_json unless request.get?
    next
  end

  # The crawler label comes from the User-Agent header, which anyone can send.
  # Allow it only after you verify the crawler.
  next if request.get? && actor == "crawler" && verified_crawler?(request)

  halt 403, { error: "Blocked" }.to_json if token[:decision][:verdict] == "bot"
end

use Foil\Server\SealedToken;

const MAX_TOKEN_AGE_SECONDS = 2 * 60;

function foilGuard(): void {
    $sealed = $_SERVER["HTTP_X_FOIL_TOKEN"] ?? null;
    if (!$sealed) { http_response_code(401); exit; }

    $tr = SealedToken::safeVerify($sealed, getenv("FOIL_SECRET_KEY"));
    if (!$tr->ok) { http_response_code(401); exit; }

    $age = time() - strtotime($tr->data->decision["evaluated_at"]);
    if ($age > MAX_TOKEN_AGE_SECONDS) { http_response_code(401); exit; }

    $actor = null;
    $isSignedAgent = false;
    foreach ($tr->data->attribution["bot"]["labels"] ?? [] as $label) {
        if ($label["kind"] === "actor") {
            $actor = $label["value"];
        }
        if ($label["kind"] === "trust" && $label["value"] === "verified") {
            $isSignedAgent = true;
        }
    }
    $isRead = $_SERVER["REQUEST_METHOD"] === "GET";

    // A verified signature identifies the agent. Allow it on reads only.
    if ($isSignedAgent) {
        if (!$isRead) { http_response_code(403); exit; }
        return;
    }

    // The crawler label comes from the User-Agent header, which anyone can send.
    // Allow it only after you verify the crawler.
    if ($isRead && $actor === "crawler" && isVerifiedCrawler()) {
        return;
    }

    if ($tr->data->decision["verdict"] === "bot") {
        http_response_code(403);
        exit;
    }
}

# Session readback for the Sessions API pattern (see below)
curl https://api.usefoil.com/v1/sessions/sid_... \
  -H "Authorization: Bearer sk_live_..."

Pattern B: read the verdict from the Sessions API

For very hot endpoints, verify a fresh token once, store the verified session_id in your own server-side session, and read the session’s latest verdict from Foil on later requests, caching it for a few minutes. Foil’s server-side behavioral scoring can move a verdict between the snapshot and behavioral phases as evidence accumulates, so a lookup can be more current than the token that you verified first. Take the session ID from a token that you verified, not from a request header. A client can send any session ID, including one that it copied from a real browser session.

Node.js

const { Foil, FoilApiError } = require("@abxy/foil-server");
const client = new Foil({ secretKey: process.env.FOIL_SECRET_KEY });
const verdictCache = new Map(); // session_id → { verdict, actor, fetchedAt }

const CACHE_MS = 5 * 60_000; // refresh after 5 minutes
const MAX_STALE_MS = 15 * 60_000; // never serve a verdict older than 15 minutes

// `sessionId` is the `session_id` of a token that you verified and stored in
// your own server-side session.
async function getCachedVerdict(sessionId) {
  const cached = verdictCache.get(sessionId);
  const age = cached ? Date.now() - cached.fetchedAt : Infinity;
  if (age < CACHE_MS) return cached;

  let session;
  try {
    session = await client.sessions.get(sessionId);
  } catch (err) {
    // Foil limits how many requests your secret keys can make per minute.
    // When it does, keep using the last verdict for a limited time.
    if (err instanceof FoilApiError && err.status === 429 && age < MAX_STALE_MS) {
      return cached;
    }
    throw err;
  }

  const actor =
    session.attribution?.labels.find((label) => label.kind === "actor")?.value ?? null;
  const entry = {
    verdict: session.decision.automation_status, // "automated" | "human" | "uncertain"
    actor,
    fetchedAt: Date.now(),
  };
  verdictCache.set(sessionId, entry);
  return entry;
}

The session readback returns decision.automation_status ("automated" | "human" | "uncertain") rather than the sealed-token verdict field, and its attribution is null when Foil has nothing to attribute. The session’s attribution.labels have the same kind and value as the token’s labels, plus a display label and a confidence from 0 to 100. See Server verification for the full shape. Foil limits all of your organization’s secret keys together to 600 requests per minute by default, and a limited request returns 429 with a Retry-After header. See Rate limits. The server SDKs don’t retry for you, so the example keeps using the last cached verdict when Foil returns a 429, but only until that verdict is 15 minutes old. After that, getCachedVerdict() throws, and your handler should treat the session as having no verdict and ask the client for a fresh token. Each cached session costs one request per cache period, so a longer period lets you serve more concurrent sessions within the limit.

When to re-verify

Session reuse is great for throughput, but a long-lived session becomes a stale verdict. Three refresh triggers worth coding:

High-value mutations - a user changing their email, deleting data, exporting an archive. Always ask for a fresh sealed handoff from the client, as apiMutate does in the client example.
Periodic refresh - every 5–15 minutes for long sessions, re-fetch GET /v1/sessions/:sessionId. Foil’s server-side behavioral scoring can move a verdict between snapshot and behavioral phases as evidence accumulates, and you want the latest. Each refresh counts against your secret-key rate limit, as described in Pattern B.
Suspicious pattern observed - your own application logic (burst of identical queries, geographic jump) can trigger a client-side foil.getSession() that produces a new, freshly-attested handoff.

Splitting policy by attribution

For read APIs, the policy matrix that works on most sites:

Session	GET	POST / PUT / DELETE
`human` verdict	Allow	Allow
Signed agent (`trust` label `verified`)	Allow	Block (agents generally shouldn’t mutate)
`crawler` that you verified	Allow	Block
`crawler` that you couldn’t verify	Treat as `bot`: block or throttle	Block
`automation`	Block or throttle	Block
`ai-agent` without a `verified` trust label	Block or throttle	Block
`unknown` or no actor label, with a `bot` verdict	Throttle	Block

Check the trust label before you check the actor label. Foil gives signed agents from signers it recognizes the actor label ai-agent, so a policy that blocks every ai-agent session also blocks the agents that identify themselves. “Allow” doesn’t mean unlimited. Keep a generous rate limit on signed agents and verified crawlers - a badly-written crawler can still hurt you - but don’t return 403s. If you want AI agents to read your content but not scrape it, set different caps for ai-agent: a low QPS limit that’s fine for an agent answering one user’s question and painful for a training-data crawler.

APIs called directly (no browser)

Some API consumers don’t run a browser at all - a mobile app, a server-to-server integration, a CLI tool. Requests to your own API go to your servers, not to Foil, so Foil never sees them. Direct API traffic has no Foil session, and the browser SDK doesn’t apply. For this traffic:

Require API keys. Issue a key to each customer or integration, and route requests that don’t carry a Foil token down the API-key path. Apply Foil only to requests from your browser front end.
Rate limit by key. Limit each API key, and limit by IP address the traffic that has no key. A per-key limit contains a leaked or shared key.
Use the native SDKs for your own mobile apps. The iOS SDK and the Android SDK return a sealed handoff that you verify on your server in the same way as a browser token.

What’s next

User-generated content

The write-side counterpart: stop LLM posts at the composer.

Server verification

Reference for both sealed-token and session-readback paths.

Detection categories

How Foil combines its checks into a verdict.

Going to production

Rollout plan for API-wide enforcement.

Overview

Get started

Integrate

Use cases

Learn

Resources

API abuse & scraping

The threat

The flow

Client integration

Server verification: two patterns

Pattern A: verify the sealed token per request

Pattern B: read the verdict from the Sessions API

When to re-verify

Splitting policy by attribution

APIs called directly (no browser)

What’s next

User-generated content

Server verification

Detection categories

Going to production

​The threat

​The flow

​Client integration

​Server verification: two patterns

​Pattern A: verify the sealed token per request

​Pattern B: read the verdict from the Sessions API

​When to re-verify

​Splitting policy by attribution

​APIs called directly (no browser)

​What’s next

User-generated content

Server verification

Detection categories

Going to production

The threat

The flow

Client integration

Server verification: two patterns

Pattern A: verify the sealed token per request

Pattern B: read the verdict from the Sessions API

When to re-verify

Splitting policy by attribution

APIs called directly (no browser)

What’s next