AI analytics

The AI section shows how AI assistants and crawlers interact with your site: when ChatGPT, Claude, Gemini or Perplexity send you visitors, which crawlers index your pages, and which ones train on your content. Visits from AI chats are tracked automatically; crawler tracking needs a one-line middleware β€” this guide sets up both.

What the AI section shows

The card sits on your site dashboard, between the Goals card and the growth (MRR/ARR/Impressions) card, and has three tabs:

  • πŸ€– AI answers β€” people who clicked a link to your site inside an AI chat (β€œClicks”), plus the user-triggered fetches behind them (β€œAI reads” β€” when ChatGPT-User or Claude-User opens a page to answer someone's question).
  • πŸ” Indexing β€” autonomous crawlers that fetch your pages to index them for retrieval: OAI-SearchBot, Claude-SearchBot, PerplexityBot, Applebot and friends.
  • 🧠 Training β€” crawlers that collect training data: GPTBot, ClaudeBot, Common Crawl's CCBot, Bytespider and others.

Each tab shows its total next to the label, a multi-line graph of activity over your selected date range, and a ranked list of the AI products or pages behind it. The units stay honest: AI answers are counted in visitors, crawler tabs in requests β€” one fetch is one row, never a sampled estimate.

The AI section with real crawler data: tabs with counts, a multi-line graph and the ranked crawler list
The AI section with data: tabs carry their totals, the graph plots one line per AI product, and the list on the right ranks them.

AI answers: clicks and AI reads

Clicks need no setup. When a visitor arrives from chatgpt.com, claude.ai, gemini.google.com, perplexity.ai, copilot.microsoft.com, meta.ai, grok.com or chat.deepseek.com, the session is tagged with the product and lands in the AI answers tab. Links that carry utm_source=chatgpt.com (ChatGPT appends it to citations, and its mobile apps often send no referrer at all) are caught too.

AI-chat clicks are ordinary visits β€” they still count in your Visitors tile, channels and pages like any other traffic. The AI section is a lens on them, not a separate dataset. Historical clicks are backfilled from your stored referrers, so the tab has data from day one.

AI reads (the second toggle on the answers tab) are fetches made on a human's behalf β€” ChatGPT-User opening your page to quote it, for example. They prove an AI read your content for someone; they are not people visiting. Like all crawler fetches, they need the middleware from the next section.

Set up crawler tracking

Your tracking beacon is a JavaScript file, and most AI crawlers never execute JavaScript β€” GPTBot, ClaudeBot, CCBot and Bytespider fetch raw HTML only. To see them, your server reports each request's user agent to statiz. It's one middleware on your side:

  1. Open the setup snippet. On your site dashboard, scroll to the AI section and click Set up crawler tracking. The snippet is pre-filled with your deployment URL and a key that is unique to this site (stzai.<site-id>.<key>) β€” owners and editors see it, viewers are told to ask an admin.
    The AI section's setup state with the middleware snippet revealed
    The setup state β€” copy the middleware from here; the key is already yours.
  2. Add the middleware. For a Next.js site, paste this into middleware.ts in your project root (replace the host and key with the ones from your snippet):
    1import { NextResponse, type NextRequest, type NextFetchEvent } from "next/server";23export function middleware(req: NextRequest, event: NextFetchEvent) {4  const ua = req.headers.get("user-agent") ?? "";5  if (/bot|crawl|spider|-user|agent|notebook/i.test(ua)) {6    event.waitUntil(fetch("https://YOUR-STATIZ-HOST/api/track/hit", {7      method: "POST",8      headers: {9        authorization: "Bearer stzai.<site-id>.<key>",10        "content-type": "application/json",11      },12      body: JSON.stringify({ ua, path: req.nextUrl.pathname, host: req.nextUrl.hostname }),13    }));14  }15  return NextResponse.next();16}1718export const config = { matcher: ["/((?!_next/|api/|.*\\..*).*)"] };
    Any stack that can POST works the same way β€” Cloudflare Workers, log drains and plain servers send the exact JSON shape { ua, path, host } to /api/track/hit with the same bearer key.
  3. Verify it. Run the curl tab from the snippet β€” the response is { "accepted": 1 } for a recognized crawler. Real traffic shows up within minutes: the crawler tabs display a Crawler tracking since date and start plotting. The pre-filter in the middleware is deliberately broad; statiz's own registry decides what is an AI crawler, so your copy never goes stale.
A few agents do run JavaScript (Applebot renders pages). Those are counted automatically even without the middleware β€” and once your middleware reports, statiz stops double-counting beacon-origin hits so nothing is ever counted twice.

Read the graphs

Every tab plots one colored line per AI product over your date range. Hovering the graph shows each product's count per bucket; hovering a row in the list emphasizes its line and dims the rest.

The Training tab: GPTBot, ClaudeBot, Common Crawl and Bytespider lines over a month
The Training tab β€” one line per AI product, the ranked list on the right.

Two controls in the corner change how the graph groups:

  • The Crawlers / Pages pill opens a checkbox filter over the current grouping. Uncheck a crawler to remove its curve β€” isolate exactly the lines you care about. Clear all and All N reset the selection.
  • The icon button next to it flips the grouping: β€œGrouped by crawlers β€” tap to group by pages instead”. Pages mode replots the graph per URL β€” which pages get crawled, indexed or read β€” with a search box in the filter dropdown for long page lists.
The crawler filter dropdown open: checkbox rows with brand icons and counts
The crawler filter β€” uncheck a product to take its curve off the graph.
Pages grouping with the page filter dropdown and its search box
Grouped by pages: search a path, check the pages you want plotted.

Which crawlers are recognized

statiz maintains a registry of AI agents and classifies every request by its user agent before anything is stored. The products you'll most often see:

  • ChatGPT (OpenAI) β€” GPTBot trains, OAI-SearchBot indexes, ChatGPT-User fetches for a person.
  • Claude (Anthropic) β€” ClaudeBot trains, Claude-SearchBot indexes, Claude-User fetches for a person.
  • Perplexity β€” PerplexityBot indexes (not used for training, per their docs), Perplexity-User fetches.
  • Gemini (Google) β€” appears as AI-chat clicks from gemini.google.com and user fetches by Google-Agent. There is deliberately no Gemini line on the crawler tabs: Google trains Gemini via the Google-Extended robots.txt token, which never appears in any request log.
  • Also tracked: Meta (meta-externalagent, meta-webindexer), Amazon (Amazonbot, Amzn-SearchBot), Apple (Applebot), Mistral, DuckAssist, Common Crawl (CCBot), ByteDance (Bytespider), Grok, DeepSeek and the long tail.
Googlebot and bingbot feed their companies' AI features from their search indexes, but they are classic search crawlers β€” statiz counts them as bots and they stay out of your analytics entirely, including the AI section. Clicks from Google AI Overviews arrive as ordinary Google search traffic and cannot be separated.

Notes & caveats

  • Exclusions apply. Paths you exclude in Settings β†’ Exclusions are not reported by the middleware, and excluded IPs and hostnames are dropped at ingestion, same as human visits.
  • Filters don't narrow crawlers. Dashboard filters describe visitors (country, device, browser). Crawlers have none of those attributes, so the Indexing and Training tabs show a note instead of silently-empty zeros while a filter is active. AI answers clicks honor every filter like the rest of your analytics.
  • Query strings are dropped from reported paths β€” /pricing?ref=x and /pricing are the same page. URLs can carry tokens; statiz keeps only the path.
  • Crawler history starts at install. Clicks are backfilled from stored referrers; crawler fetches were never recorded before the middleware existed, so Indexing and Training accrue from the day you set them up β€” the β€œtracking since” date says so honestly.
  • A user agent is a claim, not proof. Anyone can send a GPTBot string. Your human stats are unaffected (spoofers never become visitors); treat crawler counts as self-reported. The reporting key is unique to your site β€” rotate it by rotating APP_SECRET.
  • Privacy: statiz never stores visitor IPs, and crawler reports carry no IPs at all β€” only the user agent (kept for registry upkeep), path and timestamp.
  • robots.txt still rules access. This is analytics, not access control β€” blocking GPTBot happens in your robots.txt or firewall, not here.