An AI platform can only cite a page its crawler has read. AI crawlers reads your server logs and shows what GPTBot, ClaudeBot, PerplexityBot, Googlebot and the other AI crawlers actually fetched from your site.
Open it from Explorer > Search > AI crawlers in the sidebar (/analytics/ai-crawlers). It is included in Pro and above and costs no credits.
Rank Prompt looks at how your site is served and offers that way first; the others stay one click away:
Cloudflare when Cloudflare sits in front of the site. It sees every request, even when Vercel or WordPress run behind it.
Vercel when Vercel serves the site.
WordPress when the site runs WordPress and nothing in front of it reports visits.
Upload a log for everything else. A Netlify site uploads a log too: Netlify's log drains need its Enterprise plan.
A Rank Prompt hosted blog reports its own AI crawlers: the source appears by itself once the blog is live, on the plan that has the blog (Starter and above), with nothing to connect. Its history starts the day it started reporting. A blog in a folder of your site whose Cloudflare or Vercel source already reads that host is counted once, by that source.
Click Upload a log
Choose a file or drop it in the box
Click Upload
The file is read in the background and the page updates when it is done. The Uploaded logs table shows each file with its status, the requests read and the bot hits found.
What you can upload (up to 100 MB per file, gzipped files are fine):
Source | Format |
|---|---|
nginx, Apache, LiteSpeed | The access log in the Combined Log Format, the default on most servers |
Cloudflare | Logpush in JSON lines |
Netlify, other hosts and CDNs | Any export as JSON lines or CSV with a time, a URL or path, a status and a user agent column |
A file over 100 MB can be split and uploaded in parts. Uploading the same period twice counts those visits twice, so upload each period once.
If your site runs behind Cloudflare, Rank Prompt can read its AI crawler visits every hour with a token that can only read. Any Cloudflare plan works, the free one included.
On the AI crawlers page, click Connect Cloudflare (in Sources, or in Integrations > Measure)
In Cloudflare, open My Profile > API Tokens and choose Create Token, then Create Custom Token
Add two permissions: Zone, Zone, Read and Zone, Analytics, Read
Under Zone Resources, include your site's zone, then create the token and copy it
Back in Rank Prompt, check the Domain (it starts as the brand's site), paste the token in API token and click Connect
A Cloudflare zone serves every subdomain under it, so Rank Prompt reads only your site's domain and its www.. If your zone serves other hosts, Rank Prompt lists them after you connect: tick the ones that are your site, type any other host of the zone under Add (for example blog.yourbrand.com), and click Save hosts. Up to 20 hosts can be read. You can change them later with Choose hosts on the source. Visits from addresses a crawler's company does not publish still show, marked as not verified.
The first read brings the last 30 days Cloudflare keeps, and the source shows Reading the last month until it is done, usually a few minutes. A host you add later gets its last month read too. After that each hour is read once it has settled, so a visit shows up one to two hours after it happened. Cloudflare samples busy sites, and Rank Prompt counts each sampled visit as the visits it stands for. If Cloudflare stops accepting the token, the source shows Error with the reason and a Connect Cloudflare again button: Cloudflare refused the token means it was deleted or expired, and The token cannot read this domain means its Zone Resources leave your site's zone out. Connecting again replaces the token. The token is kept encrypted.
Cloudflare only sees the requests that pass through it. If your site's DNS record is DNS only (grey cloud) and another host answers it, Netlify for example, Cloudflare counts nothing. Rank Prompt checks how your site answers and then says Cloudflare is not in front on the page and on the source, offers the source that fits instead, and leaves Cloudflare's days out of the chart. Turn on the proxy (orange cloud) on the record, or use another source.
If Vercel serves your site, Rank Prompt creates a log drain on your project that sends each AI crawler visit as it happens, usually within a minute. Log drains need a Pro or Enterprise team on Vercel, and Vercel bills the drained volume to that team (see Vercel's drains pricing). Rank Prompt keeps only the AI crawler visits.
On the AI crawlers page, click Connect Vercel (in Sources, or in Integrations > Measure)
In Vercel, open Account Settings > Tokens and choose Create Token. Set its Scope to the team that owns the site's project and pick an expiration
Paste the token in Rank Prompt and click Find projects
Choose the project. The one serving your site is marked Serves your site
Click Connect
The drain sends only production traffic, signed with a secret only Rank Prompt and Vercel hold; anything not signed with it is refused. The token is kept encrypted and used to create the drain, check it once a day and remove it when you disconnect. Until the first crawler visit arrives the source shows Waiting for the first delivery.
If the drain stops, the source shows Error with the reason and a Connect Vercel again button: the token was refused, the team left Pro, the drain was deleted or paused in Vercel, or the token cannot reach the project anymore. Connecting again makes a new drain and removes the old one. Disconnect removes the drain from Vercel; if Vercel cannot be reached, delete it in your team's Settings > Drains.
A WordPress site can send its AI crawler visits on its own, every five minutes, through the free Rank Prompt Connector plugin (version 1.1.0 or later).
Connect the site in Integrations > Publish content with a WordPress user who is an administrator
On the AI crawlers page, click Connect WordPress (in Sources)
In WordPress, open Settings > Rank Prompt and click Send AI crawler visits to Rank Prompt
Nothing is noted or sent until step 3. Pages a cache serves before WordPress loads (WP Rocket, Varnish, a CDN) are not seen, so counts can run low; an access log from the server or CDN counts everything. Disconnect in Sources stops sending.
Bot hits: requests from AI crawlers in the period
Verified: the share of hits that came from an address the crawler's company publishes. Someone can send requests with GPTBot's name; only OpenAI's own addresses count as verified. Crawlers whose companies publish no addresses (Meta, Amazon, ByteDance) are never verified
Pages crawled: distinct pages AI crawlers fetched
Error rate: hits that got a 4xx or 5xx answer. A crawler that hits an error cannot read the page
Hits per day for the six most active crawlers
Crawlers: each crawler with its company, what it fetches for (Training, Search index, User request, Ads), its hits, verified share, pages, errors and last visit
Pages crawled: each page with the crawlers that fetched it, its hits, errors, last status and last visit. Filter it by crawler
Pick the period (7, 30 or 90 days) at the top of the page.
Search index and User request crawlers (OAI-SearchBot, ChatGPT-User, PerplexityBot, Claude-SearchBot) are the ones behind citations. A page they never fetch cannot be cited in their answers.
Errors on a page a crawler keeps fetching are the first thing to fix.
Training crawlers (GPTBot, ClaudeBot, CCBot) gather data for future models; their volume says nothing about citations today.
Few or no AI crawlers at all usually means robots.txt or a firewall blocks them: run a Technical SEO audit on your home page.
Only daily counts per crawler and page are kept. IP addresses are used to verify each hit and then dropped, query strings are removed from every path, and the uploaded file is deleted as soon as it is read. Counts are kept for 13 months.
AI crawlers is part of Pro, Agency and Agency Plus. If a brand's plan drops below Pro, its uploads and connected sources stop being read and the page shows the upgrade; a Vercel drain is switched off so Vercel stops billing its volume. Back on Pro, the sources pick up where they stopped.