An AI platform can only cite a page its crawler has read. AI crawlers reads your server logs and shows what GPTBot, ClaudeBot, PerplexityBot, Googlebot and the other AI crawlers actually fetched from your site.
Open it from Explorer > Search > AI crawlers in the sidebar (/analytics/ai-crawlers). It is included in Pro and above and costs no credits.
Click Upload a log
Choose a file or drop it in the box
Click Upload
The file is read in the background and the page updates when it is done. The Uploaded logs table shows each file with its status, the requests read and the bot hits found.
What you can upload (up to 100 MB per file, gzipped files are fine):
Source | Format |
|---|---|
nginx, Apache, LiteSpeed | The access log in the Combined Log Format, the default on most servers |
Cloudflare | Logpush in JSON lines |
Netlify, Vercel, other hosts and CDNs | Any export as JSON lines or CSV with a time, a URL or path, a status and a user agent column |
A file over 100 MB can be split and uploaded in parts. Uploading the same period twice counts those visits twice, so upload each period once.
If your site runs behind Cloudflare, Rank Prompt can read its AI crawler visits every hour with a token that can only read. Any Cloudflare plan works, the free one included.
On the AI crawlers page, click Connect Cloudflare (in Sources, or in Integrations > Measure)
In Cloudflare, open My Profile > API Tokens and choose Create Token, then Create Custom Token
Add two permissions: Zone, Zone, Read and Zone, Analytics, Read
Under Zone Resources, include your site's zone, create the token and paste it in Rank Prompt with the site's domain
Click Connect
A Cloudflare zone serves every subdomain under it, so Rank Prompt reads only your site's domain and its www.. If your zone serves other hosts, Rank Prompt lists them after you connect: tick the ones that are your site and click Save hosts. You can change them later with Choose hosts on the source. Visits from addresses a crawler's company does not publish still show, marked as not verified.
The first read brings the last 30 days Cloudflare keeps, and the source shows Reading the last month until it is done, usually a few minutes. A host you add later gets its last month read too. After that each hour is read once it has settled, so a visit shows up one to two hours after it happened. Cloudflare samples busy sites, and Rank Prompt counts each sampled visit as the visits it stands for. If Cloudflare stops accepting the token, the source shows Error with the reason and a Connect again button. The token is kept encrypted.
A WordPress site can send its AI crawler visits on its own, every five minutes, through the free Rank Prompt Connector plugin (version 1.1.0 or later).
Connect the site in Integrations > Publish content with a WordPress user who is an administrator
On the AI crawlers page, click Connect WordPress (in Sources)
In WordPress, open Settings > Rank Prompt and click Send AI crawler visits to Rank Prompt
Nothing is noted or sent until step 3. Pages a cache serves before WordPress loads (WP Rocket, Varnish, a CDN) are not seen, so counts can run low; an access log from the server or CDN counts everything. Disconnect in Sources stops sending.
Bot hits: requests from AI crawlers in the period
Verified: the share of hits that came from an address the crawler's company publishes. Someone can send requests with GPTBot's name; only OpenAI's own addresses count as verified. Crawlers whose companies publish no addresses (Meta, Amazon, ByteDance) are never verified
Pages crawled: distinct pages AI crawlers fetched
Error rate: hits that got a 4xx or 5xx answer. A crawler that hits an error cannot read the page
Hits per day for the six most active crawlers
Crawlers: each crawler with its company, what it fetches for (Training, Search index, User request, Ads), its hits, verified share, pages, errors and last visit
Pages crawled: each page with the crawlers that fetched it, its hits, errors, last status and last visit. Filter it by crawler
Pick the period (7, 30 or 90 days) at the top of the page.
Search index and User request crawlers (OAI-SearchBot, ChatGPT-User, PerplexityBot, Claude-SearchBot) are the ones behind citations. A page they never fetch cannot be cited in their answers.
Errors on a page a crawler keeps fetching are the first thing to fix.
Training crawlers (GPTBot, ClaudeBot, CCBot) gather data for future models; their volume says nothing about citations today.
Few or no AI crawlers at all usually means robots.txt or a firewall blocks them: run a Technical SEO audit on your home page.
Only daily counts per crawler and page are kept. IP addresses are used to verify each hit and then dropped, query strings are removed from every path, and the uploaded file is deleted as soon as it is read. Counts are kept for 13 months.
AI crawlers is part of Pro, Agency and Agency Plus. If a brand's plan drops below Pro, its uploads and connected sources stop being read and the page shows the upgrade. Back on Pro, the sources pick up where they stopped.
