We Logged 30 Days of AI-Bot Hits. The Number Agencies Should Report Was Much Smaller

On two of our busiest paths, roughly 70% of requests claiming to be AI bots could not be verified. Our /robots.txt file logged 13,294 claimed hits in 30 days; Cloudflare verified 3,730. The homepage logged 6,912 claimed hits and 2,046 verified. Those requests happened, but a User-Agent string alone does not tell you who sent them. If an agency reports the larger number as AI-engine activity, the report is doing more work than the evidence.

The more useful contrast sits deeper in our logs. Three category hubs drew about 685 verified fetches between them and zero recorded live-answer fetches. One dated article, /post/google-ads-updates-2026, drew 537 verified fetches and 330 recorded live-answer events. That does not prove the article won 330 citations. It does tell us which pages the engines fetched while building answers. For an agency retainer, I would sell and measure those answer pages, not a hub refresh and a chart of raw bot hits.

What 30 days of edge logs can—and cannot—tell us

We examined 30 days of Cloudflare edge logs for the 25 most-fetched paths on workwithgroas.com. We used an open bot-analytics method that separates User-Agent matches from verified bots. This is one site and a selected set of paths, not a benchmark for every client or every AI engine. The figures in the tables below come from those logs unless a separate source is linked in the prose.

The distinction starts with identity. A request can claim to be ChatGPT-User or Claude-User by putting that name in its User-Agent header. Cloudflare describes the fake-bot failure mode: a string matches a known bot, but the source IP does not. Verification adds an identity check; it does not turn every verified fetch into a citation.

That last distinction matters. I would rather give a client a smaller, defensible number than put AI visibility above a large number the logs cannot explain.

Finding 1: The claimed-hit column overwhelms the verified one

The gap appears immediately on the three busiest paths in our 30-day sample:

A request did reach the edge each time. What we cannot do is credit every request to the operator named in its header. Cloudflare’s fake-bot guidance explains why the string alone fails that test, and OpenAI provides a published ChatGPT-User IP list for checking that identity.

There is a limit in the other direction, too. This Cloudflare-based method only verifies operators registered for its checks. An unverified request is not automatically an impostor; it is a request this setup did not confirm. On /robots.txt and the homepage, the verified share landed around 28–30%. I would not apply that percentage to the rest of the internet, or even to every path on this site. The reporting rule is narrower and sturdier: label claimed hits as claimed, and use verified fetches when you say an engine visited.

Finding 2: The hubs got verified fetches, not recorded answers

Across three category hubs, our logs show roughly 685 verified fetches and zero recorded live-answer fetches during the same 30 days. The bots did not ignore those pages. The recorded answer-building activity did.

That is a useful warning if a retainer promises AI visibility through hub rewrites. A hub can organize a site and still give an answer engine little to use for a particular question. Our logs show that split on these pages; they do not establish that hubs everywhere fail, or that no answer ever mentioned one of ours outside what this measurement captured.

External citation research points in a similar direction without measuring the same thing. BrightEdge’s analysis of AI Overview citations found that 82.5% pointed to deep content pages at least two clicks from the homepage, 0.5% to homepages, and about 17% to hubs. It also found that 86% of cited pages appeared for only one keyword. Those are citation figures from a different dataset, not a conversion rate I can paste onto our Cloudflare logs. They are a reason to test specific answer pages before making another category hub the centerpiece of a client package.

Finding 3: One dated article drew the answer activity

The outlier was /post/google-ads-updates-2026. It recorded 537 verified fetches and 330 live-answer events in our sample. Of the answer events, 326 were attributed to ChatGPT-User and four to Claude-User. Set that beside the hubs and the difference is hard to miss:

There is a verification wrinkle worth keeping in the table’s interpretation, not hiding in a footnote. This Cloudflare setup does not tag Claude-User as verified. The four Claude-associated answer events therefore cannot meet the same verification standard as the confirmed fetch count. On the confirmed ChatGPT-User portion, 326 answer events against 537 verified fetches is about 61%. That is a useful answer-activity ratio for this page, not a count of visible citations and not proof that each fetch caused an answer.

Why did this page stand apart? Its title names a dated subject, and the article gives the reader numbers, steps, and a bounded answer. A category hub points onward to several answers. The logs cannot isolate which piece of that structure made the difference, but they show where answer-building fetches accumulated. I used to tell clients a strong category page would collect citations because it looked authoritative. I would no longer make that the default bet. Put the usable answer on the page you want fetched.

Finding 4: One bot’s pattern is not another bot’s pattern

Our logs also recorded 834 ChatGPT-User hits on the homepage and 1,216 Claude-User hits on /robots.txt. Those are path-level observations, not enough to declare that one engine always heads for a homepage while another always checks permission first. They are enough to make a single combined AI bots line look crude.

The operators describe different controls. OpenAI describes ChatGPT-User as user-initiated and says robots.txt rules may not apply to it. Anthropic describes Claude-User as honoring robots.txt. Our two path counts do not independently prove those policies; they show why an agency should break activity out by engine and path instead of assigning one behavior to every bot in the report.

The retainer deliverable: answer pages, not a hub-refresh tally

If I were packaging this work for an agency client, I would not sell a hub refresh as the main AI-visibility deliverable. Our hubs received verified attention without recorded answer activity. The dated article received both. That is one site and one month, so it is a direction for the next piece of work, not a guarantee that every answer page beats every hub.

The deliverable I would test is specific: one page that answers one dated question with a passage an engine can use, followed by measurement of verified fetches and recorded answer activity. If that page draws fetches but no answer events, the next edit has a clear target. If a hub keeps drawing fetches and nothing else, I would not call its traffic an AI-visibility win.

I learned to distrust the impressive-looking page back when I managed PPC landing pages. The category page looked strategic in the deck and converted nothing; the specific page with the price, timeline, and use case carried the account. I would not claim search-answer systems work exactly like landing-page conversion. The practical discipline does carry over: measure what the page was built to do, not how important it looks in a site diagram.

Put three defensible numbers in the client report

The claimed-hit chart belongs in a diagnostic appendix, if it belongs anywhere. On the paths above, it is several times larger than the verified count, and its label invites the wrong interpretation. A client-facing report can be shorter and more useful:

1. Verified fetches by engine and path. Show which operator the setup could confirm, alongside any coverage limits. Do not relabel an unverified Claude-User request as a confirmed Claude visit.
2. Recorded live-answer events by page and engine. For our dated article, that is 330 recorded events, including four Claude-associated events this setup could not verify to the same standard.
3. Answer activity relative to verified fetches. Keep the numerator’s verification status visible. The hubs had zero recorded answer events against about 685 verified fetches; the article had 326 confirmed ChatGPT-User answer events against 537 verified fetches, or about 61%.

That third number helps choose what to revise or publish next. It is not a revenue metric. Neither an answer-building fetch nor a citation tells you whether someone became a qualified lead, much less a customer. Report the step you measured; do not rename it as the outcome you hope follows.

Demand execution from the tools behind the report

A dashboard can show a problem without fixing a page. HubSpot’s AEO tool is described as a way to see what is happening and prioritize work, rather than write content, build schema, or carry out technical fixes. That can be useful for an audit. It is a weak foundation for an execution retainer if nobody is responsible for shipping the work. What the deck calls “coverage,” I call a to-do list you still have to pay someone to do.

Before putting a platform on client domains, I would ask for three things in writing:

• Verified-bot data by path, with the method and its operator-coverage limits stated plainly.
• Per-page answer-activity reporting, split by engine and clear about which events are independently verified. Otherwise an answer rate can mix unlike numbers.
• Content execution in the service, so the answer pages and technical fixes actually ship instead of remaining recommendations in another login.

That is why I do not sell AEO by the prompt. A prompt count is easy to invoice and hard to connect to work shipped. Pages, verified fetches, and recorded answer activity make a more honest operating loop. It is also the model groas offers agencies through white-label delivery under their brand: content, technical fixes, and citations shipped each week, for a flat monthly fee per client domain, rather than a dashboard the agency still has to operate.

Price the retainer around the work you must deliver

Tool prices are easy to compare badly. One published comparison lists Profound at $99 a month billed yearly for ChatGPT only, Semrush at $99 a month per domain for 25 prompts, Peec at $95 a month for three of six engines, and Scrunch at $250 a month billed yearly for all six. Those figures describe different limits. They do not tell you which product supplies verified, per-path fetch data or publishes an answer page for you.

White-label plans introduce another unit. LLM Pulse lists Starter at €49 a month for 50 prompts and Growth at €99 for 150 prompts with unlimited seats; the same comparison lists AgencyAnalytics from $79 for five clients to $479 for 15. A seat, a client, a domain, a tracked engine, and a prompt are not interchangeable. If you charge $500–$2,000 a month per client for an AEO add-on, check which of those units will become the constraint before you promise a monthly deliverable.

I would price the reporting and the content work together. A cheap prompt dashboard does not become cheap delivery when your team still has to verify the fetches, write the page, publish it, and explain what the resulting count means. Buy the measurement you can defend and the execution you can deliver, not the largest-looking visibility score.

The decision these logs support

This sample covers workwithgroas.com, 30 days, and only its 25 most-fetched paths. It does not tell us what every page on the site did, whether a user saw a particular citation, or how another client’s hubs would perform. Claude-User verification is incomplete in this setup. I would not turn our 28–30% verified shares on two paths into an industry benchmark, or turn one article’s answer activity into a universal rule.

I would still change the retainer. I would scope four specific answer pages a month, each tied to one question, with a date and a passage that answers it directly. I would review verified fetches and recorded live-answer events by engine each week, keeping the verification gap visible. I would not promise a hub rewrite as the lead deliverable, and I would not put claimed bot hits in the headline box of the report.

The decision is not that hubs have no purpose. It is that our 685 verified hub fetches produced no recorded answer events, while one specific article recorded 330. Raw hits flatter an agency report. Pages shipped and answer activity give the client something to inspect, improve, and hold us accountable for. That is the number I would put my name on.