Crawler reports can't tell you whether ChatGPT finds you. We checked on our own site.

Key findings

• In 22 daily readings of one site's crawler logs, 9 September to 4 October 2026, OpenAI's search crawler OAI-SearchBot made zero requests. The only reading that counted any, 29 on 3 October 2026, came from one address in the Netherlands sending requests under the names of at least 9 different crawlers.
• Asked 3 questions about that site, two of which only its own pages can answer, ChatGPT answered 2 of the 3 from those pages and linked the site on 21 September 2026, though OpenAI's search crawler had never visited it.
• In the seven days to 2 October 2026, Googlebot made 182 requests to the site, BingBot 147, GPTBot 7, Meta-ExternalAgent 86 and OAI-SearchBot none.
• Anthropic's training crawler ClaudeBot made 98, 104 and 18 requests a day in its first readings of the site and zero in each of the 21 readings that recorded it since then.

How this was measured

When

Every morning, 9 September to 4 October 2026, at a fixed hour, by a scheduled task that writes its figures to a stored daily row: 24 readings in all, 22 of them recording OpenAI's search crawler by name or by an OpenAI total of zero.

Instrument

Cloudflare's AI Crawl Control on our own zone, read each morning, with every crawler recorded by name, including the ones at zero. The raw rows behind every figure on this page are saved in the site's repository, beside a file that lists the requests we took out and why, and the page is rebuilt from them.

Window

Most readings record the last 24 hours. The 6 readings of 19 to 24 September 2026 recorded the last seven days instead, and from 26 September 2026 on, 8 readings have recorded both windows side by side. Week figures here come from the latest seven-day reading whose window does not include the hour of fake crawler traffic described below. The ClaudeBot counts are 24-hour counts. A rolling window cannot be added up across days, so no figure on this page is a sum over readings.

Names

The dashboard files each request under the crawler name the visitor gives, so a name in the log is a claim. We checked the one reading that counted OAI-SearchBot against Cloudflare's security log, which shows where each request came from. It failed, and its 29 requests are left out of every figure here.

Configuration

Checked by reading it, not assumed: the zone's policies for search, agent and training crawlers all read Allow; robots.txt carries one group for every agent, disallows only utility paths and blocks no page in the sitemap; bot-fighting features that can challenge crawlers are off. Nothing in our settings refuses any crawler. The dashboard counts every answer in the 4xx range as an unsuccessful request, and on this site nearly all of those were 404, page not found, for addresses that do not exist here, and a 404 is not a refusal.

The retrieval check

We asked the same 3 questions in a logged-in ChatGPT account, first on 21 September 2026, then twice a week, one per fresh conversation, web search on, no follow-ups. The answers to two of them exist only on our pages. Each answer was compared with the live page before it was called correct.

What the logs contain

In the seven days to 2 October 2026, AI crawlers made 435 requests to this site. Googlebot made 182. BingBot made 147. GPTBot made 7. Meta-ExternalAgent made 86. ChatGPT-User, which fetches a page because a person's question needed it, made 9. OAI-SearchBot made none.

Two of those readers moved in ways worth naming. Meta-ExternalAgent made 16 requests in the seven days to 19 September 2026, 353 at its highest, in the seven days to 26 September 2026, and 86 in the seven days to 2 October 2026. We hold no instrument yet for what Meta's assistant does with what it reads.

ClaudeBot stopped altogether. It made 98, 104 and 18 requests a day in its first readings, while the site was new, and zero in each of the 21 readings that recorded it since then. We do not know why. A training crawler that has taken what it wanted would look like this, and so would one that moved a small site down its list.

The crawler that is supposed to matter most has never come

OpenAI documents three agents with different jobs. GPTBot gathers content that may be used to train its models. ChatGPT-User fetches a page when a user's question calls for it. OAI-SearchBot is the one OpenAI says is used to surface websites in ChatGPT's search features, and its help pages tell site owners who want to appear there to allow it.

In 22 daily readings, 9 September to 4 October 2026, it did not make a single request to this site. Not one that was allowed, and not one that was refused.

One morning it looked as if it had. The reading of 3 October 2026 counted 29 requests from OAI-SearchBot, the first in the whole series. All of them came in one hour on 2 October 2026. In that hour a single address in the Netherlands sent requests under the names of at least 9 different crawlers, OpenAI's search crawler and ChatGPT-User among them, and asked for files such as /credentials.json and /api/.env, where careless servers keep passwords and keys. That was not OpenAI. Anyone can put a crawler's name on a request, and the dashboard filed these under the names they gave. We read them as OpenAI's first visit for a day before we checked the address. They are left out of every figure on this page.

We spent days treating the crawler's absence as a problem to solve, and there was nothing to solve. OpenAI offers no way to submit an address: nothing like Google's Request Indexing, and IndexNow does not reach it. Every setting on our side was already open, and we checked each one by reading it.

So we asked the assistant instead

A crawler log answers whether a robot fetched a page. It does not answer the question underneath, which is whether an assistant can find the page when someone asks. A zero in the log fits a site that is missing from the index, and it fits a site that is in it.

So on 21 September 2026 we asked ChatGPT 3 questions whose answers a model cannot guess. Asked what our research says about how stable AI recommendations are, it came back with our ten-minute overlap finding and our finding that no company was named in all three places we asked from, both matching what our pages say. Each is a measurement we published and nobody else has. It linked the site. In all, 2 of the 3 questions were answered from our pages.

One question failed in the first readings, and the reason was ours rather than the assistant's. The page that holds the answer published its business names as data that a browser paints in after the page loads, not as text, so there was nothing on the page for a search to find. That was a defect on our side; we recorded it as one, and that page now carries the names as text.

We have taken the check 4 times so far, with the same questions. In 3 of those readings the assistant answered at least one of them from our pages; in 1 it answered none, so retrieval by search is itself a draw that can fail. That is enough to answer the question that was asked: the index held us, and the crawler with search in its name had never called.

What to do when somebody shows you a crawler count

Crawler counts are easy to collect and easy to sell. They are the input, not the outcome: they say a robot fetched something, not that a buyer was given your name. A vendor who shows you that OpenAI's search crawler is not visiting, and then sells you work to make it visit, is selling a lever nobody outside OpenAI holds.

The opposite mistake is ours, and we made it first: we read our own zero as proof that we could not be found, and it was not that either. OpenAI's own crawler page says that when a site allows both of its crawlers, it may use the results of one crawl for both uses. Our readings are what a site looks like when that happens, though we cannot see inside OpenAI to confirm it.

Check who it was, too. A crawler's name in a report is whatever the visitor called itself. OpenAI publishes the address ranges its crawlers use, and a request from anywhere else is not OpenAI, whatever name it gives.

The honest test takes five minutes and needs nothing beyond the assistant itself: ask it something that is true only on your pages, and see whether it comes back with your words.

What this means if you are buying

• A crawler that never visits is not proof that an assistant cannot find you, and a crawler that visits daily is not proof that it can. Neither number is an outcome, and a name in the log is not proof of who came.
• To know whether an assistant can retrieve your site, ask it a question whose answer exists only on your pages. A correct answer is proof; a plausible answer that does not match your page is not.
• Publishing something an assistant cannot read as text is close to not publishing it. Names, figures and lists that a browser paints in after the page loads can be invisible to an assistant, as they were to ChatGPT here.

Honest limits of this measurement

• One site, one zone, 24 readings. A site with a different age, history or link profile may see a different set of crawlers entirely.
• The per-crawler counts are rolling seven-day figures read at one moment each morning, so two readings are not two separate weeks and nothing here is summed across days.
• We checked where requests came from only for the one reading that counted OAI-SearchBot. Every other crawler's figures are taken as the dashboard names them, except that the week the impostor touched is left out.
• The retrieval check is 3 questions in one account, asked 4 times. Our own research measures that a single reading of an assistant is one draw, not a census, so repeated readings are what give it weight, and a success is stronger evidence than a failure would have been.
• We cannot see inside OpenAI's index. That one crawl served both uses is the reading most consistent with the log and the retrieval, and it is an inference, not something we measured directly.

Questions about this research

We run this measurement for a living. If you want it run on your own category and your own city, the free scan is the same instrument on one question, and the report is the full set. Nothing here is behind a form.

More measurements from the same instrument

• Ask an assistant for an electrician in Austin and it hands you a list of…
• People ask about AI visibility on Google, not inside ChatGPT
• All eleven measurements →

Every category we measure has its own page, with the questions customers put to an assistant in that category and what a reading costs. See all industries we measure →

Published 25 September 2026, updated 4 October 2026. Every figure on this page is recomputed from raw observation files by a single script, so any of them can be traced back to the readings behind it. If you are named here and believe a reading is wrong, tell us: we will run the measurement again and publish what it returns, whichever way it goes.