# [AI agent user agents: a field guide to every string in your logs](https://rebilder.com/learn/ai-agent-user-agents)

> Real AI agent user agent strings (ChatGPT-User, GPTBot, claude-code, PerplexityBot), the exact headers each one sends, and how to detect them properly.

- **Updated:** 2026-08-20
- **Author:** Rebilder
- **Published:** 2026-08-05

This page is the living reference for AI agent user agents: ChatGPT and its crawlers, the Claude family, Perplexity, the terminal agents, and the search crawlers you must never mis-serve. Every full header set below is transcribed from our detection fixture corpus. Rows marked (UA marker) are detector patterns awaiting a first full capture, and rows marked (vendor-published) are transcribed from the vendor’s own crawler documentation rather than from a request we captured.

Two warnings before the tables. A user agent is a claim, not proof: anyone can send any string. And the strongest identification signals are usually in the other headers, not the UA.

## How to read AI agent user agents

The strings sort into four families, and the family matters more than the version number. It tells you why the request happened and how much intent sits behind it:

- **User-initiated fetchers** are sent while a human is mid-conversation with an assistant: `ChatGPT-User`, `Perplexity-User`. One fetch maps to one real person asking about your product right now.
- **Index and training crawlers** are bulk fetches building a corpus: `GPTBot`, `OAI-SearchBot`, `PerplexityBot`. No live user attached, but they decide whether the assistant knows your store exists.
- **Terminal agents** are CLI coding agents that fetch pages during a working session and ask for markdown outright: `claude-code`, `opencode`.
- **Search crawlers** are `Googlebot` and `bingbot`. Recognized by the same detector, but classified as crawlers, not agents, and always served canonical HTML.

> **A user agent is a claim** UA strings are trivially spoofable, so a UA match may only ever select a format transformation of the same substance: the same prices, availability, and policies in different packaging. Never gate anything substantive on an unverified string.

## OpenAI: ChatGPT-User, GPTBot, and OAI-SearchBot

Merchants searching for the ChatGPT shopping user agent are almost always looking for `ChatGPT-User`, the string that appears when a shopper asks ChatGPT about a product and the assistant fetches your page mid-conversation. The GPTBot user agent is a different animal: OpenAI’s bulk crawler, fetching broadly with `Accept: */*` and no live user attached. Three strings, three jobs:

| User-agent string | What it does | Verified? | How to detect it |
| --- | --- | --- | --- |
| `Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0; +https://openai.com/bot` | Fetches a page live while a ChatGPT user is asking about it. User-initiated browsing, not crawling. | Sometimes: our `ChatGPT-User/2.0` captures carry Web Bot Auth signature headers. Parsed on every request, and cryptographically verifiable against a key registry you supply. | Prefer `Signature-Agent: "https://chatgpt.com"` when present (high confidence); else the `ChatGPT-User` UA token (medium). |
| `Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.2; +https://openai.com/gptbot` | OpenAI’s bulk crawler. Broad fetches with `Accept: */*`, not tied to a live question. | No, UA-only identification. | `GPTBot` UA token, capped at medium confidence. Format decisions only. |
| `Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; OAI-SearchBot/1.0; +https://openai.com/searchbot` | OpenAI’s search fetcher, classified to the `chatgpt` platform like GPTBot. | No, UA-only identification. | `OAI-SearchBot` UA token, medium confidence. |

Why the first row matters most: OpenAI has shifted toward **discover in chat, transact on the merchant’s site**: the assistant shortlists, then the human lands on your store to buy. A `ChatGPT-User` fetch is the discover step happening in real time. Serve it a parseable page and you are in the shortlist; serve it 90KB of theme markup and you are trusting a parser to dig.

## Anthropic: the Claude user agent family

The Claude family member our corpus holds full captures for is `claude-code`, Anthropic’s terminal coding agent, and the most explicit requester in that corpus, because it does not make you guess: it sends an `Accept: text/markdown` media range alongside its CLI user agent. A captured request, verbatim:

Claude Code fetching a product page: captured headers

```http
GET /products/olive-oil-500ml HTTP/1.1
Host: store.example.com
User-Agent: claude-code/1.0.58 (external, cli)
Accept: text/markdown;q=1.0, text/html;q=0.8, text/plain;q=0.5, */*;q=0.1
Accept-Encoding: gzip, deflate, br
```

| User-agent string | What it does | Verified? | How to detect it |
| --- | --- | --- | --- |
| `claude-code/1.0.58 (external, cli)` | Claude Code fetching a page during a working session. | No signature, but it declares intent with `Accept: text/markdown`, a stronger signal than any UA. | Match the `Accept` header first (high confidence); the `claude-code` UA token then attributes the platform. |
| `claude-code/2.0.13 (external, cli)` | Same agent, next major version, captured against a shipping-policy page with `Accept: text/markdown, text/plain;q=0.8, */*;q=0.5`. | Same as above. | Same as above. Versions churn; match the token, never pin a version. |
| `ClaudeBot` · `Claude-User` · `claude-web` · `anthropic-ai` (UA markers) | Claude fetching pages for a user or for retrieval outside the CLI. | No, UA-only. | Substring match on any marker → `claude` platform, medium confidence. |

One corpus note: the last row is matched by our detector, but we do not yet hold full raw-header captures for those markers. When one lands in live traffic it becomes a fixture first (headers, URL, expected classification), and this page gets updated.

## Perplexity: PerplexityBot and Perplexity-User

The Perplexity user agent comes in two forms, and the split mirrors OpenAI’s: an index crawler that builds the corpus behind its answers, and a user-initiated fetcher that fires when a person asks something your page answers. Both classify as agent traffic on the `perplexity` platform:

| User-agent string | What it does | Verified? | How to detect it |
| --- | --- | --- | --- |
| `Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)` | Index crawler: the corpus behind Perplexity’s answers. | No, UA-only. | `PerplexityBot` UA token, medium confidence. |
| `Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Perplexity-User/1.0; +https://perplexity.ai/perplexity-user)` | User-initiated fetch: a live question is behind it. | No, UA-only. | `Perplexity-User` UA token, medium confidence. |

Two practitioner details from the captures. First, both send fully browser-like `Accept: text/html,...` headers, so nothing but the UA separates them from a human visitor, which is why UA matching sits last in the detection order. Second, note the parenthesis placement: unlike `ChatGPT-User`, the `compatible; ...` clause sits **inside** the `(KHTML, like Gecko; ...)` parenthetical. Regexes built on the ChatGPT shape silently miss Perplexity. Match tokens, not shapes.

## OpenCode, Googlebot, and bingbot

The rest of the corpus: a second markdown-requesting terminal agent, and the search crawlers every store must keep on canonical HTML. The three Microsoft rows are the ones merchants ask about after reading anything on Copilot, so they are listed in full.

| User-agent string | What it does | Verified? | How to detect it |
| --- | --- | --- | --- |
| `opencode/0.5.29` · `opencode/1.0.2 (linux; x64)` | OpenCode (SST), an open-source terminal coding agent. Sends `Accept: text/markdown` like Claude Code. | No signature; the `Accept` header carries the intent. | `Accept` header first; `opencode` UA token attributes the platform. |
| `Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)` | Google’s desktop search crawler. | Special-cased: always classified `crawler`, never `agent`. | `Googlebot` UA token → serve canonical HTML, no exceptions. |
| `Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.6422.175 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)` | Googlebot smartphone: a full Chrome-on-Android UA with the Googlebot clause at the end. | Same special case. | Match the `Googlebot` token **anywhere** in the string, not just as a prefix. |
| `Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)` (vendor-published) | Microsoft’s search crawler, and the path Copilot answers from. Copilot has no user agent of its own, so allowing bingbot is what keeps a site reachable by Copilot. | Same special case. | Substring match on `bingbot` → `crawler`; canonical HTML always. |
| `Mozilla/5.0 (compatible; adidxbot/2.0; +http://www.bing.com/bingbot.htm)` (vendor-published) | Bing Ads landing-page quality crawler. It audits ad destinations for policy compliance. | Same special case. | Serve it the canonical page. An ads-policy crawler seeing something a customer would not is the cloaking fact pattern in its purest form. |
| `Mozilla/5.0 (compatible; MicrosoftPreview/2.0; +https://aka.ms/MicrosoftPreview)` (vendor-published) | Link unfurling in Microsoft products: the preview card when someone pastes your URL into Teams or Outlook. | Same special case. | Its own platform, not `bingbot`. It never touched the search index, and counting it as Bing would inflate that figure. |

> **There is no Copilot user agent** Copilot rides Bing. Its fetches arrive as `bingbot`, and Microsoft’s guidance for being reachable by Copilot is to allow bingbot. Copilot appears as itself in exactly one place: the `Referer` header on a human arrival from `copilot.microsoft.com`. So the Console attributes Copilot on the arrivals side and `bingbot` on the fetch side.

> **The crawler guardrail** A known crawler UA always classifies as `crawler`, even when other signals fire. Our corpus deliberately includes a capture of a Googlebot UA sending `Accept: text/markdown`; it still classifies `crawler` and still receives canonical HTML. Serving a crawler a format it did not negotiate is how a store gets flagged for cloaking.

## Headers beat user agents: Accept and Web Bot Auth

`Accept: text/markdown` is ordinary HTTP content negotiation, and it outranks every UA check. It states what the requester wants instead of who it claims to be. Honor q-values (`q=0` means excluded) and treat an explicit `text/markdown` listing as a high-confidence agent signal, whatever the UA says.

The second header family is **Web Bot Auth**: three headers that let an agent platform cryptographically sign its requests. Here is a captured `ChatGPT-User/2.0` request carrying all three:

ChatGPT-User with Web Bot Auth headers, captured verbatim

```http
GET /products/espresso-grinder HTTP/1.1
Host: store.example.com
User-Agent: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/2.0; +https://openai.com/bot
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Signature-Agent: "https://chatgpt.com"
Signature-Input: sig1=("@authority" "signature-agent");created=1754265600;expires=1754266200;keyid="JrQLj5C_-uusgQwmSFvHYK2PxYlbcO9zRMoDHIQdVRo";tag="web-bot-auth"
Signature: sig1=:TWFrZSBhZ2VudHMgZmlyc3QtY2xhc3MgY2l0aXplbnMgb2YgdGhlIHdlYi4=:
```

- **`Signature-Agent`** is a quoted origin naming who signs (`"https://chatgpt.com"`). Present → platform attribution at high confidence.
- **`Signature-Input`** lists which request components are covered, plus metadata: `created`, `expires`, a `keyid`, and the `tag="web-bot-auth"` label.
- **`Signature`** carries the signature bytes themselves. A bare `Signature` + `Signature-Input` pair without `Signature-Agent` still means agent: unknown platform, medium confidence.

> **Parsed always, verified against keys you supply** Our gateway parses these headers on every request and can verify the signature chain cryptographically (RFC 9421 Ed25519) against a key registry you inject. The registry we ship is **empty on purpose**, because we will not invent production platform keys, so `verified` reads `false` until you populate it from the platforms’ published directories. Until you do, treat signature headers as strong attribution, not proof, and gate nothing security-sensitive on them.

- **~4.2x**: Accurate retrieval: content negotiation vs llms.txt
- **98.6%**: Payload reduction when markdown is served
- **~38%**: Conversion lift, AI-referred visitors vs search

## How to detect AI agents properly

Detection order matters more than detection coverage. One guardrail runs before everything, then the cheap, strong signals, then the weak ones. This is the exact classification order our gateway ships, pure compute, well under 1ms:

1. **Known crawler UA? Classify `crawler` and stop.** The guardrail runs before everything: a `Googlebot` or `bingbot` token always classifies as `crawler` and gets canonical HTML, even when every other signal fires.
2. **Then check `Accept: text/markdown`.** Explicitly listed with q > 0 → an agent asked for machine-readable output. High confidence, and the only check that carries intent rather than identity.
3. **Then Web Bot Auth headers.** `Signature-Agent` attributes the platform at high confidence; a bare `Signature` pair means agent, unknown platform, medium.
4. **Then protocol routes.** Requests to `/.well-known/ucp`, `/.well-known/acp`, `/mcp`, or `/acp/*` are protocol probes, worth logging today, even though our protocol endpoints are announced rather than live.
5. **UA heuristics last, capped at medium confidence.** Every string on this page, matched as a fallback. A UA match may select a format transformation of the same substance, never different prices, claims, or availability.
6. **Default: human.** A browserish `Accept: text/html` or `Mozilla/` UA → high confidence; empty or garbage headers → low.

You can implement this order yourself (headers first, UA last, crawlers pinned to canonical HTML) or install it: [Rebilder’s gateway](/docs/quickstart) ships this exact classifier free, with adapters for Next.js, Shopify, Express, Fastify, and Cloudflare Workers, and answers `Accept: text/markdown` with markdown rendered from your own product data. The full ordering is documented in [Classification](/docs/classification).

## A living corpus, not a static list

Every user agent on this page exists because a fixture exists: a captured sample carrying the raw headers, the URL, and the expected classification. The detection test suite loads every fixture and asserts every sample, so adding a newly observed agent to the corpus *is* adding a regression test. When a new string or signature shows up in live traffic, it gets captured first, matched second, and published here third. The updated date at the top of this page tracks the corpus, not copyedits.

The fastest way to see which of these strings are already hitting your store: the free Console records every classified request (agent, human, crawler, or protocol probe, which platform, and what was served) and its side-by-side preview shows exactly [what each requester receives](/help/agent-traffic/which-agents-detected).

## What user agent does ChatGPT use?

Three, for three jobs. `ChatGPT-User` fetches pages live while a user is asking (`Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0; +https://openai.com/bot`). `GPTBot` is OpenAI’s bulk crawler, and `OAI-SearchBot` is its search fetcher. All three classify to the same `chatgpt` platform; newer `ChatGPT-User` fetches also carry Web Bot Auth signature headers.

## What is the GPTBot user agent?

The full string in our captures is `Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.2; +https://openai.com/gptbot`. It is OpenAI’s bulk crawler: broad fetches with `Accept: */*`, no live user attached, identifiable only by the UA token.

## What user agent does Claude use?

The family member our fixture corpus holds full captures for is Claude Code, Anthropic’s terminal agent: `claude-code/1.0.58 (external, cli)` and later versions, always alongside an explicit `Accept: text/markdown` header. The wider Claude family is matched by the UA markers `ClaudeBot`, `Claude-User`, `claude-web`, and `anthropic-ai`.

## Can AI agent user agents be spoofed?

Trivially, since a UA is a self-reported string. That is why UA matching sits last in the detection order, capped at medium confidence, and only ever changes the format of a response, never its substance. Stronger signals exist: an explicit `Accept: text/markdown` header, and Web Bot Auth signatures, designed to be cryptographically verifiable.

## Should I block AI agents like GPTBot from my store?

For a merchant, blocking is usually self-harm: an assistant that cannot read your pages recommends the store it can read. AI-referred visitors convert ~38% higher than search visitors (Cyber Week 2025 retail data). The alternative to blocking is serving agents efficiently: in our reference capture, answering `Accept: text/markdown` cut the same product page from 91,226 bytes to 1,315, a 98.6% reduction.

## How do I see which AI agents are visiting my website?

Server-side request logs are the ground truth, because agent fetches run no JavaScript and analytics scripts miss them. Grep for the tokens on this page (`ChatGPT-User`, `GPTBot`, `claude-code`, `PerplexityBot`), or install Rebilder’s free gateway and read the Console visit log, which classifies every request by kind and platform automatically.

## Related

- [AI search optimization: the definitive guide](https://rebilder.com/learn/ai-search-optimization)
- [What is agentic commerce?](https://rebilder.com/learn/what-is-agentic-commerce)
- [How agent detection works](https://rebilder.com/docs/classification)
- [Which agents Rebilder detects](https://rebilder.com/help/agent-traffic/which-agents-detected)
- [Gateway quickstart](https://rebilder.com/docs/quickstart)
- [Free llms.txt generator](https://rebilder.com/tools/llms-txt-generator)