---
title: "PerplexityBot: user agent, robots.txt rules and IP verification"
description: PerplexityBot indexes pages for Perplexity search. Its user agent, robots.txt rules, published IPs, and how Perplexity-User handles robots.txt.
canonical_url: https://rebilder.com/learn/perplexitybot
last_updated: "2026-11-15"
---

# [PerplexityBot: user agent, robots.txt rules and IP verification](https://rebilder.com/learn/perplexitybot)

> PerplexityBot indexes pages for Perplexity search. Its user agent, robots.txt rules, published IPs, and how Perplexity-User handles robots.txt.

- **Updated:** 2026-11-15
- **Author:** Rebilder
- **Published:** 2026-11-15

**PerplexityBot is Perplexity’s web crawler: it indexes pages so Perplexity can surface and link them in its search answers.** Perplexity states that it is not used to crawl content for AI foundation models. It identifies itself with the `PerplexityBot` user-agent token, follows `robots.txt`, and sends requests from a published list of IP addresses.

It has a sibling, `Perplexity-User`, which fetches a page when a person asks Perplexity a question, and which Perplexity says generally ignores `robots.txt` because a user requested the fetch. This guide covers both: the exact user-agent strings, the `robots.txt` rules for each, how to verify the traffic against Perplexity’s IP lists, and how to block Perplexity-User at your firewall when `robots.txt` is not enough. For other AI user agents, see the [AI agent user agents reference](/learn/ai-agent-user-agents).

## What is PerplexityBot?

Perplexity is an answer engine: it responds to questions with a written answer and links to the pages it drew on. PerplexityBot builds the index behind those answers. Perplexity’s [crawler documentation](https://docs.perplexity.ai/docs/resources/perplexity-crawlers) describes it as “designed to surface and link websites in search results on Perplexity” and recommends allowing it in `robots.txt`, and permitting requests from its published IP ranges, if you want your site to appear in those results.

- **It indexes for search.** PerplexityBot is how your pages become candidates for a Perplexity answer and its citation links.
- **It is not a training crawler.** Perplexity states that PerplexityBot is not used to crawl content for AI foundation models.
- **It follows `robots.txt`.** A `User-agent: PerplexityBot` group controls it, and each Perplexity setting works independently of the other.
- **Changes take time to apply.** Perplexity says its systems can take up to a day to reflect a `robots.txt` change.

That makes the decision different from Anthropic’s or OpenAI’s training crawlers. Blocking [ClaudeBot](/learn/claudebot) or [GPTBot](/solutions/gptbot) is a training opt-out. Blocking PerplexityBot is a search opt-out: it removes your pages from the index Perplexity answers from.

## PerplexityBot user agent string

Perplexity publishes an example string for each bot and notes that the version number may change:

Perplexity’s published user-agent examples

```http
# PerplexityBot (search indexing)
User-Agent: Mozilla/5.0 (compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)

# Perplexity-User (fetch for a user’s question)
User-Agent: Mozilla/5.0 (compatible; Perplexity-User/1.0; +https://perplexity.ai/perplexity-user)
```

You will also see a longer, browser-shaped form. Rebilder’s detector fixtures use this shape, with the `compatible; ...` clause inside the `(KHTML, like Gecko; ...)` parenthetical:

Longer form, as in Rebilder’s detector fixtures

```http
User-Agent: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)
User-Agent: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Perplexity-User/1.0; +https://perplexity.ai/perplexity-user)
```

Both forms carry the same token, which is what to match. Search case-insensitively for `PerplexityBot` or `Perplexity-User` anywhere in the string, and never pin the version or the surrounding layout. A regex written for one shape silently misses the other.

## PerplexityBot vs Perplexity-User

The two bots do different jobs, follow different rules and arrive from different IP lists:

| Bot | Role | Honors robots.txt | robots.txt token | Published IP list |
| --- | --- | --- | --- | --- |
| **PerplexityBot** | Crawls and indexes pages so Perplexity can surface and link them in search answers. Not used for AI foundation model training. | Yes | `PerplexityBot` | `perplexitybot.json` |
| **Perplexity-User** | Visits a page when a user’s question needs it, and may link the page in the answer. Not used for crawling or training. | Generally no: Perplexity says it generally ignores `robots.txt` because a user requested the fetch. | `Perplexity-User` | `perplexity-user.json` |

The split matches the one other AI platforms draw between a crawler that builds an index on its own schedule and a fetcher acting on a live request. What is unusual is the second column. Perplexity’s documentation says Perplexity-User “controls which sites these user requests can access” and, in the same entry, that it generally ignores `robots.txt`. Treat a `robots.txt` rule for Perplexity-User as a stated preference, and enforce a block at your firewall if you need one.

## How to block PerplexityBot in robots.txt

PerplexityBot reads a group that names it. To remove your site from Perplexity’s index:

robots.txt: block PerplexityBot

```text
User-agent: PerplexityBot
Disallow: /
```

To keep Perplexity out of private flows while leaving the pages you want cited readable, disallow paths instead of the whole site:

robots.txt: block private paths, allow the rest

```text
User-agent: PerplexityBot
Disallow: /account/
Disallow: /checkout/
Disallow: /cart
Allow: /
```

To state that you want neither bot on the site, name both. PerplexityBot will follow the rule; Perplexity-User generally will not, so pair this with the firewall rule in the next section:

robots.txt: decline both Perplexity bots

```text
User-agent: PerplexityBot
Disallow: /

User-agent: Perplexity-User
Disallow: /
```

> **A named group replaces your wildcard rules** Once PerplexityBot finds a group with its own name, it stops reading your `User-agent: *` rules. An `Allow: /` group for PerplexityBot therefore also lifts any wildcard `Disallow` lines, such as `/admin/` or `/cart`. Repeat those lines inside the PerplexityBot group.

After deploying, fetch `/robots.txt` from outside your network and confirm it returns plain text, not a challenge page. Allow up to a day for Perplexity to pick up the change.

## Does Perplexity-User honor robots.txt? Blocking it at the firewall

Perplexity’s documentation says that, since a user requested the fetch, Perplexity-User generally ignores `robots.txt` rules. If you need it kept out, enforce that at the edge by matching the user-agent token together with Perplexity’s published IP list. Perplexity documents this pattern for allow rules; the same expression with a block action denies the traffic instead.

Cloudflare custom rule expression (from Perplexity’s documentation)

```text
(http.user_agent contains "PerplexityBot" and ip.src in {<PerplexityBot IP ranges>})
or (http.user_agent contains "Perplexity-User" and ip.src in {<Perplexity-User IP ranges>})
```

1. **Load the current ranges.** Replace each placeholder with the prefixes from the matching JSON file, separated by spaces. On paid Cloudflare plans you can store each list as an IP list and reference it, so updates touch the list, not the rule.
2. **Choose the action.** Use **Block** to deny Perplexity, or **Skip** to let it past your other bot rules, which is what Perplexity recommends if you want to be indexed.
3. **Order the rule.** A Skip rule only bypasses rules that run after it, so place it above any rule that blocks or challenges. Cloudflare’s Bot Fight Mode cannot be skipped by a custom rule and may still challenge Perplexity’s bots.
4. **On AWS WAF**, create an IP set per bot and a rule that ANDs the IP set match with a `User-Agent` string match on the token.
5. **Refresh on a schedule.** Perplexity updates the ranges regularly and recommends fetching them automatically.

Matching the token alone also catches impostors, which you may want anyway. Matching the token **and** the IP list targets Perplexity’s genuine traffic precisely, which is what you want when you allow it.

## PerplexityBot IP addresses: how to verify a request

Anyone can send `PerplexityBot` in a user agent. Perplexity publishes the source addresses for each bot as JSON, which is how you tell real traffic from a scraper borrowing the name:

Fetch Perplexity’s published IP lists

```bash
curl -s https://www.perplexity.ai/perplexitybot.json | jq -r '.prefixes[].ipv4Prefix'
curl -s https://www.perplexity.ai/perplexity-user.json | jq -r '.prefixes[].ipv4Prefix'

# Each file has the shape:
# { "creationTime": "...", "prefixes": [ { "ipv4Prefix": "x.x.x.x/32" }, ... ] }
```

- **Check the right list for the token.** A `PerplexityBot` request should come from `perplexitybot.json`; a `Perplexity-User` request from `perplexity-user.json`.
- **A claimed PerplexityBot outside the list is not Perplexity.** Handle it as any unidentified scraper.
- **IP lists, not reverse DNS.** Perplexity documents the JSON endpoints as the source of truth. It does not document a reverse-DNS hostname pattern or Web Bot Auth request signatures for either bot.

Verification matters here more than for most bots. Cloudflare has [published research](https://blog.cloudflare.com/perplexity-is-using-stealth-undeclared-crawlers-to-evade-website-no-crawl-directives/) reporting undeclared crawling attributed to Perplexity from addresses outside its published ranges, and removed Perplexity from its verified bot program; Perplexity disputed the findings. Whichever account you accept, the practical step is the same: decide on the declared bots using their tokens and IP lists, and treat everything else as unidentified traffic.

## PerplexityBot crawl rate and Crawl-delay

Perplexity’s crawler documentation does not mention `Crawl-delay`. You can add one, and it does no harm, but don’t count on it. To cap request rates, use a rate-limiting rule at your CDN or server keyed on the `PerplexityBot` token and its IP list:

robots.txt: an undocumented hint for PerplexityBot

```text
User-agent: PerplexityBot
Crawl-delay: 5
Allow: /
```

A rate limit that answers with a `Retry-After` header tells a well-behaved client when to come back, without removing you from the index, which is usually a better outcome than a block.

## What happens if you block PerplexityBot

- **Block PerplexityBot.** Your pages drop out of the index Perplexity answers from, so they are much less likely to appear or be linked in its answers. Because PerplexityBot is not a training crawler, blocking it does not change whether your content trains foundation models.
- **Block Perplexity-User at the firewall.** When a user asks about your page or pastes your URL, Perplexity cannot read it, and the answer is written without it.
- **Block both.** Neither of Perplexity’s declared bots can read your site.
- **Allow both.** Your pages can be indexed, cited and read on demand. Make sure what they read is accurate and complete.

For a business whose pages exist to be found, such as a store, a practice or a services firm, allowing both is usually the right call. A publisher whose product is the text itself may weigh it differently.

## Does PerplexityBot render JavaScript?

Perplexity’s documentation is silent on JavaScript execution for both bots. Perplexity quotes and cites the text it retrieves, so a figure assembled by a script after load may never reach the answer. Make sure the response body itself carries the details a buyer compares: the price, stock, delivery terms and return window. A plain request shows what arrives:

See the raw HTML a crawler receives

```bash
curl -s -A "Mozilla/5.0 (compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)" \
  https://your-site.example/products/example | grep -i "price"
```

## How to find PerplexityBot in your logs and CDN

Count Perplexity requests in an access log

```bash
# Requests per Perplexity token
grep -ioE "perplexitybot|perplexity-user" /var/log/nginx/access.log \
  | tr "[:upper:]" "[:lower:]" | sort | uniq -c

# Most-fetched paths (combined log format: path is field 7)
grep -i "perplexitybot" /var/log/nginx/access.log | awk '{print $7}' | sort | uniq -c | sort -rn | head -20

# Source IPs claiming PerplexityBot, to compare against perplexitybot.json
grep -i "perplexitybot" /var/log/nginx/access.log | awk '{print $1}' | sort -u
```

- **Cloudflare.** The AI Crawl Control dashboard shows Perplexity’s declared bots next to every other AI crawler, with a per-crawler allow or block switch. The custom rule above adds the IP check the switch does not.
- **Vercel.** Run the AI Bots managed ruleset in log mode for a week before denying anything, so you can see how often Perplexity-User arrives compared with PerplexityBot.
- **Other hosts.** Any CDN or load-balancer log with user agent and source IP works with the same commands.

Check your firewall and your `robots.txt` together. A managed bot rule that challenges every AI user agent blocks PerplexityBot even when `robots.txt` allows it.

## Serve Perplexity a clean answer with Rebilder

Perplexity answers with citations, so the page it fetches becomes the source a user sees next to the answer. Rebilder puts a clean version of that source in front of it: the gateway answers Perplexity’s requests with markdown built from your product, service and policy records, while people and search engines load the usual HTML from the same address.

- **Detection.** The open-source detector, `@rebilder/agent-detect`, matches both `PerplexityBot` and `Perplexity-User` to the `perplexity` platform at medium confidence. Its fixtures include both bots; they send browser-like `Accept: text/html` headers, so the user-agent token is the signal that identifies them. See [Classification](/docs/classification).
- **Serving.** Where one of your sources covers the requested URL, Perplexity receives the markdown version. Pages without a source keep serving their standard HTML, so nothing breaks while you connect more of the site.
- **Access rules.** Use the [access policy](/docs/access-control) to throttle or refuse Perplexity at the gateway. Platform-named rules such as `perplexity` act only on Web Bot Auth-verified requests, and Perplexity does not document signatures, so use the `unverified` or `*` subjects for its user-agent-only traffic.
- **Reporting.** With an event sink connected, the Console lists Perplexity’s requests under the `perplexity` platform and records whether each one got markdown or HTML. See [Which agents Rebilder detects](/help/agent-traffic/which-agents-detected).

Identity checks stay at your edge: the detector works from headers, so pair it with the Perplexity IP-list rule wherever a request must be proven genuine.

## What is PerplexityBot?

PerplexityBot is Perplexity’s web crawler. It indexes pages so Perplexity can surface and link them in search answers, and Perplexity states it is not used to crawl content for AI foundation models. It identifies itself with the `PerplexityBot` user-agent token and follows `robots.txt`.

## What is the PerplexityBot user agent?

Perplexity’s published example is `Mozilla/5.0 (compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)`, and the version may change. A longer browser-shaped form with the same token also appears. Match the `PerplexityBot` token, not the full string.

## How do I block PerplexityBot?

Add `User-agent: PerplexityBot` followed by `Disallow: /` to your `robots.txt`. Perplexity says changes can take up to a day to apply. Blocking it removes your pages from Perplexity’s search index.

## Does Perplexity-User respect robots.txt?

Generally no. Perplexity’s documentation says that because a user requested the fetch, Perplexity-User generally ignores `robots.txt` rules. To block it, use a firewall rule that matches the `Perplexity-User` token and Perplexity’s published IP list.

## What IP addresses does PerplexityBot use?

Perplexity publishes them at `https://www.perplexity.ai/perplexitybot.json`, and Perplexity-User’s at `https://www.perplexity.ai/perplexity-user.json`. The lists change regularly, so fetch them on a schedule rather than copying addresses by hand.

## Does blocking PerplexityBot stop AI training on my content?

No. Perplexity states that PerplexityBot is not used to crawl content for AI foundation models, so blocking it is a search opt-out, not a training opt-out. Training opt-outs belong to crawlers such as ClaudeBot and GPTBot.

## Does PerplexityBot support Crawl-delay?

Perplexity’s documentation does not mention `Crawl-delay`. Add it if you like, but rely on a rate-limiting rule at your CDN or server, keyed on the token and IP list, to cap request rates.

## Related

- [AI agent user agents: the full reference](https://rebilder.com/learn/ai-agent-user-agents)
- [ClaudeBot: user agent, robots.txt and blocking](https://rebilder.com/learn/claudebot)
- [How agent detection works](https://rebilder.com/docs/classification)
- [Allow, deny or rate-limit agents](https://rebilder.com/docs/access-control)
- [Which agents Rebilder detects](https://rebilder.com/help/agent-traffic/which-agents-detected)
- [Gateway quickstart](https://rebilder.com/docs/quickstart)
- [Free AI robots.txt generator](https://rebilder.com/tools/robots-txt-generator)