GPTBot

GPTBot is OpenAI's web crawler that gathers training and live-browsing data for ChatGPT.

GPTBot is OpenAI's user-agent for crawling the public web. There are two related agents: GPTBot (training) and OAI-SearchBot (live browse-tool retrieval). Allowing both in robots.txt is a near-universal recommendation for brands wanting AI visibility. Disallowing GPTBot — common in 2024 across publishers worried about training data — actively excludes you from one of the highest-traffic AI surfaces. Verify allow status with `curl -A 'GPTBot' https://yourdomain.com/robots.txt`.

Related terms

ChatGPT

ChatGPT is OpenAI's consumer AI engine — the largest single AI surface, with hybrid retrieval-and-recall sourcing.

llms.txt

llms.txt is an emerging standard file at the domain root that declares canonical content sources for AI engines.

See whether your brand shows up in AI search.

Free for 14 days. No credit card. Full Pro features.

Start free