FetchGradeBot
The user agent our audits send is FetchGradeBot/1.0. Here is exactly what it fetches and what it will never do.
- What it is
- FetchGradeBot is the fetcher behind every FetchGrade audit. It loads the one public URL a user submits in an isolated, headless browser to check whether an AI agent can read and operate that page.
- What it fetches
- Only the single submitted URL, rendered once in a headless browser, plus a short sample of first-party links already present on that page — checked with lightweight requests solely for whether they respond. Nothing else is requested.
- What it never does
- It does not crawl your site — no following links beyond the sampled reachability check, no visiting other pages on your own initiative. It never submits forms, signs in, makes a purchase, types into a field, or attempts to bypass a CAPTCHA or other anti-bot challenge. A blocked page is reported as blocked, never guessed at.
- About robots.txt
- FetchGradeBot is not a crawler, so it does not fetch or evaluate robots.txt. It only ever loads the one page a person explicitly submits for audit — there is nothing to disallow because it never wanders beyond that page.
- Questions
- Reach us at privacy@fetchgrade.com.