SchemaStatsBot
SchemaStatsBot
The crawler behind the Schema.org validator on Schema Stats. It only fetches URLs that a visitor explicitly submits.
What it does
SchemaStatsBot downloads a single page when a user runs the validator or compare tool. It extracts JSON-LD, Microdata and RDFa to show structured data alongside Schema.org usage statistics. It is not used to build search indexes or to crawl your site on its own.
Why it is a good bot
- It runs only after a human submits a URL in the validator — never autonomously across the web.
- It checks robots.txt before each fetch and stops if your rules disallow SchemaStatsBot.
- It uses a clear, stable User-Agent string that links back to this page.
- Requests are rate-limited and capped (timeouts, size limits, no private-network access).
User-Agent
Use this string to identify our crawler in logs, WAF rules or robots.txt:
SchemaStatsBot/1.0 (+https://schemastats.com/bot)
How to block it
You can disallow SchemaStatsBot for all or part of your site. If blocked, the validator will show a robots.txt message instead of fetching the page.
Example robots.txt
User-agent: SchemaStatsBot Disallow: /