all toolsRebupl · 03 · On-page checks

Robots.txt Generatorrobots.txt builder

Build a clean robots.txt with sitemap URL, common AI-scraping blocks, and instant copy.

runs in your browser — nothing is sent
Datacenter originthe trailing /sitemap.xml is appended automatically
Block AI training crawlersyour call — indexation vs training
  • OpenAI training bots (GPTBot)
  • Common Crawl (CCBot)
  • Anthropic training bots (ClaudeBot)
robots.txtpaste at the origin root
# robots.txt — yoursite.com
User-agent: *
Allow: /
Run sheet
  1. 01Enter your origin; the sitemap line points at your /sitemap.xml.
  2. 02Review the AI-bot toggles before copying — they're explicit by default.
  3. 03Upload at the site root; verify it with the agent audit.
More in On-page checksall tools →
Meta & SERP Previewmeta budget + SERP previewLaunch Copy Counterwords, timings, budgets
Launch on Republfree · no sign-up · nothing stored