🤖 robots.txt / sitemap.xml Generator

robots.txt tells crawlers what they may and may not fetch; sitemap.xml tells search engines which pages are worth indexing. Generate both here and understand each rule.

robots.txt & sitemap.xml Generator

Interactively configure crawl rules and your sitemap, preview in real time, and copy / download with one click. Everything runs locally in your browser, with no network requests.

Quick presets:

Rule group #1

User-agent

Crawl-delay (optional, seconds)

Allow paths

None

Disallow paths

Sitemap URL (global, optional)

Generated robots.txt

User-agent: *
Disallow: /admin/

Sitemap: https://example.com/sitemap.xml

Where to put them

robots.txt must live at the site root (e.g. https://example.com/robots.txt). Put sitemap.xml at the root too, and reference it from robots.txt with a Sitemap: line so crawlers can find it.