robots.txt Generator
Generate a robots.txt file quickly.
User-agent
Allow paths (one per line)
Disallow paths (one per line)
Crawl-delay (optional)
Sitemap URL (optional)
robots.txt
Tip: rules are evaluated from most specific to least specific by many crawlers.
Related tools
sitemap.xml GeneratorGenerator
Generate a sitemap.xml for a list of URLs.
Open Graph PreviewValidator
Preview how a page looks when shared on Facebook, Twitter, LinkedIn, and Discord.
robots.txt TesterValidator
Test robots.txt rules against URL paths and user-agents.
UUID GeneratorGenerator
Generate UUIDv4 locally in your browser.
UUIDv7 GeneratorGenerator
Generate time-ordered UUIDv7 values locally in your browser.
NanoID GeneratorGenerator
Generate NanoID identifiers locally in your browser.