robots.txt Tester
Paste robots.txt content, enter a URL path and user-agent to test if it is allowed or disallowed.
robots.txt Content
Parsed Rules (10)
L1: User-agent: *
L2: Allow: /
L3: Disallow: /admin/
L4: Disallow: /private/*
L5: Disallow: /api/
L7: User-agent: Googlebot
L8: Allow: /api/public
L9: Disallow: /api/
L11: Crawl-delay: 10
L13: Sitemap: https://example.com/sitemap.xml
Test Configuration
ALLOWED
/api/public for Googlebot
Matching rule (line 8): Allow: /api/public
Crawl-delay: 10s
Quick Test Paths
Related tools
Open Graph PreviewValidator
Preview how a page looks when shared on Facebook, Twitter, LinkedIn, and Discord.
robots.txt GeneratorGenerator
Generate a robots.txt file quickly.
sitemap.xml GeneratorGenerator
Generate a sitemap.xml for a list of URLs.
Email Address ValidatorValidator
Validate email addresses per RFC 5322 with detailed breakdown and batch support.
Credit Card ValidatorValidator
Validate credit card numbers with Luhn algorithm and detect card type.
IBAN ValidatorValidator
Validate IBAN numbers, verify check digits, and show country and BBAN breakdown.