Robots.txt tester
Paste the file, name the crawler, and enter a URL or a path. The page reports allowed or blocked, and which rule matched.
- Longest matching rule
- Free, no sign-up
Free to use, no account. More seo tools
How to use it
- Paste robots.txt.
- Set the crawler name. Googlebot is the default.
- Enter a full URL or a path such as /private.
Which rule wins
The page picks the user-agent group whose name appears in the crawler name, preferring the longest name. If none match, it uses *. Empty Disallow and empty Allow lines do not match a path. Among rules that match, the longer pattern wins. If two patterns are the same length, Allow wins.
Wildcards use * for any characters and a trailing $ for the end of the path. The page does not fetch the live robots.txt. Paste the file you want checked.
Private by design
Privacy
The file and the URL stay in this tab and are not uploaded.
Questions
Why is /private/ok allowed when /private is blocked?
The longer Allow pattern wins over the shorter Disallow.
Does this ask Google?
No. It applies the rules in the box. A live index can differ.
Are comments kept?
Lines after # are ignored, which matches the usual robots.txt reading.