# robots.txt Rule Check (RFC 9309)

> Say whether a crawler may fetch a path under a robots.txt text, by RFC 9309: allowed or disallowed, the deciding rule and its line, the group used, and every line the protocol does not accept, with a signed record.

URL: https://llpro.ai/robots-txt-check
State: testing

Part of: Home > Agents > Tools

---
MCP: https://llpro.ai/mcp | llms.txt: https://llpro.ai/llms.txt
