whosthatbot.com

tools/call with name check_robots checks a pasted robots.txt and a path. list_crawlers and get_crawler return cards. Each fact was checked against the vendor page in source.url on source.retrieved. check_robots applies RFC 9309 to a robots.txt the caller pastes and a path. It does not fetch any host. Do not send a URL, a host, or a fetch flag. A disallow is the site owner's request, not legal advice. Read stance_note. ChatGPT-User can be disallow under the RFC while the vendor says robots.txt may not apply. Perplexity-User can be disallow under the RFC while Perplexity says the fetcher generally ignores robots.txt. PerplexityBot's stance is a recommendation to allow the token so the site appears in search. That is not a claim that a Disallow removes the site. Google-CloudVertexBot is not merged with Googlebot. Applebot's RFC group is not replaced by Googlebot. Apple says it follows a Googlebot group only when Applebot is not mentioned, and that it does not follow crawl-delay. CCBot's page says User-agent: CCBot and Disallow: / prevents the crawl. It also says some crawlers falsely use that name. A separate post says the Opt-Out Registry is now the Opt-Out Ledger of legal requests. The checker reports that ledger and does not apply it to the RFC decision. suggest_robots writes a robots.txt from these cards for the kinds you allow or block. Each group cites source_url. effect request_only means the vendor does not say the crawler obeys that line. control_token is not training. The file has no User-agent: * group. It does not fetch any host.