INDIVIDUAL MCP TOOL

scan_site

Run a live AI-accessibility audit of a domain: robots.

crawlcensus.comnone authenticationAvailability not checked

LIVE ENDPOINT

https://crawlcensus.com/mcp

No auth detected

Connect to this endpoint to inspect the live schema for scan_site and invoke it with your own arguments.

{}

Inferred read-only · medium confidence · heuristic, not a guarantee.

No write-capable action terms were found; this is not proof that invocation has no side effects.

CONNECT WITH APPROVAL

Review this server and its permissions before adding it. Secret placeholders must be set locally.

Codex

~/.codex/config.toml

[mcp_servers.crawl-census]
url = "https://crawlcensus.com/mcp"
enabled = true

Claude Code

.mcp.json

{
  "mcpServers": {
    "crawl-census": {
      "type": "http",
      "url": "https://crawlcensus.com/mcp"
    }
  }
}

Claude Desktop

Settings → Connectors → Add custom connector

Name: crawl-census
Remote MCP URL: https://crawlcensus.com/mcp

Add this remote URL as a custom connector in Claude Desktop. Availability depends on the user plan and workspace policy.

Cursor

.cursor/mcp.json

{
  "mcpServers": {
    "crawl-census": {
      "url": "https://crawlcensus.com/mcp"
    }
  }
}

Visual Studio Code

.vscode/mcp.json

Add to Visual Studio Code

{
  "servers": {
    "crawl-census": {
      "type": "http",
      "url": "https://crawlcensus.com/mcp"
    }
  }
}

Generic MCP

Client-specific MCP configuration

{
  "name": "crawl-census",
  "transport": "streamable-http",
  "url": "https://crawlcensus.com/mcp"
}

MCP Inspector

Run the official MCP Inspector locally and enter the indexed Streamable HTTP endpoint.

site_report — Return the most recent stored audit for a domain without triggering a new scan.
census_stats — Corpus-level statistics: how many measured domains block each AI crawler, mean access score, llms.
crawl_preflight — Decide whether a crawler may fetch a list of domains before spending requests on them.
agent_profile — What this census measures and publishes about one AI crawler: how often it is disallowed in robots.
census_facts — Every headline finding from the census as discrete, dated records rather than prose.