INDIVIDUAL MCP TOOL

map

List a site's URLs without fetching each page: the start URL, the links on its page (read on the http lane alone; no browser) and the entries of the sitemaps the site declares (robots.

octocrawl.devnone authenticationAvailability not checked

LIVE ENDPOINT

https://mcp.octocrawl.dev/mcp

No auth detected

Connect to this endpoint to inspect the live schema for map and invoke it with your own arguments.

Indexed input schema

{}

Risk classification

Inferred read-only · medium confidence · heuristic, not a guarantee.

A tool name or description suggests retrieving external content.

Parent server

octocrawl.dev

CONNECT WITH APPROVAL

Client installation

Review this server and its permissions before adding it. Secret placeholders must be set locally.

Codex

~/.codex/config.toml

[mcp_servers.octocrawl]
url = "https://mcp.octocrawl.dev/mcp"
enabled = true

Claude Code

.mcp.json

{
  "mcpServers": {
    "octocrawl": {
      "type": "http",
      "url": "https://mcp.octocrawl.dev/mcp"
    }
  }
}

Claude Desktop

Settings → Connectors → Add custom connector

Name: octocrawl
Remote MCP URL: https://mcp.octocrawl.dev/mcp

Add this remote URL as a custom connector in Claude Desktop. Availability depends on the user plan and workspace policy.

Cursor

.cursor/mcp.json

{
  "mcpServers": {
    "octocrawl": {
      "url": "https://mcp.octocrawl.dev/mcp"
    }
  }
}

Visual Studio Code

.vscode/mcp.json

Add to Visual Studio Code

{
  "servers": {
    "octocrawl": {
      "type": "http",
      "url": "https://mcp.octocrawl.dev/mcp"
    }
  }
}

Generic MCP

Client-specific MCP configuration

{
  "name": "octocrawl",
  "transport": "streamable-http",
  "url": "https://mcp.octocrawl.dev/mcp"
}

MCP Inspector

Run the official MCP Inspector locally and enter the indexed Streamable HTTP endpoint.

Related tools

scrape_product — Get evidence-backed JSON for one anonymous Amazon.
scrape — Fetch one URL through the Octocrawl coverage ladder.