In modern API and backend development, access to up-to-date information is essential—especially as AI assistants like Claude and ChatGPT become a daily tool for technical teams. But most AI models can’t directly reference real-time web content, often leading to outdated or incomplete answers.
What if you could instantly make any website, documentation portal, or private knowledge base accessible to your favorite LLM, all without changing the site’s code? That’s exactly what SiteMCP enables: transforming any site into a Model Context Protocol (MCP) server so your AI can fetch, search, and cite web pages on demand.
What Is SiteMCP and Why Does It Matter for Developers?
SiteMCP is an open-source utility created by ryoppippi that turns any website into an MCP server—a bridge allowing AI tools to read and reference web content programmatically.
Key Benefits for API and Backend Teams
- Universal Access: Make any documentation, internal wiki, or blog accessible to AI assistants—no native integration required.
- Efficient Token Usage: Instead of dumping entire sites into prompts (which is limited and costly), SiteMCP lets AIs fetch just what’s needed.
- Always Up-to-Date: Reference the latest docs, API changes, or support content without retraining your models.
- Custom Knowledge Bases: Easily expose your own resources, improving AI-driven support or internal bots.
任意のサイトをMCP Serverに変えちゃうヤバいやつ作ったよ！https://t.co/MTYkdu1jjO pic.twitter.com/xHkb05eWF1
— ryoppippi (@ryoppippi) April 7, 2025
Check out the SiteMCP Github repo to see the project and contribute.
Understanding Model Context Protocol (MCP)
Before diving into SiteMCP’s setup, it helps to know how MCP works:
- Model Context Protocol is a specification that lets AI assistants pull external data sources—websites, files, APIs—into their context window.
- It acts as an API bridge, so you can “ask” an AI to look up a page, fetch a file, or answer using external resources.
For API developers, this means you’re no longer limited by the static knowledge your LLM was trained on. Instead, you can provide dynamic, real-time context to your AI workflows.
How SiteMCP Solves Real Developer Problems
SiteMCP isn’t just about turning websites into data; it addresses real workflow pain points:
- No Native AI Integration Needed: Works with any static or dynamic site, even those without API endpoints.
- Selective Data Access: Fetch only relevant docs or sections, minimizing noise and token use.
- Always Current: Keep your AI’s context fresh, especially for fast-evolving technologies or internal docs.
- Private/Internal Resource Access: Expose internal portals (served locally or privately) to your AI tools, improving team knowledge sharing.
If your team uses Apidog for API design and testing, SiteMCP can expand your workflow by making all your API documentation instantly referenceable by AI—streamlining onboarding, debugging, and support.
Getting Started: Installing and Running SiteMCP
SiteMCP is designed for quick setup. Choose an installation method:
One-Off Usage
Use one of these commands in your terminal:
bunx sitemcp
npx sitemcp
pnpx sitemcp
Global Installation
For repeated use, install globally:
bun i -g sitemcp
npm i -g sitemcp
pnpm i -g sitemcp
Basic Usage Example
To convert a website into an MCP server:
sitemcp https://example.com
For large sites, control parallel fetches:
sitemcp https://example.com --concurrency 10
Advanced SiteMCP Configuration
Fine-tune your MCP server to your needs:
1. Tool Name Strategy
Control how the MCP tool is named for easier AI referencing:
- Domain as tool name:
sitemcp https://vite.dev -t domain - Subdomain as tool name:
sitemcp https://react-tweet.vercel.app/ -t subdomain - Pathname as tool name (default):
Result: indexOfVitePluginFavicons / getDocumentOfVitePluginFaviconssitemcp https://ryoppippi.github.io/vite-plugin-favicons/ -t pathname
2. Matching Specific Pages
Limit which sections or endpoints are exposed to the AI:
sitemcp https://vite.dev -m "/guide/**" "/blog/**"
This leverages micromatch for powerful pattern-based inclusion.
3. Custom Content Extraction
If automatic extraction misses important content, use a CSS selector:
sitemcp https://vite.dev --content-selector ".content"
SiteMCP Caching for Faster Performance
By default, SiteMCP caches fetched pages in ~/.cache/sitemcp. To guarantee fresh content every run, disable caching:
sitemcp https://example.com --no-cache
Integrating SiteMCP with MCP-Compatible Clients
The real power comes when you connect SiteMCP to AI tools like Claude Desktop:
Example: Claude Desktop Configuration
Add this snippet to your Claude Desktop config:
{
"mcpServers": {
"daisy-ui": {
"command": "npx",
"args": [
"-y",
"sitemcp",
"https://daisyui.com",
"-m",
"/components/**"
]
}
}
}
Now, Claude can access DaisyUI component docs live whenever you ask.
Practical Use Cases for API and Backend Teams
1. Live API Documentation for AI
Expose your favorite frameworks’ docs for accurate, up-to-date code suggestions:
{
"mcpServers": {
"svelte": {
"command": "npx",
"args": [
"-y",
"sitemcp@latest",
"https://svelte.dev",
"-m",
"/docs/**"
]
}
}
}
2. Personal or Team Knowledge Bases
Let your AI reference your technical blog or team wiki:
{
"mcpServers": {
"my-blog": {
"command": "npx",
"args": [
"-y",
"sitemcp@latest",
"https://yourblog.com"
]
}
}
}
3. Apidog Integration
If you maintain API documentation in Apidog, simply serve your docs and point SiteMCP at your Apidog docs URL. Now your AI assistant can answer questions and generate examples directly from your living API specs.
How SiteMCP Works: Technical Overview
SiteMCP uses a two-server setup:
- Index Server: Lists all available pages with titles and URLs for the AI to browse.
- Document Server: Delivers full content for each page on demand.
This staged approach lets the AI efficiently discover resources and only fetch what’s relevant—saving bandwidth and tokens, especially important for large API docs or wikis.
Long pages are paginated automatically to avoid overwhelming the AI’s context window.
Troubleshooting and Best Practices
Tool Name Length Limits
Some MCP clients restrict tool names to 64 characters. SiteMCP v0.3.0+ handles this, but update if you encounter issues.
JSONRPC Errors
If you see errors like:
{"jsonrpc":"2.0","id":XX,"error":{"code":-32601,"message":"Method not found"}}
Update to the latest SiteMCP version for better compatibility.
Optimizing for Large Sites
For huge documentation portals, always use the -m flag to filter which sections are exposed:
sitemcp https://large-docs-site.com -m "/api/**" "/guides/**"
Advanced Applications: Custom Knowledge Bases and Multiple Sources
Turn Local Docs into an AI Knowledge Base
- Serve your local docs:
npx serve ./my-docs - Start SiteMCP:
sitemcp http://localhost:3000
Combine Multiple Knowledge Sources
Configure your MCP client to point at several SiteMCP servers at once:
{
"mcpServers": {
"technical-docs": {
"command": "npx",
"args": ["-y", "sitemcp@latest", "https://docs.example.com"]
},
"company-blog": {
"command": "npx",
"args": ["-y", "sitemcp@latest", "https://blog.example.com"]
}
}
}
This lets your AI reference all your critical documentation, support content, and internal knowledge in one place.
Conclusion: Empower Your Developer Workflow with SiteMCP
SiteMCP solves a major challenge for technical teams—letting you instantly make any website or documentation accessible to your AI assistants. Whether you're maintaining API docs in Apidog, supporting internal devs, or streamlining onboarding, SiteMCP ensures your AI always references the most accurate, current information.
By integrating SiteMCP into your workflow, your team can:
- Get real-time answers referencing the latest docs
- Reduce manual copy-pasting and token waste
- Improve AI-driven support, QA, and onboarding
Explore SiteMCP on GitHub, and consider how it can fit into your Apidog-powered API development stack for faster, smarter, and more informed automation.
