webclaw
Free快速本地优先的网页内容提取工具，专为 LLM 设计。支持将网页转换为 Markdown/JSON/纯文本，内置 TLS 指纹伪装绕过反爬，无需浏览器。提供 MCP 服务器（10 个工具）和 CLI。
FreeFree tier
Inputs: url
About webclaw
Fast, local-first web content extraction tool designed for LLMs. Converts web pages into clean Markdown, JSON, or LLM-ready text. Features TLS fingerprint spoofing to bypass anti-bot measures, eliminating the need for a browser. Includes a CLI, MCP server (with 10 tools), REST API, and SDKs for easy integration into AI agents and RAG pipelines. Supports crawling with depth and page limits, content filtering via CSS selectors, and can be self-hosted. Built with Rust for performance and reliability.
Key Features
Converts web pages to clean Markdown, JSON, or LLM-ready text
CLI, MCP server, REST API, and SDKs for integration
TLS fingerprint spoofing to bypass anti-bot measures
No browser required for fast extraction
Crawl websites with configurable depth and max pages
Include/exclude CSS selectors for content filtering
Only main content extraction option
Local-first and self-hostable
Built with Rust for performance
Pros & Cons
Pros
- Fast and lightweight due to Rust implementation
- Local-first design ensures privacy and offline capability
- No browser required, reducing overhead
- TLS fingerprint spoofing helps bypass anti-bot measures
- Supports multiple output formats (Markdown, JSON, LLM text)
- Multiple integration options: CLI, MCP server, REST API, SDKs
- Open source with self-hosting capability
- Configurable crawling with depth and page limits
Cons
- No JavaScript rendering, limiting extraction of dynamic content
- Relatively new project with limited community resources and examples
Best For
Web scraping and crawling for AI agentsRetrieval-Augmented Generation (RAG) pipeline content preparationExtracting clean content from documentation sitesConverting web pages to LLM-ready text formatStructured data extraction for knowledge bases
FAQ
What is webclaw?
webclaw is a fast, local-first web content extraction tool for LLMs that converts URLs into clean Markdown, JSON, or LLM-ready text.
How can I install webclaw?
You can install via Homebrew (brew tap 0xMassi/webclaw && brew install webclaw), Docker (docker run --rm ghcr.io/0xmassi/webclaw), prebuilt binaries from GitHub Releases, or cargo install.
Can I use webclaw with Claude or Cursor?
Yes, webclaw provides an MCP server that integrates with Claude Code, Claude Desktop, Cursor, Windsurf, OpenCode, Codex CLI, and other MCP-compatible tools.
Does webclaw require a browser?
No, webclaw does not require a browser; it uses TLS fingerprint spoofing to bypass anti-bot measures.
What output formats does webclaw support?
Markdown, JSON, and LLM-ready text.
Is webclaw free?
Yes, the CLI, MCP server, extraction engine, and self-hostable server are open source and free. A hosted API is available at webclaw.io.