crw

Free

开源网页抓取工具，专为 AI 代理设计。单一 Rust 二进制文件，内置 MCP 服务器，兼容 Firecrawl API。

FreeFree tier

Inputs: url

Type

Open Source

Visit Website GitHub (40 stars)

About crw

CRW (also branded as FastCRW) is a fast, lightweight web scraping, crawling, and search API tool built in Rust. It serves as a drop-in replacement for Firecrawl and Tavily, offering a compatible API endpoint structure (/scrape, /crawl, /search). Designed for AI agents, it includes a built-in MCP (Model Context Protocol) server for seamless integration with AI coding assistants like Claude Code, Cursor, and Cline. The tool can be self-hosted as a single Rust binary (6 MB RAM usage) or used via a managed cloud service. Benchmarks claim it is 2.3x faster than Tavily and 1.5x faster than Firecrawl on 1K-URL tests. It outputs clean markdown or JSON from any website and provides SDKs for Python, TypeScript, and cURL, along with a free tier offering 500 credits (1 credit ≈ 1 page).

Key Features

Drop-in replacement for Firecrawl API (/scrape, /crawl, /search)

Built-in MCP server for AI agent integration (Claude Code, Cursor, Windsurf, Cline, Codex, Gemini CLI)

Single Rust binary with low memory footprint (6 MB RAM)

2.3x faster than Tavily and 1.5x faster than Firecrawl in 1K-URL benchmarks

Outputs clean markdown or JSON

Self-hostable open-source engine or managed cloud service

SDKs available for Python, TypeScript, and cURL

Free tier with 500 credits (1 credit ≈ 1 page) and no credit card required

Compatible with Crawl4AI on truth-recall (measured on Firecrawl's public dataset)

Pros & Cons

Pros

Significantly faster than alternatives (Tavily, Firecrawl) per benchmarks
Very lightweight – single binary, 6 MB RAM
Open-source with self-hosting option
Drop-in API compatibility with Firecrawl simplifies migration
Built-in MCP server for easy AI agent integration
Free tier available with 500 page credits (no credit card)
Multi-language SDKs (Python, TypeScript, cURL)

Cons

Primarily designed for AI agent use; general users may find limited documentation
Self-hosting requires some technical setup (Rust compilation or Docker)
Cloud managed service pricing beyond free tier is not explicitly detailed on the main page
Branding confusion between 'crw' and 'FastCRW' may cause inconsistency

Best For

Feeding clean web content into AI agents for RAG and data extractionBuilding custom search and crawl pipelines for AI applicationsScraping websites into markdown for LLM contextReplacing Firecrawl or Tavily in existing AI agent workflowsMapping and crawling entire websites for structured data collection

FAQ

What is CRW?

CRW (also called FastCRW) is an open-source web scraping, crawling, and search API built in Rust. It is designed as a lightweight, fast alternative to Firecrawl and Tavily for AI agents, and includes a built-in MCP server.

How does CRW compare to Firecrawl?

CRW is a drop-in replacement for the Firecrawl API, offering compatible endpoints (/scrape, /crawl, /search). It claims to be 1.5x faster than Firecrawl in 1K-URL benchmarks and beats Firecrawl on truth-recall measured on Firecrawl's own public dataset.

Is there a free tier?

Yes, CRW offers a free tier with 500 credits (1 credit ≈ 1 page) through its managed cloud at fastcrw.com/register. No credit card is required for the free tier.

Can I self-host CRW?

Yes, CRW is open source and can be self-hosted. It runs as a single Rust binary with low memory usage (6 MB RAM) and can be deployed via Docker.

What SDKs are available?

CRW provides SDKs for Python (pip install crw) and Node.js (npm install crw-sdk), as well as cURL for direct API calls. It also supports MCP for integration with AI coding tools like Claude Code, Cursor, and Cline.