Source: https://seerepo.com/r/unclecode/crawl4ai/built
Markdown source: https://seerepo.com/r/unclecode/crawl4ai/built.md
Description: crawl4ai is a Python repo built with FastAPI, uvicorn, AIOHTTP, and anyio. It stores its data in SQLite and signs people in with pyjwt.

# What is crawl4ai built with?

crawl4ai is a Python repo built with FastAPI, uvicorn, AIOHTTP, and anyio. It stores its data in SQLite, signs people in with pyjwt, uses AI through rank-bm25 and mcp, makes its content with lxml, beautifulsoup4, pillow, and pypdf, validates its data with email-validator and Pydantic, is tested with playwright and fake-useragent, deploys with Gunicorn, psutil, and prometheus-fastapi-instrumentator, and uses aiosqlite and Redis. It has 176,861 written lines, with 286 test files.

Read on November 4, 2026, at commit cae74b2.

Repository: https://github.com/unclecode/crawl4ai

## Frontend

### Interface

- **[Rich](https://seerepo.com/uses/rich)**: 7 files

## Backend

### The base

- **[FastAPI](https://seerepo.com/uses/fastapi)**: 7 files
- **[uvicorn](https://seerepo.com/uses/uvicorn)**: 1 file

### Data

- **aiosqlite**
- **[Redis](https://seerepo.com/uses/redis)**: 4 files · [deploy/docker/api.py, line 16](https://github.com/unclecode/crawl4ai/blob/cae74b203a33994f7e13b9052c3b0d5c93daa0e6/deploy/docker/api.py#L16)
- **[SQLite](https://seerepo.com/uses/sqlite)**: 4 files · [crawl4ai/async_database.py, line 4](https://github.com/unclecode/crawl4ai/blob/cae74b203a33994f7e13b9052c3b0d5c93daa0e6/crawl4ai/async_database.py#L4)

### Sign-in

- **[pyjwt](https://seerepo.com/uses/pyjwt)**: 2 files

### AI

- **rank-bm25**: 5 files
- **[mcp](https://seerepo.com/uses/mcp)**: 1 file

## Support

### The base

- **[AIOHTTP](https://seerepo.com/uses/aiohttp)**: 5 files
- **anyio**: 2 files

### Content

- **lxml**: 8 files
- **[beautifulsoup4](https://seerepo.com/uses/beautifulsoup4)**: 4 files
- **[pillow](https://seerepo.com/uses/pillow)**: 3 files
- **pypdf**: 1 file

### Validation

- **email-validator**
- **[Pydantic](https://seerepo.com/uses/pydantic)**: 8 files

### Testing

- **playwright**: 9 files
- **fake-useragent**: 1 file

### Deploy

- **Gunicorn**
- **[psutil](https://seerepo.com/uses/psutil)**: 8 files
- **prometheus-fastapi-instrumentator**: 1 file

### Tools

- **click**: 3 files

### Other

- **cssselect**
- **shapely**
- **httpx**: 7 files
- **requests**: 7 files
- **NumPy**: 5 files
- **aiofiles**: 4 files
- **pyyaml**: 3 files
- **lark**: 2 files
- **xxhash**: 2 files
- **brotli**: 1 file
- **chardet**: 1 file
- **colorama**: 1 file
- **humanize**: 1 file
- **pyopenssl**: 1 file
- **python-dotenv**: 1 file
- **slowapi**: 1 file
- **snowballstemmer**: 1 file
- **websockets**: 1 file

## Other repos with a similar stack

- [datawhalechina/hello-agents](https://seerepo.com/r/datawhalechina/hello-agents/built): FastAPI, uvicorn, aiosqlite, Redis, SQLite, AIOHTTP, anyio, Rich, rank-bm25, MCP, lxml, beautifulsoup4, pillow, pypdf, email-validator, Pydantic · [Compare](https://seerepo.com/compare/datawhalechina/hello-agents/vs/unclecode/crawl4ai)
- [open-webui/open-webui](https://seerepo.com/r/open-webui/open-webui/built): FastAPI, uvicorn, Redis, SQLite, AIOHTTP, pyjwt, rank-bm25, MCP, lxml, beautifulsoup4, pillow, pypdf, Pydantic, psutil · [Compare](https://seerepo.com/compare/open-webui/open-webui/vs/unclecode/crawl4ai)
- [Significant-Gravitas/AutoGPT](https://seerepo.com/r/Significant-Gravitas/AutoGPT/built): FastAPI, uvicorn, Redis, AIOHTTP, Rich, pyjwt, rank-bm25, beautifulsoup4, pillow, pypdf, Pydantic, psutil, prometheus-fastapi-instrumentator · [Compare](https://seerepo.com/compare/Significant-Gravitas/AutoGPT/vs/unclecode/crawl4ai)
- [langflow-ai/langflow](https://seerepo.com/r/langflow-ai/langflow/built): FastAPI, uvicorn, Redis, AIOHTTP, Rich, pyjwt, MCP, pillow, pypdf, Pydantic, psutil · [Compare](https://seerepo.com/compare/langflow-ai/langflow/vs/unclecode/crawl4ai)
- [unslothai/unsloth](https://seerepo.com/r/unslothai/unsloth/built): FastAPI, uvicorn, anyio, Rich, pyjwt, lxml, pillow, Pydantic, psutil · [Compare](https://seerepo.com/compare/unclecode/crawl4ai/vs/unslothai/unsloth)
- [browser-use/browser-use](https://seerepo.com/r/browser-use/browser-use/built): AIOHTTP, anyio, Rich, MCP, pillow, pypdf, Pydantic, psutil · [Compare](https://seerepo.com/compare/browser-use/browser-use/vs/unclecode/crawl4ai)

## Frequently asked questions

### What language is crawl4ai written in?

crawl4ai is written in Python, with 176,861 written lines.

### What framework does crawl4ai use?

crawl4ai is built with FastAPI, uvicorn, AIOHTTP, and anyio.

### What database does crawl4ai use?

crawl4ai stores its data in SQLite.

The proof: [crawl4ai/async_database.py, line 4](https://github.com/unclecode/crawl4ai/blob/cae74b203a33994f7e13b9052c3b0d5c93daa0e6/crawl4ai/async_database.py#L4)

### What does crawl4ai sign people in with?

crawl4ai signs people in with pyjwt.

### Does crawl4ai use AI?

Yes. crawl4ai uses AI through rank-bm25 and mcp.

### What services does crawl4ai use?

crawl4ai uses aiosqlite and Redis.

### Does crawl4ai have tests?

Yes. crawl4ai has 286 test files and is tested with playwright and fake-useragent.

### Which repos have a stack like crawl4ai's?

The ones that share most of its stack: datawhalechina/hello-agents, open-webui/open-webui, and Significant-Gravitas/AutoGPT.

## What is your repo built with?

Any public repo, with no account: https://seerepo.com/r/owner/repo/built, with the owner and name it has on GitHub.