Open source repos that use tiktoken-rs
10 public repos that SeeRepo read use tiktoken-rs. It's used most with Tokio, axum, and rusqlite.
Used most with
- Tokio5 repos
rusqlite2 repos
- tower2 repos
- Ratatui2 repos
- axum3 repos
- SQLite2 repos
- crossterm2 repos
- quick-xml2 repos
The repos, by stars
- 1vllm-project/
vllm A high-throughput and memory-efficient inference and serving engine for LLMs
Python+10297.1kAI 14.7%
- 2rtk-ai/
rtk CLI proxy that reduces LLM token consumption by 60-90% on common dev commands. Single Rust binary, zero dependencies
Rust+1286.8kAI 33.4%
- 3headroomlabs-ai/
headroom Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
Python+2279.9kAI 29.7%
- 4BerriAI/
litellm The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]
Python+11763.4kAI 20.6%
- 5meilisearch/
meilisearch A lightning-fast search engine API bringing AI-powered hybrid search to your sites and applications.
Rust+2759.9kAI 0.2%
- 6aaif-goose/
goose an open source, extensible AI agent that goes beyond code suggestions - install, execute, edit, and test with any LLM
Rust+7356.7kAI 5% - 7openobserve/
openobserve Open source observability platform for logs, metrics, traces, RUM (web, android, ios), Session replay, pipelines, SLO and LLM observability. A sophisticated, simple and highly performant alternative to Datadog, Splunk, and Elasticsearch with 140x lower storage costs and single binary deployment.
Rust+7623.1kAI 5.1% - 10mufeedvh/
code2prompt A CLI tool to convert your codebase into a single LLM prompt with source tree, prompt templating, and token counting.
Rust+107.8kAI 1.1%
Frequently asked questions
What is the most popular open source repo that uses tiktoken-rs?
The proof: vllm-project/vllm on GitHub
What other technologies is tiktoken-rs used with?
What languages are the repos that use tiktoken-rs written in?
How many repos that use tiktoken-rs have commits signed by AI?
See inside your code, today.
Paste your repo and see it drawn, part by part: a medium one in under a minute.
Free, no account.Private repo? Sign in with GitHub

