Open source repos that use tiktoken-rs

10 public repos that SeeRepo read use tiktoken-rs. It's used most with Tokio, axum, and rusqlite.

Read it as markdown

Used most with

  • Tokio5 repos
  • rusqlite2 repos
  • tower2 repos
  • Ratatui2 repos
tiktoken-rs10 repos
  • axum3 repos
  • SQLite2 repos
  • crossterm2 repos
  • quick-xml2 repos

The repos, by stars

  1. 1
    vllm-project/vllm

    A high-throughput and memory-efficient inference and serving engine for LLMs

    Python+10297.1kAI 14.7%
  2. 2
    rtk-ai/rtk

    CLI proxy that reduces LLM token consumption by 60-90% on common dev commands. Single Rust binary, zero dependencies

    Rust+1286.8kAI 33.4%
  3. 3
    headroomlabs-ai/headroom

    Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.

    Python+2279.9kAI 29.7%
  4. 4
    BerriAI/litellm

    The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

    Python+11763.4kAI 20.6%
  5. 5
    meilisearch/meilisearch

    A lightning-fast search engine API bringing AI-powered hybrid search to your sites and applications.

    Rust+2759.9kAI 0.2%
  6. 6
    aaif-goose/goose

    an open source, extensible AI agent that goes beyond code suggestions - install, execute, edit, and test with any LLM

    Rust+7356.7kAI 5%
  7. 7
    openobserve/openobserve

    Open source observability platform for logs, metrics, traces, RUM (web, android, ios), Session replay, pipelines, SLO and LLM observability. A sophisticated, simple and highly performant alternative to Datadog, Splunk, and Elasticsearch with 140x lower storage costs and single binary deployment.

    Rust+7623.1kAI 5.1%
  8. 8
    funstory-ai/BabelDOC

    Yet Another Document Translator

    Rust+179.9k
  9. 9
    deeplethe/utopia

    World's first open-source enterprise world model.

    Rust+388.7kAI 57.9%
  10. 10
    mufeedvh/code2prompt

    A CLI tool to convert your codebase into a single LLM prompt with source tree, prompt templating, and token counting.

    Rust+107.8kAI 1.1%

Frequently asked questions

What is the most popular open source repo that uses tiktoken-rs?
vllm-project/vllm, with 97,060 stars on GitHub.

The proof: vllm-project/vllm on GitHub

What other technologies is tiktoken-rs used with?
With Tokio (5), axum (3), rusqlite (2), SQLite (2), and tower (2), out of the 10 repos that use it.
What languages are the repos that use tiktoken-rs written in?
Rust (7) and Python (3).
How many repos that use tiktoken-rs have commits signed by AI?
9 of the 10; the most, deeplethe/utopia, with 57.9% of its commits.

The proof: the commits each tool signed in deeplethe/utopia

See inside your code, today.

Paste your repo and see it drawn, part by part: a medium one in under a minute.

Free, no account.Private repo? Sign in with GitHub

Or look inside SeeRepo's own →