Open source repos that use tiktoken

11 public repos that SeeRepo read use tiktoken. It's used most with OpenAI, FastAPI, and LangChain.

Read it as markdown

Used most with

  • OpenAI4 repos
  • LangChain2 repos
  • pillow2 repos
  • uvicorn2 repos
tiktoken11 repos
  • FastAPI3 repos
  • Ollama2 repos
  • Pydantic2 repos
  • datasets2 repos

The repos, by stars

  1. 1
    Significant-Gravitas/AutoGPT

    AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.

    Python+124187.9kAI 16.8%
  2. 2
    open-webui/open-webui

    User-friendly AI Interface (Supports Ollama, OpenAI API, ...)

    Svelte+66157.7kAI 1.3%
  3. 3
    langchain-ai/langchain

    The agent engineering platform.

    Python+11149kAI 0.3%
  4. 4
    vllm-project/vllm

    A high-throughput and memory-efficient inference and serving engine for LLMs

    Python+4396.3kAI 12.8%
  5. 5
    datawhalechina/hello-agents

    📚 《从零开始构建智能体》——从零开始的智能体原理与实践教程

    Python+8886kAI 2.4%
  6. 6
    unslothai/unsloth

    Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more.

    Python+9579.1kAI 1%
  7. 7
    headroomlabs-ai/headroom

    Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.

    Python+2178.9kAI 28.1%
  8. 8
    rohitg00/ai-engineering-from-scratch

    Learn AI Engineering! Learn it. Build it. Ship it for others.

    Python+1876.9kAI 0.8%
  9. 9
    mlc-ai/mlc-llm

    Universal LLM Deployment Engine with ML Compilation

    Python+1723.3kAI 0.2%
  10. 10
    comet-ml/opik

    Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

    Python+8923.2kAI 19.7%
  11. 11
    wandb/openui

    OpenUI let's you describe UI using your imagination, then see it rendered live.

    TypeScript+1822.7k

Frequently asked questions

What is the most popular open source repo that uses tiktoken?
Significant-Gravitas/AutoGPT, with 187,891 stars on GitHub.

The proof: Significant-Gravitas/AutoGPT on GitHub

What other technologies is tiktoken used with?
With OpenAI (4), FastAPI (3), LangChain (2), Ollama (2), and pillow (2), out of the 11 repos that use it.
What languages are the repos that use tiktoken written in?
Python (9), Svelte (1), and TypeScript (1).
How many repos that use tiktoken have commits signed by AI?
10 of the 11; the most, headroomlabs-ai/headroom, with 28.1% of its commits.

The proof: the commits each tool signed in headroomlabs-ai/headroom

See inside your code, today.

Paste your repo and see it drawn, part by part: a medium one in under a minute.

Free, no account.

Or look inside SeeRepo's own →