Open source repos that use sentencepiece

10 public repos that SeeRepo read use sentencepiece. It's used most with torch, transformers, and FastAPI.

Read it as markdown

Used most with

  • torch6 repos
  • FastAPI6 repos
  • pillow4 repos
  • AIOHTTP3 repos
sentencepiece10 repos
  • transformers6 repos
  • einops4 repos
  • torchvision4 repos
  • OpenAI3 repos

The repos, by stars

  1. 1
    Comfy-Org/ComfyUI

    The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface. The fastest local inference engine in the world.

    Python+8144.9kAI 0.3%
  2. 2
    vllm-project/vllm

    A high-throughput and memory-efficient inference and serving engine for LLMs

    Python+4396.3kAI 12.8%
  3. 3
    PaddlePaddle/PaddleOCR

    Turn any PDF or image document into structured data for your AI. A powerful, lightweight OCR toolkit that bridges the gap between images/PDFs and LLMs. Supports 100+ languages.

    C+++2092.3kAI 0.3%
  4. 4
    unslothai/unsloth

    Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more.

    Python+9579.1kAI 1%
  5. 5
    tensorflow/models

    Models and examples built with TensorFlow

    Python+1677.6k
  6. 6
    hiyouga/LlamaFactory

    Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

    Python+2475.9kAI 1.6%
  7. 7
    bojieli/ai-agent-book

    《深入理解 AI Agent:设计原理与工程实践》(李博杰 著)开源主仓库:全书正文、编译版 PDF 与按章配套代码

    Python+9857.2kAI 20.8%
  8. 8
    mlc-ai/mlc-llm

    Universal LLM Deployment Engine with ML Compilation

    Python+1723.3kAI 0.2%
  9. 9
    microsoft/unilm

    Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

    Python+2822.2k
  10. 10
    PromtEngineer/localGPT

    Chat with your documents on your local device using GPT models. No data leaves your device and 100% private.

    Python+1722.2kAI 17.8%

Frequently asked questions

What is the most popular open source repo that uses sentencepiece?
Comfy-Org/ComfyUI, with 144,948 stars on GitHub.

The proof: Comfy-Org/ComfyUI on GitHub

What other technologies is sentencepiece used with?
With torch (6), transformers (6), FastAPI (6), einops (4), and pillow (4), out of the 10 repos that use it.
What languages are the repos that use sentencepiece written in?
Python (9) and C++ (1).
How many repos that use sentencepiece have commits signed by AI?
8 of the 10; the most, bojieli/ai-agent-book, with 20.8% of its commits.

The proof: the commits each tool signed in bojieli/ai-agent-book

See inside your code, today.

Paste your repo and see it drawn, part by part: a medium one in under a minute.

Free, no account.

Or look inside SeeRepo's own →