What is inference built with?

Q: Which repos have a stack like inference's?

The ones that share most of its stack: mudler/LocalAI, bojieli/ai-agent-book, and unslothai/unsloth.

inference is a Python repo built with FastAPI, Lightning, uvicorn, AIOHTTP, anyio, and 2 more. It signs people in with python-jose and bcrypt, uses AI through descript-audiotools, en-core-web-sm, misaki, opencv-contrib-python-headless, pynini, and 41 more, makes its content with eva-decord, imageio-ffmpeg, pillow, torchcodec, highlight.js, and 7 more, validates its data with Pydantic and jsonschema, is tested with jsdom, deploys with opentelemetry-instrumentation-httpx, loguru, psutil, boto3, OpenTelemetry, and 2 more, and uses pyarrow and Hugging Face. It has 922,374 written lines, with 676 test files.

Read on November 14, 2026, at commit d2607af. Read it as markdown

The card says what it's built with. Its whole map opens it up: every part of the code and how it connects to the rest.

Frontend

Interface 3

sonner30 files
matplotlib22 files
Radix UI20 files

The base 2

React135 files · in frontend/package.json
Next.js25 files · in frontend/package.json

Styles 4

Icons 2

Lucide92 files
react-icons1 file

Backend

The base 1

FastAPI31 files

Data 1

pyarrow2 files

Sign-in 2

python-jose2 files
bcrypt1 file

AI 45

torch1,215 files
torchaudio79 files
torchvision42 files
librosa35 files
safetensors25 files
diffusers22 files
ModelScope21 files
timm17 files
accelerate7 files
funasr5 files
julius4 files
peft3 files

33 more

sentence-transformers3 files
fugashi1 file
optimum1 file
opencv-contrib-python-headless
transformers208 files
einops64 files
onnxruntime13 files
x-transformers11 files
qwen-vl-utils7 files
tiktoken7 files
bitsandbytes3 files
loralib3 files
pyworld3 files
flagembedding2 files
gruut2 files
kokoro2 files
sentencepiece2 files
silero-vad2 files
torchdiffeq2 files
vector-quantize-pytorch2 files
vocos2 files
conformer1 file
controlnet-aux1 file
g2p-en1 file
gguf1 file
langdetect1 file
tensorizer1 file
descript-audiotools
en-core-web-sm
misaki
pynini
transformers-stream-generator
wetextprocessing

Services 2

Hugging Face51 files · xinference/model/utils.py, line 48
OpenAI1 file · xinference/_compat.py, line 57

Support

The base 4

Lightning17 files
uvicorn9 files
AIOHTTP6 files
anyio2 files

Content 12

pillow120 files
highlight.js1 file
Nunjucks1 file
rehype-katex1 file
rehype-raw1 file
rehype-sanitize1 file
remark-gfm1 file
remark-math1 file
imageio-ffmpeg
react-markdown1 file
torchcodec1 file
eva-decord

Validation 2

Pydantic12 files
jsonschema1 file

Deploy 7

loguru120 files
psutil15 files
OpenTelemetry1 file
opentelemetry-exporter-otlp-proto-grpc1 file
opentelemetry-instrumentation-fastapi1 file
opentelemetry-instrumentation-httpx
boto31 file

Tools 9

click29 files
TypeScript4 files
Numba3 files
uv1 file
datamodel-code-generator
jupyterlab
tensorboard
typing-extensions8 files
argbind5 files

Testing 1

jsdom26 files

Other 45

NumPy437 files
tqdm80 files
omegaconf34 files
Axios10 files
cached-path6 files
date-fns5 files
hydra-core3 files
anyascii1 file
pandas1 file
nvidia-ml-py
soundfile50 files
requests19 files

33 more

tabulate11 files
pydub10 files
inflect6 files
jieba5 files
ormsgpack5 files
json54 files
pypinyin4 files
xxhash4 files
hyperpyyaml3 files
natsort3 files
pyloudnorm3 files
setproctitle3 files
sse-starlette3 files
tomli3 files
cn2an2 files
ffmpy2 files
flatten-dict2 files
munch2 files
num2words2 files
pykakasi2 files
async-timeout1 file
cachetools1 file
clsx1 file
gdown1 file
orjson1 file
randomname1 file
",
and
preshed
protobuf
python-multipart
s
unidic-lite

Other repos with a similar stack

Ordered by how much of its stack they share with inference, then by stars.

mudlerLocalAI
50.3k
LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.
36 of 142 in common · GoCompare →
bojieliai-agent-book
57.9k
《深入理解 AI Agent：设计原理与工程实践》（李博杰著）开源主仓库：全书正文、编译版 PDF 与按章配套代码
33 of 142 in commonCompare →
unslothaiunsloth
79.4k
Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more.
32 of 142 in commonCompare →
debpalashVoiceStudio
108.5k
VoiceStudio is the open-source, fully-local ElevenLabs alternative — voice cloning, voice design, video dubbing, dictation, transcription & audiobook creation in 646 languages.
31 of 142 in commonCompare →
ray-projectray
44.3k
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
31 of 142 in commonCompare →
datawhalechinahello-agents
86.8k
📚 《从零开始构建智能体》——从零开始的智能体原理与实践教程
29 of 142 in commonCompare →

Frequently asked questions

What language is inference written in?

inference is written in Python, with 922,374 written lines.

What framework does inference use?

inference is built with FastAPI, Lightning, uvicorn, AIOHTTP, anyio, React, and Next.js.

What does inference sign people in with?

inference signs people in with python-jose and bcrypt.

Does inference use AI?

Yes. inference uses AI through descript-audiotools, en-core-web-sm, misaki, opencv-contrib-python-headless, pynini, transformers-stream-generator, wetextprocessing, torch, transformers, torchaudio, einops, torchvision, librosa, safetensors, diffusers, ModelScope, timm, onnxruntime, x-transformers, accelerate, qwen-vl-utils, tiktoken, funasr, julius, bitsandbytes, loralib, peft, pyworld, sentence-transformers, flagembedding, gruut, kokoro, sentencepiece, silero-vad, torchdiffeq, vector-quantize-pytorch, vocos, conformer, controlnet-aux, fugashi, g2p-en, gguf, langdetect, optimum, tensorizer, and OpenAI.

What services does inference use?

inference uses pyarrow and Hugging Face.

Does inference have tests?

Yes. inference has 676 test files and is tested with jsdom.

Which repos have a stack like inference's?

The ones that share most of its stack:mudler/LocalAI bojieli/ai-agent-book unslothai/unsloth