Source: https://seerepo.com/r/NVIDIA/TensorRT-LLM/built
Markdown source: https://seerepo.com/r/NVIDIA/TensorRT-LLM/built.md
Description: TensorRT-LLM is a Python repo built with FastAPI, anyio, uvicorn, and starlette. It stores its data in pyzmq.

# What is TensorRT-LLM built with?

TensorRT-LLM is a Python repo built with FastAPI, anyio, uvicorn, and starlette. It stores its data in pyzmq, uses AI through accelerate, apache-tvm-ffi, cuda-python, flash-linear-attention, flashinfer-python, and 32 more, makes its content with jsonlines, pillow, jinja2, and mako, validates its data with Pydantic, msgspec, and jsonschema, is tested with pytest-cov, pytest-env, pytest-forked, pytest-rerunfailures, pytest-timeout, and 6 more, deploys with opentelemetry-api, opentelemetry-semantic-conventions-ai, psutil, prometheus-client, boto3, and 3 more, and uses blobfile and h5py. It has 2,884,778 written lines, with 2,703 test files.

Read on November 8, 2026, at commit b433473.

Repository: https://github.com/NVIDIA/TensorRT-LLM

## Frontend

### Interface

- **[Rich](https://seerepo.com/uses/rich)**: 12 files
- **[matplotlib](https://seerepo.com/uses/matplotlib)**: 1 file
- **[Plotly](https://seerepo.com/uses/plotly)**: 1 file

## Backend

### The base

- **[FastAPI](https://seerepo.com/uses/fastapi)**: 15 files

### Data

- **blobfile**
- **h5py**
- **[pyzmq](https://seerepo.com/uses/pyzmq)**: 13 files

### AI

- **[accelerate](https://seerepo.com/uses/accelerate)**
- **apache-tvm-ffi**
- **cuda-python**
- **flash-linear-attention**
- **flashinfer-python**
- **NVIDIA**
- **nvidia-cudnn-frontend**
- **openai-codex**
- **openai-harmony**
- **optimum**
- **quack-kernels**
- **rouge**
- **rouge-score**
- **[sentencepiece](https://seerepo.com/uses/sentencepiece)**
- **[tiktoken](https://seerepo.com/uses/tiktoken)**
- **torch-c-dlpack-ext**
- **torchao**
- **[torch](https://seerepo.com/uses/torch)**: 1,611 files
- **[triton](https://seerepo.com/uses/triton)**: 218 files
- **[transformers](https://seerepo.com/uses/transformers)**: 168 files
- **[diffusers](https://seerepo.com/uses/diffusers)**: 35 files
- **[einops](https://seerepo.com/uses/einops)**: 20 files
- **[safetensors](https://seerepo.com/uses/safetensors)**: 17 files
- **[torchvision](https://seerepo.com/uses/torchvision)**: 10 files
- **[datasets](https://seerepo.com/uses/datasets)**: 7 files
- **flash-attn**: 7 files
- **xgrammar**: 4 files
- **[librosa](https://seerepo.com/uses/librosa)**: 2 files
- **[mcp](https://seerepo.com/uses/mcp)**: 2 files
- **mistral-common**: 2 files
- **partial-json-parser**: 2 files
- **tritonclient**: 2 files
- **evaluate**: 1 file
- **llguidance**: 1 file
- **lm-eval**: 1 file
- **[peft](https://seerepo.com/uses/peft)**: 1 file

### Services

- **[OpenAI](https://seerepo.com/uses/openai)**: 3 files · [tensorrt_llm/scaffolding/task.py, line 22](https://github.com/NVIDIA/TensorRT-LLM/blob/b433473ba0d0d49764f4d2fb3d45c82716b946dc/tensorrt_llm/scaffolding/task.py#L22)

## Support

### The base

- **[anyio](https://seerepo.com/uses/anyio)**: 7 files
- **[uvicorn](https://seerepo.com/uses/uvicorn)**: 6 files
- **[starlette](https://seerepo.com/uses/starlette)**: 2 files

### Content

- **jsonlines**
- **[pillow](https://seerepo.com/uses/pillow)**: 34 files
- **[jinja2](https://seerepo.com/uses/jinja2)**: 4 files
- **mako**: 2 files

### Validation

- **[Pydantic](https://seerepo.com/uses/pydantic)**: 34 files
- **[msgspec](https://seerepo.com/uses/msgspec)**: 4 files
- **[jsonschema](https://seerepo.com/uses/jsonschema)**: 2 files

### Testing

- **pytest-cov**
- **pytest-env**
- **pytest-forked**
- **pytest-rerunfailures**
- **pytest-timeout**
- **pytest-xdist**
- **Pytest**: 2,232 files
- **parameterized**: 30 files
- **pytest-asyncio**: 4 files
- **pytest-mock**: 1 file
- **pytest-split**: 1 file

### Deploy

- **[opentelemetry-api](https://seerepo.com/uses/opentelemetry-api)**
- **opentelemetry-semantic-conventions-ai**
- **[psutil](https://seerepo.com/uses/psutil)**: 7 files
- **[prometheus-client](https://seerepo.com/uses/prometheus-client)**: 6 files
- **[boto3](https://seerepo.com/uses/boto3)**: 1 file
- **[OpenTelemetry](https://seerepo.com/uses/opentelemetry)**: 1 file
- **[opentelemetry-sdk](https://seerepo.com/uses/opentelemetry-sdk)**: 1 file
- **prometheus-fastapi-instrumentator**: 1 file

### Tools

- **autodoc-pydantic**
- **bandit**
- **build**
- **click-option-group**
- **fire**
- **graphviz**
- **grpcio-tools**
- **meson**
- **mypy**
- **ninja**
- **patchelf**
- **pre-commit**
- **pybind11**
- **Ruff**
- **Sphinx**
- **sphinx-argparse**
- **sphinx-click**
- **sphinx-copybutton**
- **sphinx-togglebutton**
- **sphinxcontrib-mermaid**
- **typeguard**
- **click**: 21 files
- **typing-extensions**: 20 files
- **packaging**: 15 files
- **cuda-tile**: 4 files
- **docstring-parser**: 1 file
- **Setuptools**: 1 file

### Other

- **aenum**
- **backoff**
- **colored**
- **ftfy**
- **grpcio**
- **grpcio-health-checking**
- **hf-transfer**
- **jieba**
- **lark**
- **lazy-loader**
- **mpmath**
- **nvidia-ml-py**
- **omegaconf**
- **protobuf**
- **pydantic-settings**
- **python-multipart**
- **NumPy**: 108 files
- **pyyaml**: 56 files
- **strenum**: 12 files
- **msgpack**: 5 files
- **nvtx**: 4 files
- **pandas**: 4 files
- **blake3**: 3 files
- **filelock**: 3 files
- **soundfile**: 3 files
- **uvloop**: 3 files
- **regex**: 2 files
- **requests**: 2 files
- **cloudpickle**: 1 file
- **distro**: 1 file
- **numexpr**: 1 file
- **ordered-set**: 1 file
- **pynvvideocodec**: 1 file
- **rapidfuzz**: 1 file
- **tabulate**: 1 file

## Other repos with a similar stack

- [ray-project/ray](https://seerepo.com/r/ray-project/ray/built): FastAPI, h5py, pyzmq, anyio, uvicorn, starlette, Rich, matplotlib, Plotly, accelerate, rouge-score, sentencepiece, tiktoken, torch, triton, transformers, diffusers, einops, safetensors, torchvision, datasets, evaluate, lm-eval, peft, jsonlines, pillow, jinja2, mako, Pydantic, jsonschema, opentelemetry-api, psutil, prometheus-client, boto3, OpenTelemetry · [Compare](https://seerepo.com/compare/NVIDIA/TensorRT-LLM/vs/ray-project/ray)
- [sgl-project/sglang](https://seerepo.com/r/sgl-project/sglang/built): FastAPI, blobfile, pyzmq, OpenAI, uvicorn, apache-tvm-ffi, cuda-python, flashinfer-python, NVIDIA, openai-harmony, quack-kernels, sentencepiece, tiktoken, torch-c-dlpack-ext, torch, transformers, einops, torchvision, datasets, xgrammar, mistral-common, partial-json-parser, llguidance, pillow, Pydantic, msgspec, psutil, prometheus-client, OpenTelemetry · [Compare](https://seerepo.com/compare/NVIDIA/TensorRT-LLM/vs/sgl-project/sglang)
- [vllm-project/vllm](https://seerepo.com/r/vllm-project/vllm/built): FastAPI, pyzmq, OpenAI, starlette, NVIDIA, quack-kernels, sentencepiece, tiktoken, torch, triton, transformers, einops, safetensors, torchvision, datasets, MCP, peft, pillow, jinja2, Pydantic, msgspec, jsonschema, opentelemetry-api, psutil, prometheus-client, OpenTelemetry, opentelemetry-sdk, prometheus-fastapi-instrumentator · [Compare](https://seerepo.com/compare/NVIDIA/TensorRT-LLM/vs/vllm-project/vllm)
- [onyx-dot-app/onyx](https://seerepo.com/r/onyx-dot-app/onyx/built): FastAPI, pyzmq, OpenAI, anyio, uvicorn, starlette, Rich, matplotlib, accelerate, NVIDIA, tiktoken, torch, triton, transformers, einops, safetensors, MCP, pillow, jinja2, mako, Pydantic, jsonschema, psutil, prometheus-client, boto3, OpenTelemetry, prometheus-fastapi-instrumentator · [Compare](https://seerepo.com/compare/NVIDIA/TensorRT-LLM/vs/onyx-dot-app/onyx)
- [run-llama/llama_index](https://seerepo.com/r/run-llama/llama_index/built): pyzmq, OpenAI, anyio, starlette, Rich, matplotlib, optimum, sentencepiece, tiktoken, torch, transformers, einops, datasets, MCP, tritonclient, peft, pillow, jinja2, Pydantic, jsonschema, opentelemetry-api, psutil, prometheus-client, boto3, OpenTelemetry · [Compare](https://seerepo.com/compare/NVIDIA/TensorRT-LLM/vs/run-llama/llama_index)
- [mudler/LocalAI](https://seerepo.com/r/mudler/LocalAI/built): FastAPI, OpenAI, anyio, uvicorn, matplotlib, accelerate, optimum, sentencepiece, tiktoken, torchao, torch, transformers, diffusers, einops, safetensors, torchvision, datasets, flash-attn, librosa, MCP, peft, pillow, jinja2, OpenTelemetry · [Compare](https://seerepo.com/compare/mudler/LocalAI/vs/NVIDIA/TensorRT-LLM)

## Frequently asked questions

### What language is TensorRT-LLM written in?

TensorRT-LLM is written in Python, with 2,884,778 written lines.

### What framework does TensorRT-LLM use?

TensorRT-LLM is built with FastAPI, anyio, uvicorn, and starlette.

### What database does TensorRT-LLM use?

TensorRT-LLM stores its data in pyzmq.

### Does TensorRT-LLM use AI?

Yes. TensorRT-LLM uses AI through accelerate, apache-tvm-ffi, cuda-python, flash-linear-attention, flashinfer-python, NVIDIA, nvidia-cudnn-frontend, openai-codex, openai-harmony, optimum, quack-kernels, rouge, rouge-score, sentencepiece, tiktoken, torch-c-dlpack-ext, torchao, torch, triton, transformers, diffusers, einops, safetensors, torchvision, datasets, flash-attn, xgrammar, librosa, mcp, mistral-common, partial-json-parser, tritonclient, evaluate, llguidance, lm-eval, peft, and OpenAI.

### What services does TensorRT-LLM use?

TensorRT-LLM uses blobfile and h5py.

### Does TensorRT-LLM have tests?

Yes. TensorRT-LLM has 2,703 test files and is tested with pytest-cov, pytest-env, pytest-forked, pytest-rerunfailures, pytest-timeout, pytest-xdist, Pytest, parameterized, pytest-asyncio, pytest-mock, and pytest-split.

### Which repos have a stack like TensorRT-LLM's?

The ones that share most of its stack: ray-project/ray, sgl-project/sglang, and vllm-project/vllm.

## What is your repo built with?

Any public repo, with no account: https://seerepo.com/r/owner/repo/built, with the owner and name it has on GitHub.