Source: https://seerepo.com/compare/EleutherAI/lm-evaluation-harness/vs/NVIDIA/TensorRT-LLM
Markdown source: https://seerepo.com/compare/EleutherAI/lm-evaluation-harness/vs/NVIDIA/TensorRT-LLM.md
Description: lm-evaluation-harness and TensorRT-LLM share 4 technologies: evaluate, rouge-score, datasets, and jinja2. lm-evaluation-harness is written in C++ and TensorRT-LLM in Python.

# lm-evaluation-harness vs TensorRT-LLM

lm-evaluation-harness and TensorRT-LLM share 4 technologies: evaluate, rouge-score, datasets, and jinja2. lm-evaluation-harness is written in C++ and TensorRT-LLM in Python.

- [EleutherAI/lm-evaluation-harness](https://seerepo.com/r/EleutherAI/lm-evaluation-harness): A framework for few-shot evaluation of language models.
- [NVIDIA/TensorRT-LLM](https://seerepo.com/r/NVIDIA/TensorRT-LLM): TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

## What they share and what sets them apart

- In common: evaluate, rouge-score, datasets, jinja2
- Only in lm-evaluation-harness: fugashi, scikit-learn, sacrebleu
- Only in TensorRT-LLM: FastAPI, blobfile, h5py, pyzmq, OpenAI, anyio, uvicorn, starlette, Rich, matplotlib, Plotly, accelerate, apache-tvm-ffi, cuda-python, flash-linear-attention, flashinfer-python, NVIDIA, nvidia-cudnn-frontend, openai-codex, openai-harmony, optimum, quack-kernels, rouge, sentencepiece, tiktoken, torch-c-dlpack-ext, torchao, torch, triton, transformers, diffusers, einops, safetensors, torchvision, flash-attn, xgrammar, librosa, MCP, mistral-common, partial-json-parser, tritonclient, llguidance, lm-eval, peft, jsonlines, pillow, mako, Pydantic, msgspec, jsonschema, opentelemetry-api, opentelemetry-semantic-conventions-ai, psutil, prometheus-client, boto3, OpenTelemetry, opentelemetry-sdk, prometheus-fastapi-instrumentator

## Side by side

| | lm-evaluation-harness | TensorRT-LLM |
|---|---|---|
| Size | 107,687 lines written, 2 code files | 2,884,778 lines written, 3,120 code files |
| Who writes it | 0.3% of its commits signed by AI. With Claude Code and Cursor | 1.7% of its commits signed by AI. With Claude Code, CodeRabbit, GitHub Copilot, and Cursor |
| Activity | Last push November 4, 2026, 3,347 commits, 14,426 stars | Last push November 8, 2026, 10,997 commits, 15,038 stars |
| Tests | 51 test files | 2,703 test files. 58.6% of its code files have a test that imports them |

- Size: Lines written by people and agents, without generated ones; code files, without tests.
- Who writes it: A commit counts as AI's when it carries its signature: a Co-Authored-By line, its author name or the merge of its branch.
- Activity: From GitHub and its history, on the day SeeRepo read it.
- Tests: The files that test the code, and the share of code files a test imports.

## Frequently asked questions

### Which is bigger, lm-evaluation-harness or TensorRT-LLM?

TensorRT-LLM, with 2,884,778 lines written; lm-evaluation-harness has 107,687.

The proof: [NVIDIA/TensorRT-LLM's code at commit b433473](https://github.com/NVIDIA/TensorRT-LLM/tree/b433473ba0d0d49764f4d2fb3d45c82716b946dc)

### Which has more GitHub stars, lm-evaluation-harness or TensorRT-LLM?

TensorRT-LLM, with 15,038 stars; lm-evaluation-harness has 14,426.

The proof: [NVIDIA/TensorRT-LLM on GitHub](https://github.com/NVIDIA/TensorRT-LLM)

### Which has more tests, lm-evaluation-harness or TensorRT-LLM?

TensorRT-LLM, with 2,703 test files; lm-evaluation-harness has 51.

The proof: [NVIDIA/TensorRT-LLM's code at commit b433473](https://github.com/NVIDIA/TensorRT-LLM/tree/b433473ba0d0d49764f4d2fb3d45c82716b946dc)

### Which was updated more recently, lm-evaluation-harness or TensorRT-LLM?

TensorRT-LLM: its last push was on November 8, 2026, and lm-evaluation-harness's on November 4, 2026.

The proof: [NVIDIA/TensorRT-LLM's commits on GitHub](https://github.com/NVIDIA/TensorRT-LLM/commits)

### Which has more commits signed by AI, lm-evaluation-harness or TensorRT-LLM?

TensorRT-LLM: an AI signed 1.7% of its commits, with Claude Code, CodeRabbit, GitHub Copilot, and Cursor; in lm-evaluation-harness, 0.3%.

The proof: [the commits each tool signed in NVIDIA/TensorRT-LLM](https://seerepo.com/r/NVIDIA/TensorRT-LLM/ai)

## Sources

- EleutherAI/lm-evaluation-harness: Read on November 9, 2026, at commit d6de816 (https://github.com/EleutherAI/lm-evaluation-harness/tree/d6de81643928d653435c431bae19945d41d32520)
- NVIDIA/TensorRT-LLM: Read on November 8, 2026, at commit b433473 (https://github.com/NVIDIA/TensorRT-LLM/tree/b433473ba0d0d49764f4d2fb3d45c82716b946dc)

## What is your repo built with?

Any public repo, with no account: https://seerepo.com/r/owner/repo/built, with the owner and name it has on GitHub.
