Source: https://seerepo.com/r/datahub-project/datahub/built
Markdown source: https://seerepo.com/r/datahub-project/datahub/built.md
Description: datahub is a Python repo built with FastAPI, Spring, pyspark, AIOHTTP, anyio, and 5 more.

# What is datahub built with?

datahub is a Python repo built with FastAPI, Spring, pyspark, AIOHTTP, anyio, and 5 more. It stores its data in org.springframework.kafka:spring-kafka and org.apache.iceberg:iceberg-core, signs people in with org.springframework.security:spring-security-oauth2-jose, software.amazon.awssdk:sts, and com.google.auth:google-auth-library-oauth2-http, stores files in AWS S3, uses AI through software.amazon.awssdk:bedrockruntime, watches its errors with Sentry, connects to Google, makes its content with velocity-engine-core, beautifulsoup4, dompurify, js-yaml, html-to-image, and 3 more, validates its data with Pydantic, jsonschema, and io.swagger.parser.v3:swagger-parser, is tested with Pytest, Vitest, and Testing Library, deploys with log4j-core, loguru, org.apache.logging.log4j:log4j-jul, python-json-logger, slf4j-log4j12, and 7 more, and uses s3, spark-sql_2.12, org.apache.iceberg:iceberg-aws, Google Cloud Storage, and Mixpanel. It has 2,991,794 written lines, with 5,307 test files.

Read on November 10, 2026, at commit 0d9b954.

Repository: https://github.com/datahub-project/datahub

## Frontend (/)

### Interface

- **click-spinner**: 2 files

## Backend (/)

### The base

- **[FastAPI](https://seerepo.com/uses/fastapi)**
- **[Spring](https://seerepo.com/uses/spring)**

### Data

- **[s3](https://seerepo.com/uses/s3)**
- **spark-sql_2.12**
- **org.springframework.kafka:spring-kafka**: 23 files
- **org.apache.iceberg:iceberg-core**: 18 files
- **org.apache.iceberg:iceberg-aws**: 1 file

### Sign-in

- **org.springframework.security:spring-security-oauth2-jose**
- **[software.amazon.awssdk:sts](https://seerepo.com/uses/softwareamazonawssdk-sts)**: 4 files
- **com.google.auth:google-auth-library-oauth2-http**: 1 file

### AI

- **software.amazon.awssdk:bedrockruntime**: 1 file

### Services

- **[AWS S3](https://seerepo.com/uses/aws-s3)**: 4 files · [metadata-service/factories/src/main/java/com/linkedin/gms/factory/objectstorage/ObjectStorageClientFactory.java, line 23](https://github.com/datahub-project/datahub/blob/0d9b95469f4b98109939b0c97826fa3787926604/metadata-service/factories/src/main/java/com/linkedin/gms/factory/objectstorage/ObjectStorageClientFactory.java#L23)
- **[Google](https://seerepo.com/uses/google)**: 3 files · [metadata-service/factories/src/main/java/com/linkedin/gms/factory/ingestion/ObjectStorageMatrixDocumentReader.java, line 3](https://github.com/datahub-project/datahub/blob/0d9b95469f4b98109939b0c97826fa3787926604/metadata-service/factories/src/main/java/com/linkedin/gms/factory/ingestion/ObjectStorageMatrixDocumentReader.java#L3)
- **[Google Cloud Storage](https://seerepo.com/uses/google-cloud-storage)**: 3 files · [metadata-service/factories/src/main/java/com/linkedin/gms/factory/ingestion/ObjectStorageMatrixDocumentReader.java, line 3](https://github.com/datahub-project/datahub/blob/0d9b95469f4b98109939b0c97826fa3787926604/metadata-service/factories/src/main/java/com/linkedin/gms/factory/ingestion/ObjectStorageMatrixDocumentReader.java#L3)
- **Mixpanel**: 1 file · [metadata-ingestion/src/datahub/telemetry/telemetry.py, line 15](https://github.com/datahub-project/datahub/blob/0d9b95469f4b98109939b0c97826fa3787926604/metadata-ingestion/src/datahub/telemetry/telemetry.py#L15)
- **[Sentry](https://seerepo.com/uses/sentry)**: 1 file · [metadata-ingestion/src/datahub/telemetry/telemetry.py, line 228](https://github.com/datahub-project/datahub/blob/0d9b95469f4b98109939b0c97826fa3787926604/metadata-ingestion/src/datahub/telemetry/telemetry.py#L228)

## Support (/)

### The base

- **pyspark**: 5 files
- **[AIOHTTP](https://seerepo.com/uses/aiohttp)**: 1 file
- **[anyio](https://seerepo.com/uses/anyio)**: 1 file

### Content

- **velocity-engine-core**
- **[beautifulsoup4](https://seerepo.com/uses/beautifulsoup4)**: 1 file

### Validation

- **[Pydantic](https://seerepo.com/uses/pydantic)**: 359 files
- **[jsonschema](https://seerepo.com/uses/jsonschema)**: 2 files
- **io.swagger.parser.v3:swagger-parser**: 1 file

### Testing

- **Pytest**: 1,224 files

### Deploy

- **[log4j-core](https://seerepo.com/uses/log4j-core)**
- **[loguru](https://seerepo.com/uses/loguru)**
- **org.apache.logging.log4j:log4j-jul**
- **[python-json-logger](https://seerepo.com/uses/python-json-logger)**
- **slf4j-log4j12**
- **[org.slf4j:slf4j-api](https://seerepo.com/uses/orgslf4j-slf4j-api)**: 55 files
- **software.amazon.awssdk:regions**: 7 files
- **[psutil](https://seerepo.com/uses/psutil)**: 2 files
- **[ch.qos.logback:logback-classic](https://seerepo.com/uses/chqoslogback-logback-classic)**: 1 file
- **[ch.qos.logback:logback-core](https://seerepo.com/uses/chqoslogback-logback-core)**: 1 file
- **[docker](https://seerepo.com/uses/docker)**: 1 file

### Tools

- **com.google.googlejavaformat:google-java-format**
- **mypy**
- **pip**
- **Ruff**
- **org.projectlombok:lombok**: 1,728 files
- **typing-extensions**: 118 files
- **click**: 54 files
- **packaging**: 11 files
- **typing-inspect**: 3 files

### Other

- **aircompressor**
- **com.google.guava:guava**
- **commons-beanutils**
- **commons-collections**
- **commons-io**
- **dnsjava**
- **httpcore5**
- **jackson-annotations**
- **jackson-dataformat-cbor**
- **jaxb-api**
- **jaxb-runtime**
- **jline**
- **json-patch**
- **netty-all**
- **netty-codec**
- **netty-codec-smtp**
- **netty-handler**
- **netty-resolver-dns-native-macos**
- **netty-transport-native-epoll**
- **netty-transport-native-kqueue**
- **netty-transport-native-unix-common**
- **okhttp**
- **org.apache.commons:commons-configuration2**
- **pyyaml**
- **reactor-netty-core**
- **reactor-netty-http**
- **com.fasterxml.jackson.core:jackson-databind**: 279 files
- **requests**: 111 files
- **com.fasterxml.jackson.core:jackson-core**: 83 files
- **jakarta.annotation:jakarta.annotation-api**: 33 files
- **urllib3**: 31 files
- **python-dateutil**: 15 files
- **humanfriendly**: 13 files
- **com.fasterxml.jackson.dataformat:jackson-dataformat-yaml**: 10 files
- **tenacity**: 10 files
- **avro**: 8 files
- **ruamel.yaml**: 8 files
- **tabulate**: 6 files
- **jsonref**: 4 files
- **cached-property**: 3 files
- **deprecated**: 3 files
- **ijson**: 2 files
- **info.picocli:picocli**: 2 files
- **org.apache.commons:commons-compress**: 2 files
- **[Apache](https://seerepo.com/uses/apache)**: 1 file
- **cramjam**: 1 file
- **net.objecthunter:exp4j**: 1 file
- **org.xerial.snappy:snappy-java**: 1 file
- **python-dotenv**: 1 file
- **requests-file**: 1 file
- **toml**: 1 file

## Frontend (datahub-web-react)

### The base

- **[React](https://seerepo.com/uses/react)**: 2,380 files · [in datahub-web-react/package.json](https://github.com/datahub-project/datahub/blob/0d9b95469f4b98109939b0c97826fa3787926604/datahub-web-react/package.json)
- **[react-router-dom](https://seerepo.com/uses/react-router-dom)**: 167 files
- **[React Router](https://seerepo.com/uses/react-router)**: 129 files
- **[history](https://seerepo.com/uses/history)**: 5 files
- **[Vite](https://seerepo.com/uses/vite)**: 3 files · [in datahub-web-react/package.json](https://github.com/datahub-project/datahub/blob/0d9b95469f4b98109939b0c97826fa3787926604/datahub-web-react/package.json)

### Styles

- **[styled-components](https://seerepo.com/uses/styled-components)**: 1,345 files
- **[Ant Design](https://seerepo.com/uses/ant-design)**: 9 files

### Interface

- **[antd](https://seerepo.com/uses/antd)**: 410 files
- **[reactflow](https://seerepo.com/uses/reactflow)**: 33 files
- **visx**: 25 files
- **@remirror/react**: 23 files
- **[react-intersection-observer](https://seerepo.com/uses/react-intersection-observer)**: 16 files
- **[dnd-kit](https://seerepo.com/uses/dnd-kit)**: 15 files
- **[@monaco-editor/react](https://seerepo.com/uses/monaco-editor-react)**: 5 files
- **rc-table**: 5 files
- **[react-helmet-async](https://seerepo.com/uses/react-helmet-async)**: 3 files
- **react-js-cron**: 3 files
- **[react-color](https://seerepo.com/uses/react-color)**: 2 files
- **[@floating-ui/react-dom](https://seerepo.com/uses/floating-ui-react-dom)**: 1 file
- **[D3](https://seerepo.com/uses/d3)**: 1 file
- **react-diff-viewer**: 1 file
- **[react-syntax-highlighter](https://seerepo.com/uses/react-syntax-highlighter)**: 1 file
- **[react-window](https://seerepo.com/uses/react-window)**: 1 file
- **virtualizedtableforantd4**: 1 file

### Icons

- **[@phosphor-icons/react](https://seerepo.com/uses/phosphor-icons-react)**: 588 files
- **@ant-design/icons**: 2 files

## Support (datahub-web-react)

### Content

- **[dompurify](https://seerepo.com/uses/dompurify)**: 8 files
- **[js-yaml](https://seerepo.com/uses/js-yaml)**: 3 files
- **[html-to-image](https://seerepo.com/uses/html-to-image)**: 1 file
- **[mammoth](https://seerepo.com/uses/mammoth)**: 1 file
- **[marked](https://seerepo.com/uses/marked)**: 1 file
- **turndown-plugin-gfm**: 1 file

### Testing

- **Vitest**: 448 files
- **Testing Library**: 396 files

### Deploy

- **opentelemetry**: 1 file

### Tools

- **sql-formatter**: 1 file

### Other

- **react-i18next**: 1,024 files
- **i18next**: 146 files
- **Lodash**: 67 files
- **@apollo/client**: 53 files
- **query-string**: 52 files
- **GraphQL**: 25 files
- **uuid**: 8 files
- **yamljs**: 8 files
- **js-cookie**: 6 files
- **color-hash**: 4 files
- **deepmerge**: 3 files
- **[OpenTelemetry](https://seerepo.com/uses/opentelemetry)**: 3 files
- **colorthief**: 2 files
- **dayjs**: 2 files
- **diff**: 2 files
- **fuse.js**: 2 files
- **web-vitals**: 2 files
- **analytics**: 1 file
- **cronstrue**: 1 file
- **graphql-tag**: 1 file
- **node-sql-parser**: 1 file

## Other repos with a similar stack

- [apache/superset](https://seerepo.com/r/apache/superset/built): Google, AIOHTTP, anyio, React, beautifulsoup4, Pydantic, jsonschema, python-json-logger, psutil, docker, Ant Design, antd, visx, react-js-cron, D3, react-syntax-highlighter, react-window, dompurify · [Compare](https://seerepo.com/compare/apache/superset/vs/datahub-project/datahub)
- [langflow-ai/langflow](https://seerepo.com/r/langflow-ai/langflow/built): FastAPI, Google, AIOHTTP, React, react-router-dom, Vite, Pydantic, jsonschema, loguru, psutil, reactflow, react-syntax-highlighter, dompurify · [Compare](https://seerepo.com/compare/datahub-project/datahub/vs/langflow-ai/langflow)
- [datawhalechina/hello-agents](https://seerepo.com/r/datawhalechina/hello-agents/built): FastAPI, AIOHTTP, anyio, React, react-router-dom, Vite, beautifulsoup4, Pydantic, jsonschema, loguru, D3, dompurify, marked · [Compare](https://seerepo.com/compare/datahub-project/datahub/vs/datawhalechina/hello-agents)
- [bojieli/ai-agent-book](https://seerepo.com/r/bojieli/ai-agent-book/built): FastAPI, Google, AIOHTTP, anyio, React, Vite, beautifulsoup4, Pydantic, loguru, psutil, D3, react-syntax-highlighter · [Compare](https://seerepo.com/compare/bojieli/ai-agent-book/vs/datahub-project/datahub)
- [apache/airflow](https://seerepo.com/r/apache/airflow/built): FastAPI, Google, AIOHTTP, React, react-router-dom, Vite, beautifulsoup4, Pydantic, jsonschema, org.slf4j:slf4j-api, psutil, docker · [Compare](https://seerepo.com/compare/apache/airflow/vs/datahub-project/datahub)
- [Significant-Gravitas/AutoGPT](https://seerepo.com/r/Significant-Gravitas/AutoGPT/built): FastAPI, Google, Sentry, AIOHTTP, React, beautifulsoup4, Pydantic, jsonschema, psutil, docker, react-window · [Compare](https://seerepo.com/compare/datahub-project/datahub/vs/Significant-Gravitas/AutoGPT)

## Frequently asked questions

### What language is datahub written in?

datahub is written in Python, with 2,991,794 written lines.

### What framework does datahub use?

datahub is built with FastAPI, Spring, pyspark, AIOHTTP, anyio, React, react-router-dom, React Router, history, and Vite.

### What database does datahub use?

datahub stores its data in org.springframework.kafka:spring-kafka and org.apache.iceberg:iceberg-core.

### What does datahub sign people in with?

datahub signs people in with org.springframework.security:spring-security-oauth2-jose, software.amazon.awssdk:sts, and com.google.auth:google-auth-library-oauth2-http.

### Does datahub use AI?

Yes. datahub uses AI through software.amazon.awssdk:bedrockruntime.

### What services does datahub use?

datahub stores files in AWS S3, watches its errors with Sentry, connects to Google, and uses s3, spark-sql_2.12, org.apache.iceberg:iceberg-aws, Google Cloud Storage, and Mixpanel.

The proof: [metadata-service/factories/src/main/java/com/linkedin/gms/factory/objectstorage/ObjectStorageClientFactory.java, line 23](https://github.com/datahub-project/datahub/blob/0d9b95469f4b98109939b0c97826fa3787926604/metadata-service/factories/src/main/java/com/linkedin/gms/factory/objectstorage/ObjectStorageClientFactory.java#L23)

### Does datahub have tests?

Yes. datahub has 5,307 test files and is tested with Pytest, Vitest, and Testing Library.

### Which repos have a stack like datahub's?

The ones that share most of its stack: apache/superset, langflow-ai/langflow, and datawhalechina/hello-agents.

## What is your repo built with?

Any public repo, with no account: https://seerepo.com/r/owner/repo/built, with the owner and name it has on GitHub.