# Unstructured

> Open-source preprocessing that turns messy documents into clean elements for LLMs

- Category: Retrieval & Knowledge (RAG) / Parsing & Ingestion
- Language: Python
- Licence: Apache-2.0
- Repository: https://github.com/Unstructured-IO/unstructured
- Website: https://www.unstructured.io/
- Canonical page: https://ai-tldr.dev/tools/unstructured/

## In short

Unstructured is an open-source Python library for ingesting and preprocessing images and text documents. It handles many file types, including PDFs, HTML, Word docs, and more, and turns them into structured outputs you can feed to a language model. It sits in the Parsing & Ingestion category of the Retrieval & Knowledge (RAG) tools and is open-source, written in Python and licensed Apache-2.0.

## Overview

Unstructured is an open-source Python library for ingesting and preprocessing images and text documents. It handles many file types, including PDFs, HTML, Word docs, and more, and turns them into structured outputs you can feed to a language model.

It is aimed at developers building RAG and LLM data pipelines who need a consistent way to extract text from varied, messy source files. Its modular partition functions and connectors form one system that simplifies data ingestion and preprocessing across different platforms.

Within the parsing and ingestion category, Unstructured sits at the front of the pipeline: it does the document parsing step so your downstream chunking, embedding, and retrieval stages receive clean, normalized elements instead of raw files.

## What it does

- Partitions many document types, including PDFs, HTML, Word docs, plain text, XML, JSON, and emails
- Modular partition functions such as partition_pdf and partition_text return documents as structured elements
- Optional install extras let you add only the dependencies you need (for example unstructured[docx,pptx]) instead of all of them
- Ships official Docker images for x86_64 and Apple silicon, so you can run it in a container without local setup
- Connectors and modular functions form a cohesive system for data ingestion across different platforms
- Apache-2.0 licensed, with a hosted Unstructured Platform available for production workloads

## Getting started

Install the library from PyPI, then partition a document into structured elements in a few lines of Python.

1. Install from PyPI

   Install the SDK with support for all document types. For plain text, HTML, XML, JSON, and emails you can use the dependency-free pip install unstructured instead, or add only the extras you need.

```bash
pip install "unstructured[all-docs]"
```

2. Install system dependencies

   Depending on which document types you parse, install the supporting system packages, for example libmagic-dev (filetype detection), poppler-utils (PDFs and images), and tesseract-ocr (OCR). You may not need all of them.

3. Partition a document

   Use a partition function to read a file and return its structured elements.

```python
from unstructured.partition.pdf import partition_pdf
elements = partition_pdf(filename="example-docs/layout-parser-paper-fast.pdf")

from unstructured.partition.text import partition_text
elements = partition_text(filename="example-docs/fake-text.txt")
```

4. Or run it in a container

   Pull the official image and shell into a running container if you would rather not install dependencies locally.

```bash
docker pull downloads.unstructured.io/unstructured-io/unstructured:latest
docker run -dt --name unstructured downloads.unstructured.io/unstructured-io/unstructured:latest
docker exec -it unstructured bash
```

## When to use it

- Extracting clean text from PDFs, Word docs, and HTML to build a RAG knowledge base
- Normalizing a mixed set of file types into one structured element format before chunking and embedding
- Preprocessing scanned documents and images with OCR ahead of an LLM pipeline
- Running document ingestion inside a container as part of an automated data workflow