# Miasma

> A small Rust server that traps AI web scrapers in an endless loop of poisoned training data and self-referential links

- Category: Data & Labeling / Web Scraping & Crawling
- Language: Rust
- Licence: GPL-3.0
- Repository: https://github.com/austin-weeks/miasma
- Website: https://austin-weeks.github.io/miasma
- Canonical page: https://ai-tldr.dev/tools/miasma/

## In short

Miasma is the defensive side of web crawling: a self-hosted server you point misbehaving scraper traffic at. Every page it serves contains poisoned training data proxied from the "poison fountain" (rnsaffn.com) plus several self-referential links, so a crawler that follows them keeps fetching more of the same. It sits in the Web Scraping & Crawling category of the Data & Labeling tools and is open-source, written in Rust and licensed GPL-3.0.

## Overview

Miasma is the defensive side of web crawling: a self-hosted server you point misbehaving scraper traffic at. Every page it serves contains poisoned training data proxied from the "poison fountain" (rnsaffn.com) plus several self-referential links, so a crawler that follows them keeps fetching more of the same. Its README describes it as "an endless buffet of slop for the slop machines", aimed at site owners who do not want AI companies harvesting their work as training data.

The trap is wired up with ordinary web plumbing. You add a few hidden links on your site (with `display: none`, `aria-hidden` and `tabindex="-1"` so humans, screen readers and keyboard users never see them), route that path through your reverse proxy to Miasma, and disallow the same path in `robots.txt` so well-behaved bots and search engines stay out. Only crawlers that ignore `robots.txt` and follow invisible links end up inside, where Miasma's links keep them circling back through the proxy.

Miasma is written in Rust, licensed GPL-3.0, and built to be fast with a minimal memory footprint — memory scales with the `max-in-flight` limit, and the README's worked example expects 50–60 MB peak at 50 connections. It installs with `cargo install miasma`, as a pre-built binary, as the official `austinweeks/miasma` Docker image, or as a library inside an existing Rust server. The project warns that deploying it carries inherent risk: it is not affiliated with the poison fountain, cannot vouch for that source's contents, and you should never direct real users to your Miasma path.

## What it does

- Serves poisoned training data from a configurable `poison-source` (the poison fountain by default), with response caching that can be switched off
- Each response includes a configurable number of self-directing links (`link-count`) under your chosen `link-prefix`, so scrapers loop back through your proxy
- `max-depth` cuts scrapers loose after a set depth, letting you cap how much poison — and egress — each one costs you
- `max-in-flight` bounds concurrent requests and answers overflow with an immediate 429; `force-gzip` compresses every response to cut egress
- Optional per-User-Agent request metrics stored in a local SQLite file and served at a basic-auth-protected endpoint
- Configured by CLI flags or a YAML, TOML or JSON config file; HTML from the poison source is escaped by default

## Getting started

The README walks through trapping scrapers on a path called `/naughty-bots` behind an Nginx reverse proxy; the same result works with other setups.

1. Install Miasma

   Cargo is the recommended route; pre-built binaries are on the GitHub releases page and community packages exist for several package managers.

```bash
cargo install miasma

# start with the default configuration
miasma

# list every configuration option
miasma --help
```

2. Or run the Docker image

   The official image takes the same flags as the local binary.

```bash
docker run --rm -p 9999:9999 austinweeks/miasma:latest \
    --link-prefix '/naughty-bots' \
    --max-in-flight 30
```

3. Embed hidden links in your site

   These attributes keep the link invisible to human visitors and ignored by screen readers and keyboard navigation — only scrapers see it.

```text
<a
  href="/naughty-bots/"
  style="display: none;"
  aria-hidden="true"
  tabindex="-1"
>
  Amazing high quality data here!
</a>
```

4. Proxy the trap path to Miasma

   Route every `/naughty-bots` variant to Miasma (here on port 9855), with aggressive per-User-Agent rate limiting so you don't DDoS yourself.

```text
http {
  # Reserve 8MB memory for tracking user agents
  limit_req_zone $http_user_agent zone=miasma:8m rate=1r/s;

  server {
    location = /naughty-bots {
      port_in_redirect off;
      return 301 /naughty-bots/;
    }
    location /naughty-bots/ {
      # Rate limit via the 'miasma' zone with no queueing
      limit_req_status 429;
      limit_req zone=miasma burst=5 nodelay;

      # Proxy requests to Miasma
      proxy_pass http://localhost:9855/;
    }
  }
}
```

5. Start Miasma with the matching prefix

   50 in-flight connections, forced gzip, and five links per page down to depth 8 — the README estimates scrapers are released after ~100K poisoned pages, around 250 MB each.

```bash
miasma --link-prefix '/naughty-bots' -p 9855 -c 50 --force-gzip --link-count 5 --max-depth 8
```

6. Protect well-behaved bots in robots.txt

   Disallow the trap path so search engines and compliant crawlers never enter it.

```text
User-agent: *
Disallow: /naughty-bots
```

## When to use it

- Make AI training crawlers that ignore robots.txt pay for scraping a personal site or blog by feeding them poisoned pages instead of your content
- Identify which bots hit a site hardest using Miasma's per-User-Agent request metrics
- Add a scraper tarpit to an existing Rust web server by using Miasma as a library
- Run a low-memory trap alongside an Nginx (or other reverse-proxy) deployment with bounded egress via max-depth and forced gzip