Is Mesh LLM free to run, and can I use it commercially?

Mesh LLM is Apache-2.0 licensed, so commercial and private use is permitted at no cost. The public mesh is free to join for community compute; you can also run a private mesh among your own boxes without touching the public network, and the launch post is explicit that no central server or paid tier is required.

How is Mesh LLM different from Petals or a vLLM cluster?

Mesh LLM ships as one 18 MB Rust binary with iroh baked in, so peers connect over authenticated QUIC without a central rendezvous server — the launch post frames this as the main split from Petals's Bittorrent-style hub model. Split mode partitions a model by layer ranges across nodes, matching vLLM's pipeline parallel, but Mesh LLM handles peer discovery and version compatibility itself instead of needing a Ray cluster.

What models are in the Mesh LLM catalog at launch?

Mesh LLM ships with 40+ models spanning 500M to 235B parameters. Small models fit on a laptop and run in local mode; the 235B mixture-of-experts entries require Skippy split mode, where layers 0–15 land on one node, 16–31 on the next, and activations flow along the pipeline.

Do I need a public IP address to join a mesh?

No. iroh's hole-punching and relay fallback let Mesh LLM connect two machines behind separate NATs without either exposing an inbound port. Peers are addressed by public key, not by IP, so the transport works from a laptop on hotel Wi-Fi as well as from a dedicated server.

Is my code and data private when Mesh LLM routes a request to a peer?

iroh gives each peer a public-key identity and encrypts every hop with QUIC. The launch post is careful to note that the public mesh is a shared trust boundary — any peer running the model can see the prompt — so sensitive workloads should be pinned to a private mesh of machines the owner controls.

Mesh LLM · 2026-07-11 · major

Mesh LLM — distributed inference on iroh's peer-to-peer network

Mesh LLM pools GPUs across machines and exposes them as one OpenAI-compatible endpoint. Rae McKelvey's launch post details three run modes — local, peer routing, and a split pipeline for models too big for any single box.

Mesh LLM GitHub repo social card — distributed AI/LLM for the people

Pool GPUs across your machines and serve any of 40+ models through one OpenAI-compatible endpoint at localhost:9337.

Key specs

GitHub stars	1,334
Install size	~18 MB
Catalog size	40+ models

Quick facts

Maker	Mesh LLM team, launched on iroh.computer
License	Apache-2.0
Language	Rust
API	OpenAI-compatible, localhost:9337/v1
Model catalog	40+ models, 500M–235B parameters
Run modes	Local GPU, peer routing, Skippy pipeline split
Transport	iroh QUIC with NAT traversal and peer keys

What is it?

Mesh LLM is an Apache-2.0 Rust binary that turns whatever mix of GPU and CPU boxes you already own into one shared inference cluster. The July 11 launch post from Rae McKelvey introduces the daemon, the CLI, and a public mesh users can join or leave; the whole install is about 18 MB and exposes an OpenAI-compatible API at localhost:9337/v1.

How does it work?

Every request goes through one of three paths inside Mesh LLM. Local mode runs the model on the machine's own GPU. Peer routing forwards the call to another mesh node that already has the model loaded. Split mode — internally 'Skippy' — partitions a model too big for any single box by layer ranges, so activations pipeline from stage to stage across several modest machines. iroh handles the QUIC transport, NAT traversal, and authenticated peer identity underneath.

Why does it matter?

For teams that already own idle GPUs, Mesh LLM replaces the pay-per-token bill with a p2p pool that speaks the same OpenAI SDK. The 235B mixture-of-experts models the catalog supports are impossible to load on a single consumer card, and Skippy makes a 4-machine setup a viable alternative to a rented H100 without changing client code.

Who is it for?

Self-hosters, small labs, and startups with unused GPU capacity.

Frequently asked questions

Is Mesh LLM free to run, and can I use it commercially?: Mesh LLM is Apache-2.0 licensed, so commercial and private use is permitted at no cost. The public mesh is free to join for community compute; you can also run a private mesh among your own boxes without touching the public network, and the launch post is explicit that no central server or paid tier is required.
How is Mesh LLM different from Petals or a vLLM cluster?: Mesh LLM ships as one 18 MB Rust binary with iroh baked in, so peers connect over authenticated QUIC without a central rendezvous server — the launch post frames this as the main split from Petals's Bittorrent-style hub model. Split mode partitions a model by layer ranges across nodes, matching vLLM's pipeline parallel, but Mesh LLM handles peer discovery and version compatibility itself instead of needing a Ray cluster.
What models are in the Mesh LLM catalog at launch?: Mesh LLM ships with 40+ models spanning 500M to 235B parameters. Small models fit on a laptop and run in local mode; the 235B mixture-of-experts entries require Skippy split mode, where layers 0–15 land on one node, 16–31 on the next, and activations flow along the pipeline.
Do I need a public IP address to join a mesh?: No. iroh's hole-punching and relay fallback let Mesh LLM connect two machines behind separate NATs without either exposing an inbound port. Peers are addressed by public key, not by IP, so the transport works from a laptop on hotel Wi-Fi as well as from a dedicated server.
Is my code and data private when Mesh LLM routes a request to a peer?: iroh gives each peer a public-key identity and encrypts every hop with QUIC. The launch post is careful to note that the public mesh is a shared trust boundary — any peer running the model can see the prompt — so sensitive workloads should be pinned to a private mesh of machines the owner controls.

Try it

curl https://meshllm.cloud/install.sh | sh (Linux/macOS); see the README on github.com/Mesh-LLM/mesh-llm.