ODS

// Overview

ODS — the Osmantic Deployment System — installs and wires together the pieces of a private AI stack so you do not have to assemble them by hand. The problem it addresses is one most self-hosters recognise: running AI locally means standing up an inference server, a chat UI, an automation engine, an image-generation tool and the privacy and auth plumbing around them, then keeping that assembly working. ODS treats homelab AI setup as a solved problem and ships the solution as a single installer.

What you get after one command is a running stack: local model inference on your own hardware, a ChatGPT-style web UI reachable from any browser, and a control dashboard for managing models, services, setup, GPU status and extensions from one place. On top of that sit voice, agents and workflows for building automations that can listen, speak and call tools; RAG and search over local documents and private search; and local image generation. Service auth, secrets, observability and diagnostics stay in the same local stack.

The privacy position is explicit: no cloud is required and no subscription is required, and your prompts and data stay on your machine unless you choose otherwise — cloud and hybrid API modes exist but are opt-in. The project takes release engineering seriously for something in this category, validating operational changes against a fleet and distro lab covering zero-prereq bootstrap, fresh installs, product flows, lifecycle recovery and a final user-facing gate. `main` moves quickly; for forks, appliances, labs or production-like installs the maintainers tell you to pin a tagged release or an audited commit and keep your own validation receipt.

// What it does

• One installer that provisions and wires the whole stack, rather than leaving you to integrate the pieces yourself
• Local model inference on your own hardware, with a ChatGPT-style web UI reachable from any browser on the network
• A control dashboard for models, services, setup, GPU status and extensions in one place
• Voice, agents and workflows for automations that can listen, speak, call tools and complete work
• RAG and search over local documents plus private search and retrieval workflows
• Local image generation, so prompts are not sent to a hosted API
• Local service auth, secrets, observability and diagnostics; cloud and hybrid API modes are optional
• Release channels with a validated stable line, plus documented installer-trust and forkability guidance

// Getting started

ODS requires Docker to be installed and running — on Windows, Docker Desktop with the WSL2 backend. The installer picks a model for your hardware, starts the services and hands you the local web UI.

curl -fsSL https://install.osmantic.com/ods.sh | bash

$ProgressPreference = "SilentlyContinue"
$odsSrc = Join-Path $env:TEMP ("ods-install-" + [guid]::NewGuid().ToString("N"))
$odsZip = Join-Path $odsSrc "ods-main.zip"
New-Item -ItemType Directory -Path $odsSrc | Out-Null
Invoke-WebRequest "https://github.com/Osmantic/ODS/archive/refs/heads/main.zip" -OutFile $odsZip
Expand-Archive -LiteralPath $odsZip -DestinationPath $odsSrc -Force
cd (Get-ChildItem -LiteralPath $odsSrc -Directory | Select-Object -First 1).FullName
Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
.\install.ps1

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Stand up a private AI server on a spare desktop or homelab box without integrating inference, a UI and automation tooling by hand
• Keep prompts and documents on your own hardware while still getting chat, RAG, voice and image generation
• Give a household or small team one local endpoint and web UI for models running on a shared machine
• Build local automations that combine an LLM with voice and tool calls, managed from a single dashboard

// How ODS compares

ODS alongside other open-source local runtimes tools AI/TLDR tracks, ranked by GitHub stars.