Antoine Zambelli · 2026-05-19 · notable
Forge — Open-Source Guardrails Lift a Self-Hosted 8B Local Model From 53% to 99% on Multi-Step Agentic Tool-Calling Workflows
Antoine Zambelli's Python framework wraps Ollama, llama-server, and Llamafile agents with rescue parsing, retry nudges, and step enforcement. Show HN hit 293 points; an ACM CAIS '26 demo accompanies the eval suite.
Drop-in reliability layer that pushes 8B local models toward frontier-API accuracy on tool-calling agents.
Key specs
| GitHub stars | 405 |
|---|---|
| Show hn points | 293 |
| Show hn comments | 110 |
| Eval scenarios | 26 |
| Top self hosted score | 86.5% |
| Hardest tier score | 76% |
What is it?
Forge is a small Python framework that sits in front of a self-hosted LLM backend (Ollama, llama.cpp's llama-server, Llamafile, or the Anthropic API) and makes its tool-calling more reliable. The author, Antoine Zambelli, published the project today on Hacker News with the claim that guardrails alone move a Ministral-3 8B model from roughly 53% to ~99% accuracy on a 26-scenario agentic eval. The framework also has a companion demo paper at ACM CAIS '26.
How does it work?
Three guardrail layers wrap each tool call: rescue parsing recovers malformed JSON / tool invocations across model output formats, retry nudges feed errors back so the model can self-correct, and step enforcement gates the workflow on prerequisite conditions. A separate context-management layer enforces VRAM-aware budgets with tiered compaction so long agent loops stay inside hardware limits. Users plug Forge in via a WorkflowRunner, composable middleware, or a proxy server that sits between a client and the local model.
Why does it matter?
Self-hosted 8B models normally cascade-fail on multi-step agents: 90% per-step accuracy still produces a 40% failure rate over five steps. Closing that gap on consumer hardware means privacy-sensitive workflows — code review, incident response, scraping pipelines — can run locally instead of paying frontier API per token. The eval also exposes a 75-point spread between backends on the same weights, which is a load-bearing finding for anyone choosing a server.
Who is it for?
indie devs and researchers running local agents on consumer GPUs
Try it
pip install forge-guardrails