RubyLLM

// Overview

RubyLLM is the AI framework for people whose application is already Ruby. Most LLM tooling assumes Python or TypeScript, which leaves Rails teams writing HTTP clients by hand or shelling out to a sidecar service. RubyLLM puts chat, tools, agents, files, embeddings, reranking, moderation and media generation behind one consistent Ruby API — `RubyLLM.chat.ask`, `RubyLLM.embed`, `RubyLLM.paint`, `RubyLLM.transcribe` — so provider differences stay inside the gem.

Seventeen providers are supported behind that single API, including OpenAI, Azure, Anthropic, Gemini, VertexAI, Bedrock, xAI, Cohere, DeepSeek, Mistral, Perplexity, Ollama and Ollama Cloud, OpenRouter, GPUStack, ElevenLabs, Deepgram, and any OpenAI-compatible endpoint. Moving between a hosted and a local model is a model string, not a rewrite. A built-in model registry exposes each model's capabilities, limits and pricing, and a per-attempt usage ledger sits behind `chat.tokens` and `chat.cost` so spend is visible without bolting on a separate observability layer.

The Rails integration is the part that distinguishes it from a thin API wrapper. Generators scaffold a working chat UI, conversations persist through Active Record with `acts_as_chat`, attachments go through Active Storage, and replies stream over Hotwire. Beyond the basics it covers the agentic loop (`ask_later`, `step`, `complete?`), tool approval gates via `requires_approval`, provider-side tools such as web search and MCP connectors, structured output through Ruby schema classes, prompt caching, fallbacks and cancellation, batches, compaction and extended thinking. MIT licensed; the examples in the README target release 2.0.0.

// What it does

• One Ruby API across 17 providers — hosted, local via Ollama, or any OpenAI-compatible endpoint
• Chat, vision, audio transcription and speech, OCR, image and video generation, embeddings, reranking and moderation
• Tools as plain Ruby classes, reusable `RubyLLM::Agent` definitions, and `requires_approval` human gates
• Rails-native: `acts_as_chat` Active Record persistence, Active Storage attachments, Hotwire streaming, scaffold generators
• Structured output from Ruby schema classes, read back with `response.parsed`
• Cost and token ledger behind `chat.tokens` / `chat.cost`, plus a model registry with limits and pricing
• Prompt caching, model fallbacks, cancellation, batches, compaction and extended thinking

// Getting started

The commands below follow the README's 2.0.0 examples. If your app is on 1.16, read the upgrade guide before deploying 2.0.

bundle add ruby_llm --version 2.0.0

require 'ruby_llm'

RubyLLM.configure do |config|
  config.openai_api_key = ENV.fetch('OPENAI_API_KEY')
end

RubyLLM.chat.ask "What's the best way to learn Ruby?"

chat = RubyLLM.chat(model: "gemini-3.7-flash")
chat.ask "What's in this image?", with: "ruby_conf.jpg"

class Weather < RubyLLM::Tool
  description "Get current weather"

  def execute(latitude:, longitude:)
    url = "https://api.open-meteo.com/v1/forecast?latitude=#{latitude}&longitude=#{longitude}&current=temperature_2m,wind_speed_10m"
    JSON.parse(Faraday.get(url).body)
  end
end

chat.with_tools(Weather).ask "What's the weather in Berlin?"

bin/rails generate ruby_llm:install
bin/rails db:migrate
bin/rails ruby_llm:load_models

# optional chat UI
bin/rails generate ruby_llm:chat_ui

class Chat < ApplicationRecord
  acts_as_chat
end

chat = Chat.create! model: "gpt-5.6-luna"
chat.ask "What's in this file?", with: "report.pdf"

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Add AI features to an existing Rails app without standing up a Python service beside it
• Move between hosted and local models — or swap providers on a price change — without rewriting call sites
• Ship a persisted, streaming chat interface from generators rather than building one
• Track per-conversation token spend inside the app that made the calls

// How RubyLLM compares

RubyLLM alongside other open-source app frameworks tools AI/TLDR tracks, ranked by GitHub stars.