# RubyLLM

> One Ruby API across 17 model providers, at home in Rails

- Category: App Orchestration / App Frameworks
- Language: Ruby
- Licence: MIT
- Repository: https://github.com/crmne/ruby_llm
- Website: https://rubyllm.com/
- Canonical page: https://ai-tldr.dev/tools/ruby-llm/

## In short

RubyLLM is the AI framework for people whose application is already Ruby. Most LLM tooling assumes Python or TypeScript, which leaves Rails teams writing HTTP clients by hand or shelling out to a sidecar service. It sits in the App Frameworks category of the App Orchestration tools and is open-source, written in Ruby and licensed MIT.

## Overview

RubyLLM is the AI framework for people whose application is already Ruby. Most LLM tooling assumes Python or TypeScript, which leaves Rails teams writing HTTP clients by hand or shelling out to a sidecar service. RubyLLM puts chat, tools, agents, files, embeddings, reranking, moderation and media generation behind one consistent Ruby API — `RubyLLM.chat.ask`, `RubyLLM.embed`, `RubyLLM.paint`, `RubyLLM.transcribe` — so provider differences stay inside the gem.

Seventeen providers are supported behind that single API, including OpenAI, Azure, Anthropic, Gemini, VertexAI, Bedrock, xAI, Cohere, DeepSeek, Mistral, Perplexity, Ollama and Ollama Cloud, OpenRouter, GPUStack, ElevenLabs, Deepgram, and any OpenAI-compatible endpoint. Moving between a hosted and a local model is a model string, not a rewrite. A built-in model registry exposes each model's capabilities, limits and pricing, and a per-attempt usage ledger sits behind `chat.tokens` and `chat.cost` so spend is visible without bolting on a separate observability layer.

The Rails integration is the part that distinguishes it from a thin API wrapper. Generators scaffold a working chat UI, conversations persist through Active Record with `acts_as_chat`, attachments go through Active Storage, and replies stream over Hotwire. Beyond the basics it covers the agentic loop (`ask_later`, `step`, `complete?`), tool approval gates via `requires_approval`, provider-side tools such as web search and MCP connectors, structured output through Ruby schema classes, prompt caching, fallbacks and cancellation, batches, compaction and extended thinking. MIT licensed; the examples in the README target release 2.0.0.

## What it does

- One Ruby API across 17 providers — hosted, local via Ollama, or any OpenAI-compatible endpoint
- Chat, vision, audio transcription and speech, OCR, image and video generation, embeddings, reranking and moderation
- Tools as plain Ruby classes, reusable `RubyLLM::Agent` definitions, and `requires_approval` human gates
- Rails-native: `acts_as_chat` Active Record persistence, Active Storage attachments, Hotwire streaming, scaffold generators
- Structured output from Ruby schema classes, read back with `response.parsed`
- Cost and token ledger behind `chat.tokens` / `chat.cost`, plus a model registry with limits and pricing
- Prompt caching, model fallbacks, cancellation, batches, compaction and extended thinking

## Getting started

The commands below follow the README's 2.0.0 examples. If your app is on 1.16, read the upgrade guide before deploying 2.0.

1. Add the gem

```bash
bundle add ruby_llm --version 2.0.0
```

2. Configure a provider

   In a plain script, or in `config/initializers/ruby_llm.rb` under Rails. Add the other providers as the features you use require them.

```ruby
require 'ruby_llm'

RubyLLM.configure do |config|
  config.openai_api_key = ENV.fetch('OPENAI_API_KEY')
end
```

3. Ask something

   `with:` accepts images, video, audio, PDFs and source files on models that support them.

```ruby
RubyLLM.chat.ask "What's the best way to learn Ruby?"

chat = RubyLLM.chat(model: "gemini-3.7-flash")
chat.ask "What's in this image?", with: "ruby_conf.jpg"
```

4. Give the model your code as a tool

```ruby
class Weather < RubyLLM::Tool
  description "Get current weather"

  def execute(latitude:, longitude:)
    url = "https://api.open-meteo.com/v1/forecast?latitude=#{latitude}&longitude=#{longitude}&current=temperature_2m,wind_speed_10m"
    JSON.parse(Faraday.get(url).body)
  end
end

chat.with_tools(Weather).ask "What's the weather in Berlin?"
```

5. Wire it into Rails

   The generators create the migrations and, optionally, a working chat UI at `/chats`.

```bash
bin/rails generate ruby_llm:install
bin/rails db:migrate
bin/rails ruby_llm:load_models

# optional chat UI
bin/rails generate ruby_llm:chat_ui
```

6. Persist a conversation

```ruby
class Chat < ApplicationRecord
  acts_as_chat
end

chat = Chat.create! model: "gpt-5.6-luna"
chat.ask "What's in this file?", with: "report.pdf"
```

## When to use it

- Add AI features to an existing Rails app without standing up a Python service beside it
- Move between hosted and local models — or swap providers on a price change — without rewriting call sites
- Ship a persisted, streaming chat interface from generators rather than building one
- Track per-conversation token spend inside the app that made the calls