# LiteRT-LM — Google's Cross-Platform Edge LLM Inference Framework

> One inference framework that deploys the same LLM to phones, browsers, desktops, and Raspberry Pis.

- Org: Google
- Date: 2026-04-08
- Category: tool, repo
- Source: https://github.com/google-ai-edge/LiteRT-LM
- Canonical page: https://ai-tldr.dev/releases/google-litert-lm/

## In short

Open-source inference framework for deploying LLMs on edge devices across Android, iOS, Web, Desktop, and IoT with GPU/NPU acceleration, vision/audio input, and function calling.

## What it is

LiteRT-LM is Google's open-source inference framework for running large language models on edge devices. It provides a unified API across Android (Kotlin), iOS (Swift, in development), Web, Desktop (Python, C++), and IoT platforms like Raspberry Pi. It powers the Google AI Edge Gallery app.

## How it works

The framework handles model loading, tokenization, and hardware-accelerated inference across GPU and NPU backends. It supports multimodal inputs (vision and audio), function calling for agentic workflows, and works with models from Gemma, Llama, Phi-4, and Qwen families. Quantized models run efficiently on constrained hardware.

## Why it matters

Deploying an LLM to a phone is one problem; deploying the same model to a phone, a browser, a Raspberry Pi, and a desktop with one API is a different problem. LiteRT-LM solves the second one, removing the need to maintain separate inference stacks per platform.

## Who it's for

Mobile and IoT developers deploying LLMs to edge devices.

## Try it

pip install litert-lm

## Links

- [Repo](https://github.com/google-ai-edge/LiteRT-LM)
- [Docs](https://ai.google.dev/edge/litert-lm/overview)
- [Coverage](https://aitoolly.com/ai-news/article/2026-04-08-google-launches-litert-lm-a-high-performance-open-source-framework-for-edge-device-llm-inference)
