# GenieX

> Run LLMs and VLMs locally on Qualcomm Snapdragon NPU, GPU, or CPU

- Category: Models & Inference / Local Runtimes
- Language: Rust, C++, Python
- Licence: BSD-3-Clause
- Repository: https://github.com/qualcomm/GenieX
- Website: https://geniex.aihub.qualcomm.com
- Canonical page: https://ai-tldr.dev/tools/nexa-sdk/

## In short

GenieX is Run LLMs and VLMs locally on Qualcomm Snapdragon NPU, GPU, or CPU. It sits in the Local Runtimes category of the Models & Inference tools and is open-source, written in Rust, C++, Python and licensed BSD-3-Clause. To get started: install the CLI.

## Overview

GenieX is Qualcomm's on-device generative AI inference runtime, published under the qualcomm/GenieX repository as the community version of Qualcomm GENIE. It was previously developed as the Nexa SDK. Point it at almost any GGUF model from Hugging Face, or at a pre-compiled bundle from Qualcomm AI Hub, and it runs locally on the Hexagon NPU, the Adreno GPU, or the CPU.

One C SDK sits underneath, exposed through a CLI, a Python package, Kotlin/Java bindings for Android, Docker, and an OpenAI-compatible local server. Under the hood it dispatches either to the llama.cpp runtime (GGML kernels over CPU, GPU, or Hexagon HTP) or to the Qualcomm AI Engine Direct runtime on the NPU, depending on the model format you hand it.

It runs only on Qualcomm Snapdragon hardware: Windows ARM64 compute devices (Snapdragon X and X Elite), Android phones (Snapdragon 8 Elite and 8 Elite Gen 5), and Linux ARM64 IoT boards such as Dragonwing QCS9075. The project marks itself as a developer preview and is licensed BSD-3-Clause; Qualcomm Device Cloud offers remote sessions for developers without the hardware on hand.

## What it does

- Runs GGUF models from Hugging Face or pre-compiled bundles from Qualcomm AI Hub, on the Hexagon NPU, Adreno GPU, or CPU
- One C SDK exposed through a CLI, Python, Kotlin/Java, Docker, and an OpenAI-compatible server
- Two runtimes under the hood: llama.cpp (GGML) and Qualcomm AI Engine Direct for the NPU
- Supports Windows ARM64, Android, and Linux ARM64 Snapdragon devices
- Python API mirrors Hugging Face transformers: from_pretrained() then .generate(), with streaming
- Vision-language models supported from the CLI, including dragging an image into the prompt

## Getting started

Pick the interface that matches your stack. The Python package and the CLI both take a Hugging Face GGUF repo id or a Qualcomm AI Hub bundle id.

1. Install the CLI

   On Windows ARM64, download the installer from the GitHub releases page and open a new terminal. On Linux ARM64 the install script needs no sudo.

```bash
curl -fsSL https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-geniex/install.sh | sh
```

2. Chat with a model from the CLI

   One line per model. A GGUF repo goes through llama.cpp; a Qualcomm AI Hub bundle goes through Qualcomm AI Engine Direct on the NPU.

```bash
# GGUF from Hugging Face → llama.cpp (NPU / GPU / CPU)
geniex infer google/gemma-4-E4B-it-qat-q4_0-gguf

# Pre-compiled bundle from Qualcomm AI Hub → AI Engine Direct (NPU)
geniex infer ai-hub-models/Qwen2.5-VL-7B-Instruct
```

3. Install the Python package

   The Python interface is available on PyPI for Windows ARM64 and Linux ARM64.

```bash
pip install geniex
```

4. Generate text in Python

   The API mirrors Hugging Face transformers: load with from_pretrained(), apply the chat template, then stream tokens from generate().

```python
from geniex import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("unsloth/Qwen3.5-2B-GGUF", precision="Q4_0")

messages = [{"role": "user", "content": "What is 2+2?"}]
prompt = model.tokenizer.apply_chat_template(messages, add_generation_prompt=True)

for chunk in model.generate(prompt, max_new_tokens=256, stream=True):
    print(chunk, end="", flush=True)

model.close()
```

## When to use it

- Add an offline chat or assistant feature to a Snapdragon laptop or Android app without calling a hosted API
- Take advantage of the Hexagon NPU for low-power local inference on Windows ARM64 or Android
- Run vision-language models on-device for image understanding without uploading the image
- Serve a local OpenAI-compatible endpoint on a Snapdragon device so existing client code keeps working