GenieX

// Overview

GenieX is Qualcomm's on-device generative AI inference runtime, published under the qualcomm/GenieX repository as the community version of Qualcomm GENIE. It was previously developed as the Nexa SDK. Point it at almost any GGUF model from Hugging Face, or at a pre-compiled bundle from Qualcomm AI Hub, and it runs locally on the Hexagon NPU, the Adreno GPU, or the CPU.

One C SDK sits underneath, exposed through a CLI, a Python package, Kotlin/Java bindings for Android, Docker, and an OpenAI-compatible local server. Under the hood it dispatches either to the llama.cpp runtime (GGML kernels over CPU, GPU, or Hexagon HTP) or to the Qualcomm AI Engine Direct runtime on the NPU, depending on the model format you hand it.

It runs only on Qualcomm Snapdragon hardware: Windows ARM64 compute devices (Snapdragon X and X Elite), Android phones (Snapdragon 8 Elite and 8 Elite Gen 5), and Linux ARM64 IoT boards such as Dragonwing QCS9075. The project marks itself as a developer preview and is licensed BSD-3-Clause; Qualcomm Device Cloud offers remote sessions for developers without the hardware on hand.

// What it does

• Runs GGUF models from Hugging Face or pre-compiled bundles from Qualcomm AI Hub, on the Hexagon NPU, Adreno GPU, or CPU
• One C SDK exposed through a CLI, Python, Kotlin/Java, Docker, and an OpenAI-compatible server
• Two runtimes under the hood: llama.cpp (GGML) and Qualcomm AI Engine Direct for the NPU
• Supports Windows ARM64, Android, and Linux ARM64 Snapdragon devices
• Python API mirrors Hugging Face transformers: from_pretrained() then .generate(), with streaming
• Vision-language models supported from the CLI, including dragging an image into the prompt

// Getting started

Pick the interface that matches your stack. The Python package and the CLI both take a Hugging Face GGUF repo id or a Qualcomm AI Hub bundle id.

curl -fsSL https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-geniex/install.sh | sh

# GGUF from Hugging Face → llama.cpp (NPU / GPU / CPU)
geniex infer google/gemma-4-E4B-it-qat-q4_0-gguf

# Pre-compiled bundle from Qualcomm AI Hub → AI Engine Direct (NPU)
geniex infer ai-hub-models/Qwen2.5-VL-7B-Instruct

pip install geniex

from geniex import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("unsloth/Qwen3.5-2B-GGUF", precision="Q4_0")

messages = [{"role": "user", "content": "What is 2+2?"}]
prompt = model.tokenizer.apply_chat_template(messages, add_generation_prompt=True)

for chunk in model.generate(prompt, max_new_tokens=256, stream=True):
    print(chunk, end="", flush=True)

model.close()

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Add an offline chat or assistant feature to a Snapdragon laptop or Android app without calling a hosted API
• Take advantage of the Hexagon NPU for low-power local inference on Windows ARM64 or Android
• Run vision-language models on-device for image understanding without uploading the image
• Serve a local OpenAI-compatible endpoint on a Snapdragon device so existing client code keeps working

// How GenieX compares

GenieX alongside other open-source local runtimes tools AI/TLDR tracks, ranked by GitHub stars.