cuTile Rust

// Overview

cuTile Rust is NVIDIA Labs' kernel-programming DSL for Rust. Instead of writing CUDA C++ and reasoning about threads and synchronisation by hand, you write kernels over tiles — whole blocks of a tensor — inside a `#[cutile::module]` block, and the macro embeds the Rust AST so the kernel can be JIT-compiled for the GPU.

The point of building it in Rust is the ownership model. cuTile extends Rust's borrow discipline across the GPU launch boundary: a mutable tensor is partitioned into disjoint pieces before a kernel runs, so the compiler rules out the data races that manual kernel synchronisation is famous for. On the host side the same API covers synchronous launches, asynchronous pipelines and CUDA graph replay.

It targets NVIDIA GPUs with compute capability sm_80 or higher, is developed against CUDA 13.3 and Rust 1.89+, and is tested on Linux (Ubuntu 24.04). The project is Apache-2.0 licensed with documentation at nvlabs.github.io/cutile-rs.

// What it does

• Tile-based kernel authoring — operate on blocks of a tensor rather than individual threads
• Rust ownership extended across the launch boundary, with mutable tensors partitioned into disjoint pieces
• JIT compilation through the `#[cutile::module]` macro, which embeds the Rust AST
• Host API covering synchronous launches, asynchronous pipelines and CUDA graph replay
• Data-race freedom enforced by the compiler instead of by hand-written synchronisation

// Getting started

You need a Rust toolchain (1.89+), a CUDA install (13.3 recommended) and an NVIDIA GPU of compute capability sm_80 or newer, on Linux.

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
rustup default stable

[env]
CUDA_TOOLKIT_PATH = { value = "/usr/local/cuda-13", relative = false }

cargo run -p cutile-examples --example hello_world

#[cutile::module]
mod kernel {
    use cutile::core::*;
    #[cutile::entry()]
    fn add<const B: i32>(
        z: &mut Tensor<f32, { [B] }>,
        x: &Tensor<f32, { [-1] }>,
        y: &Tensor<f32, { [-1] }>,
    ) {
        let tx = x.load_like(z);
        let ty = y.load_like(z);
        z.store(tx + ty);
    }
}

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Write custom GPU kernels from a Rust codebase without dropping to CUDA C++
• Rule out data races in kernel code at compile time rather than in a debugger
• Build asynchronous or CUDA-graph-replayed pipelines behind a safe host API
• Prototype tile-level numerical kernels for inference or training workloads

// How cuTile Rust compares

cuTile Rust alongside other open-source gpu kernels & compilers tools AI/TLDR tracks, ranked by GitHub stars.