# Volcano

> Kubernetes-native batch scheduler for AI, big data, and HPC workloads

- Category: Models & Inference / GPU & Compute Clouds
- Language: Go
- Licence: Apache-2.0
- Repository: https://github.com/volcano-sh/volcano
- Website: https://volcano.sh
- Canonical page: https://ai-tldr.dev/tools/volcano/

## In short

Volcano is a Kubernetes-native batch scheduling system that extends the standard kube-scheduler. It sits in the GPU & Compute Clouds category of the Models & Inference tools and is open-source, written in Go and licensed Apache-2.0. To get started: install Volcano.

## Overview

Volcano is a Kubernetes-native batch scheduling system that extends the standard kube-scheduler. It adds features built for batch and elastic workloads, so jobs that need many pods to start together (gang scheduling) and to share a cluster fairly (queue management) run the way training and HPC frameworks expect.

It targets teams running AI, machine learning, and deep learning, plus bioinformatics and other big-data jobs on Kubernetes. Volcano integrates with common frameworks such as Spark, Flink, Ray, TensorFlow, PyTorch, Kubeflow, MPI, Horovod, MindSpore, and PaddlePaddle, so you keep your existing job definitions and let Volcano handle the scheduling.

In the compute-orchestration space, Volcano sits between your workloads and the cluster, deciding when and where grouped jobs run. It is an incubating project of the Cloud Native Computing Foundation (CNCF) and builds on the earlier kube-batch scheduler.

## What it does

- Gang scheduling: a multi-pod job only starts once its minimum number of members can run, avoiding partial deployments that deadlock distributed training
- Queue management for sharing a cluster across teams and workloads with priorities
- Built as a Kubernetes-native scheduler that extends and enhances the standard kube-scheduler
- Integrations with AI/big-data/HPC frameworks including Spark, Flink, Ray, TensorFlow, PyTorch, Kubeflow, MPI, Horovod, MindSpore, and PaddlePaddle
- VolcanoJob (vcjob) CRD with task definitions, lifecycle management, specific queue, and priority
- CNCF incubating project with broad adoption across cloud, finance, manufacturing, and medical sectors

## Getting started

Install Volcano into an existing Kubernetes cluster, then submit a batch job using the Volcano Job (vcjob) custom resource.

1. Install Volcano

   Apply the Volcano installer manifest to your cluster. This deploys the scheduler, controllers, and admission components.

```bash
kubectl apply -f https://raw.githubusercontent.com/volcano-sh/volcano/master/installer/volcano-development.yaml
```

2. Define a Volcano Job

   Create a vcjob manifest. minAvailable sets how many pods must be schedulable before the job runs (gang scheduling), and queue selects the scheduling queue.

```yaml
apiVersion: batch.volcano.sh/v1alpha1
kind: Job
metadata:
  name: test-job
spec:
  minAvailable: 3
  queue: default
  tasks:
    - replicas: 6
      name: "default-nginx"
      template:
        metadata:
          name: web
        spec:
          containers:
            - image: nginx
              name: nginx
              resources:
                requests:
                  cpu: "1"
          restartPolicy: OnFailure
```

3. Submit and check the job

   Apply the manifest and watch the job status.

```bash
kubectl apply -f test-job.yaml
kubectl get vcjob test-job
```

## When to use it

- Run distributed deep-learning training (TensorFlow, PyTorch, MPI/Horovod) that needs all workers to start together via gang scheduling
- Share a GPU/CPU cluster across multiple teams using queues with priorities and fair-share policies
- Schedule big-data jobs such as Spark or Flink on Kubernetes with batch-aware scheduling
- Manage HPC and bioinformatics/genomics batch workloads on a Kubernetes cluster
