Kubeflow

// Overview

Kubeflow is a collection of open-source, Kubernetes-native projects that cover the stages of the AI lifecycle, from notebooks and pipelines to model training, tuning, and serving. It is described by its maintainers as the foundation of tools for building AI platforms on Kubernetes.

It is built for AI practitioners, platform administrators, and development teams who already run on Kubernetes and want a common set of tools for machine learning work. You can adopt a single project on its own or deploy the full reference platform.

Within the MLOps and compute-orchestration space, Kubeflow acts as a modular, composable layer on top of Kubernetes. Each component (Pipelines, Notebooks, Katib, Trainer, Spark Operator, Model Registry, KServe) addresses one part of the workflow and can be used independently or together.

// What it does

• Composable set of Kubernetes-native projects that can be used independently or as one reference platform
• Kubeflow Pipelines for building and running multi-step ML workflows
• Kubeflow Notebooks for in-cluster development environments
• Kubeflow Trainer and Katib for distributed training and hyperparameter tuning
• Model Registry and KServe integration for tracking and serving models
• Central Dashboard and Profile Controller for multi-tenant platform management

// Getting started

The README presents Kubeflow as an umbrella of projects rather than a single binary; the simplest way to try the full reference platform is the community distribution, which installs into a local Kind cluster. You can also install individual components on their own.

git clone https://github.com/kubeflow/community-distribution.git
cd community-distribution

./tests/install_KinD_create_KinD_cluster_install_kustomize.sh
kind get kubeconfig --name kubeflow > /tmp/kubeflow-config
export KUBECONFIG=/tmp/kubeflow-config

while ! kustomize build example | kubectl apply --server-side --force-conflicts -f -; do
  echo "Retrying to apply resources"; sleep 20;
done

kubectl port-forward svc/istio-ingressgateway -n istio-system 8080:80
# Visit http://localhost:8080

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Standing up a shared, multi-tenant ML platform on an existing Kubernetes cluster
• Authoring and scheduling repeatable ML pipelines instead of running steps by hand
• Running distributed training and hyperparameter tuning jobs at scale
• Giving data science teams in-cluster notebooks plus a path to model registry and serving

// How Kubeflow compares

Kubeflow alongside other open-source gpu & compute clouds tools AI/TLDR tracks, ranked by GitHub stars.