NVIDIA Opens Run:ai Source as Part of the NVIDIA DSX AI Factory Platform
NVIDIA Opens Run:ai Source as Part of the NVIDIA DSX AI Factory Platform NVIDIA has released the full platform as open source software. The release provides customers, developer...
By Hardware Team
NVIDIA Opens Run:ai Source as Part of the NVIDIA DSX AI Factory Platform
NVIDIA has released the full NVIDIA Run:ai platform as open source software. The release provides customers, developers, and ecosystem partners with access to production-grade AI workload and GPU orchestration capabilities.
The move expands the NVIDIA DSX AI Factory Platform by making NVIDIA Run:ai an open foundation for building and delivering AI infrastructure solutions.
For several years, NVIDIA Run:ai has helped organizations run AI workloads on Kubernetes at scale. It dynamically allocates GPU resources and manages workloads throughout their lifecycle, from data processing and training to inference. The open source release is intended to help NVIDIA DSX ecosystem partners and AI cloud providers increase workload throughput, improve GPU utilization, and operate shared AI infrastructure more efficiently.
The release includes the complete NVIDIA Run:ai platform, along with four components that can be adopted independently:
Together with additional platform code, these components form the integrated NVIDIA Run:ai platform. Platform teams and ecosystem partners can deploy the complete platform, build products and services on top of it, or integrate individual components.
Adopting open source NVIDIA Run:ai
NVIDIA Run:ai can be adopted directly as open source software or through offerings from NVIDIA DSX ecosystem partners.
Deploying the platform directly
The complete source code is available in the NVIDIA Run:ai repository, part of the NVIDIA DSX AI Factory GitHub organization. Teams can inspect the code, contribute to its development, build on the platform, or use its components independently.
The integrated platform is also available on NVIDIA NGC as a Community Product. It includes a Helm chart for installing the full stack.
The NVIDIA Run:ai source code is governed by its applicable open source licenses. Downloading the Community Product from NGC requires acceptance of the applicable NVIDIA Software License Agreement and Product-Specific Terms for NVIDIA AI Products. A paid NVIDIA Run:ai subscription or NVIDIA support entitlement is not required, but enterprise support is not included with the Community Product.
Adoption through NVIDIA DSX ecosystem partners
The open source release gives NVIDIA DSX ecosystem partners direct access to AI workload and GPU orchestration components for their own NVIDIA DSX AI factory solutions. Partners can use the complete platform or integrate individual components.
- Crusoe integrates NVIDIA Run:ai with Crusoe Managed Kubernetes, combining GPU workload orchestration with managed Kubernetes and GPU infrastructure.
- Mirantis extends k0rdent AI with NVIDIA Run:ai for a lifecycle-managed, Metal-to-Model AI factory stack that automates infrastructure provisioning and GPU utilization.
- Nebius offers NVIDIA Run:ai as a one-click application in Nebius AI Cloud for deployment on Nebius Managed Service for Kubernetes.
- OpenNebula Systems integrates NVIDIA Run:ai with the OpenNebula AI Cloud Platform through OneKS, combining GPU workload orchestration with infrastructure provisioning and lifecycle management for open and sovereign AI clouds.
- Rafay is expanding its integration to provide NVIDIA Run:ai as a fully managed service. The service automates GPU infrastructure, clusters, the NVIDIA Run:ai control plane, and tenants for AI cloud providers and enterprises.
- Saturn Cloud integrates NVIDIA Run:ai with its Token Factory platform for white-label AI inference services, combining GPU orchestration with model serving, per-token metering, and automated billing.
- Spectro Cloud integrates NVIDIA Run:ai into PaletteAI, allowing customers to deploy it through PaletteAI Studio and manage it with the platform's lifecycle automation.
- vCluster provides pre-validated deployment templates that create a complete environment for each tenant while NVIDIA Run:ai schedules workloads across a shared GPU fleet.
For NVIDIA Enterprise AI Factory deployments, NVIDIA Run:ai remains available through NVIDIA-supported commercial offerings, including as a standalone product, through NVIDIA Mission Control and NVIDIA AI Enterprise, and through selected cloud marketplaces. Availability, licensing, and support terms vary by offering.
Teams that already operate an AI platform can also adopt individual components to address specific capability gaps.
Modular components of NVIDIA Run:ai
The open source release includes KAI Scheduler, GPU Fractioning Operator, KAI Resource Management, and Workload Map. With additional platform code, these components make up the integrated NVIDIA Run:ai platform.
KAI Scheduler
KAI Scheduler is the Kubernetes-native scheduling engine behind NVIDIA Run:ai. NVIDIA released it under the Apache 2.0 license and contributed it to CNCF as a Sandbox project.
KAI remains the upstream engine for NVIDIA Run:ai. Scheduler development takes place in the community repository, allowing improvements to flow through a shared codebase. NVIDIA continues to support the project's maintainers and collaborate with the broader ecosystem on large-scale AI workload scheduling.
KAI Scheduler coordinates several GPU workload capabilities:
- Gang scheduling with prescheduling in-memory simulation, so multipod jobs are placed as all-or-nothing units and are not evicted unnecessarily
- Fair-share scheduling, including time-based fair-share, and quota allocation across teams
- Topology-aware placement and bin-packing at the scheduling-decision level
- GPU sharing across one or multiple GPUs using MIG, MPS, time sharing, HaMi, and CUDA 13.4 MPS fractioning
- Dynamic Resource Allocation (DRA) support
KAI Scheduler is intended for diverse AI workloads that require fair cluster access and topology-aware placement. It can be deployed in an existing Kubernetes cluster alongside the current software stack without installing the complete NVIDIA Run:ai platform.
More information is available in the KAI Scheduler repository.
GPU Fractioning Operator
The NVIDIA Run:ai platform has supported fractional GPU allocation, allowing multiple workloads to share a GPU with hard memory isolation. Fractions can be requested on demand and configured without advance per-node setup by an administrator. This is useful for workloads such as interactive notebooks and inference for smaller models that do not require a full GPU.
The GPU Fractioning Operator brings CUDA-native GPU fractioning to Kubernetes. NVIDIA's proprietary implementation has been retired in favor of the native capability available in CUDA 13.4.
The CUDA driver enforces a hard GPU memory limit for each workload, and each workload sees only the memory assigned to its fraction. GPU Fractioning Operator provides the Kubernetes integration layer for consuming this capability, along with a metrics exporter that reports per-pod fraction usage. It is scheduler-agnostic.
The operator can be used for notebooks, development sessions, and smaller models that share underutilized GPU capacity. It integrates with KAI Scheduler to translate available GPU capacity into placement decisions and remains compatible with other schedulers.
Installation details are available in the GPU Fractioning Operator documentation.
KAI Resource Management
While the scheduler determines where work runs, KAI Resource Management manages how resources are organized and allocated. It provides:
- Projects and departments: An organizational layer that maps resources to teams and initiatives
- Resource management with an organizational hierarchy: Allocation, guarantees, and sharing of GPU capacity using fair-share and quota semantics
- Node pools: Groups of heterogeneous hardware that can be targeted by workloads
These capabilities allow platform teams to manage GPU capacity by team, department, initiative, and resource guarantee rather than relying only on raw Kubernetes scheduling.
KAI Resource Management is intended for heterogeneous clusters shared by multiple teams with defined capacity allocations. For example, a platform team can guarantee 50% of a cluster to research while preventing one team from monopolizing the remaining capacity. The component can replace manually maintained spreadsheets, quotas, and team-specific namespaces, and it can be combined with KAI Scheduler.
Additional information is available in the KAI Resource Management documentation.
Workload Map
Workload Map defines the workload contract used by NVIDIA Run:ai. KAI Scheduler uses it for scheduling and gang semantics, while other platform components use it for workload discovery, monitoring, and lifecycle management.
AI workloads on Kubernetes are often represented by custom resources such as a PyTorchJob, RayCluster, JobSet, KServe InferenceService, Dynamo graph deployment, or an organization-specific resource. Each workload type can have different structures, roles, scaling behavior, gang semantics, and status reporting.
Workload Map provides a consistent way to handle these workloads throughout their lifecycle, including:
- Describing pods, components, and roles
- Submitting and configuring workloads
- Scheduling with gang and topology awareness
- Observing status and health
- Applying policy at admission or runtime
The layer sits between Kubernetes primitives and the schedulers, dashboards, SRE and troubleshooting tools, cost tools, backup tools, and ML platforms that serve users. Instead of requiring every tool to create a separate integration for each custom resource definition, Workload Map defines each workload type once so platforms and tools can schedule, observe, manage, and govern it consistently.
Workload Map is distributed as a Go library that operates offline on unstructured objects, so it can be adopted without adding components to a cluster. It describes workload structure but does not schedule, queue, autoscale, or propagate workloads across clusters.
The Workload Map repository contains additional information and a list of adopter projects.
Related open source projects
Several additional open source projects address related AI infrastructure challenges. They can be used independently or incorporated into solutions that include NVIDIA Run:ai or selected components.
Model Streamer
Cold-starting an inference workload on Kubernetes can take several minutes while model weights are loaded from storage into GPU memory. During this period, GPUs may be allocated but idle. Model Streamer reads weights concurrently and streams them directly into GPU memory to reduce loading time for large models.
Model Streamer has been integrated into parts of the inference stack, including inference engines, cloud services, and storage platforms.
Inference engines
Model Streamer is integrated upstream in vLLM and SGLang. It can be enabled with the --load-format runai_streamer flag and is also integrated into ModelExpress. Supported sources include local filesystems, S3-compatible stores, network file shares, and several cloud backends.
Cloud providers
- AWS: Model Streamer ships in the vLLM Deep Learning Container and is identified in EKS documentation for AI and ML inference, including tuned concurrency and chunk sizing for S3.
- Google Cloud: It includes native Cloud Storage support developed with NVIDIA, with GKE documentation for streaming from a bucket into accelerator memory.
- Microsoft Azure: Azure Blob Storage integration was developed by Microsoft and NVIDIA Run:ai. Published benchmarks on eight NVIDIA H100 VMs measured loading speeds 3x to 6x faster than the default loader, with sustained throughput of 80 Gbps compared with approximately 40 Gbps for the default path.
Storage platforms
Storage infrastructure providers have also published Model Streamer measurements:
- WEKA loaded models into GPU memory in 3.5 seconds, approximately 40% faster than local NVMe.
- VAST Data measured 1.7 seconds, a 60% improvement over local NVMe.
- DDN streamed the 65.5 GB Qwen3-32B model across eight GPUs in under four seconds and reported savings of up to 10% of daily GPU runtime compared with local storage.
Model Streamer is intended for deployments where starting new replicas and loading model weights is the primary bottleneck.
Grove
Workload patterns such as agentic pipelines, disaggregated serving, and mixture-of-experts training use multiple connected components that must scale together. Grove provides a Kubernetes API for orchestrating these workloads, from a single pod to deployments spanning tens of thousands of GPUs.
Grove provides topology-aware autoscaling and hierarchical gang scheduling. Components such as prefill and decode can start as all-or-nothing units. On NVIDIA GB200 NVL72, Grove also manages ComputeDomain resources automatically, enabling multinode NVLink performance through a single annotation instead of manually written YAML.
Grove is a modular part of NVIDIA Dynamo that can also be used independently. It supports training and inference pipelines, with recent work adding llm-d and Volcano as scheduler backends and Kueue support in progress.
Grove is intended for graph-based workloads with distinct roles and coupled scaling requirements.
Open source repositories
NVIDIA Run:ai is available as open source software in the NVIDIA DSX AI Factory GitHub organization. The main repositories include:
- NVIDIA Run:ai, the complete platform
- KAI Scheduler, the scheduling engine
- KAI Resource Management, for projects, departments, quotas, and node pools
- GPU Fractioning Operator, for CUDA GPU fractions on Kubernetes and per-pod metrics
- Workload Map, the workload-management contract
- Model Streamer, for model weight loading
- Grove, for multicomponent inference orchestration