VMware Introduces Private AI Cloud and AI Factory as Workloads Move On-Premises

Rising concerns about cost, data sovereignty, and security are encouraging enterprises to move AI workloads from public clouds to on-premises private clouds and AI factories as projects progress from experimentation to large-scale production. These environments are becoming a major datacenter model for many organizations.

Research firm Omdia forecasts that global datacenter investment will approach $1.6 trillion by 2030. It also estimates that technology enterprises will spend more than $600 billion on AI infrastructure in 2026.

The move from traditional datacenters to AI factories represents a substantial infrastructure change. Omdia defines an AI factory as infrastructure designed to produce intelligence, with token production at its center. These systems are characterized by high capital intensity, geopolitical importance, and complex engineering requirements.

Nvidia coined the term “AI factory.” In practice, the model generally refers to supercomputing infrastructure running generative AI inference to produce tokens that directly or indirectly support revenue. OEMs including Dell, Hewlett Packard Enterprise, and others offer AI factory systems, many of which rely heavily on Nvidia GPUs and related technologies.

VMware targets private AI production

At VMware Explore 2026 in Las Vegas, VMware and its owner, Broadcom, outlined a stronger focus on on-premises AI production environments. The approach builds on VMware Cloud Foundation 9 (VCF 9), introduced two years ago, with VMware Private AI Cloud and the VMware AI Factory model-as-a-service offering at its foundation.

VCF 9 includes features designed to address the cost and management challenges associated with AI workloads. These include NVMe memory tiering, which company executives say can reduce per-host costs by as much as 42 percent, and VMware AI Assistant, an interface for resolving complex issues involving CPUs, memory, and hypervisors.

The latest updates include enhancements to Tanzu and additional security capabilities for AI environments. According to Prashanth Shenoy, vice president of marketing for Broadcom’s VMware Cloud Foundation Division, these changes are intended to help VMware users move AI workloads into their own environments more easily.

“As organizations are moving from pilot deployments of AI applications and workloads to more production deployments of AI done at scale, they are facing major challenges around cost and security when it comes to data privacy, resiliency, availability of the infrastructure,” Shenoy told journalists during a pre-Explore media briefing. “That is causing a lot of organizations to repatriate workloads and run some of the production AI workloads back to an on-premises and private cloud environment.”

VMware’s June survey of 1,800 IT decision-makers found that 56 percent of respondents said their organizations were running or planning to run production inference in a private cloud. Sixty-two percent identified cost as a concern, while 51 percent said they were repatriating AI workloads because of security concerns. The survey also reported that public-cloud use for these workloads had fallen 15 percent year over year, to 41 percent.

Shenoy said organizations are choosing private clouds for production inference, fine-tuning, and rack-scale AI use cases because of cost and security considerations. He said VMware Private AI Cloud addresses cost efficiency, security in AI model-driven environments, and the development of agentic AI applications and workflows.

AI Factory infrastructure

VCF 9 supports CPUs, GPUs, and other accelerators from multiple vendors, as well as server hardware from OEMs and ODMs. VMware AI Factory adds VCF AI ReadyNodes from hardware manufacturers including Dell Technologies, Cisco Systems, Lenovo, and Supermicro.

The platform also uses AMD technologies, including Instinct MI350 Series GPUs and ROCm, AMD’s open software platform for GPU accelerators, AI, and high-performance computing.

VMware and Broadcom have tested the AI factory systems, according to Shenoy. The testing addresses the challenge of assembling the infrastructure required to run AI where an organization’s data resides.

“Customers want to run AI where their data lives, not the other way around,” Shenoy said. “They want to bring data to the model and not model to the data. But building this entire AI infrastructure from the metal to the module is a very manual process.”

That process can require organizations to deploy the base infrastructure, configure servers with the appropriate GPU size and form factor, add networking and storage, install Kubernetes for container workloads, deploy the AI software stack, and select models ranging from small language models (SLMs) and large language models (LLMs) to open-source and open-weight models.

VMware AI factories package these components into an integrated system orchestrated through MetalSoft’s platform, which provides unified operation of the underlying hardware and software.

VCF 9 also expands the number of open and commercial AI models supported by AI Factory to more than 150. Newly supported models include Nvidia’s Nemotron 3, Google DeepMind’s Gemma 4, Alibaba’s Qwen 3.7-Max, NEC’s cotomi, and GLM 5.2 from Z.ai.

Security for models and agents

VMware is adding services intended to support model sharing, governance, and agent security. AI models can be shared between tenants and lines of business through isolated nameplates, allowing reuse while maintaining separation and avoiding repeated deployments.

AI Gateway provides unified model governance across cloud and on-premises environments. Secure AI sandboxes provide virtualized container spaces for isolating agent code execution. Strong sandboxing has become an important concern in agentic AI after instances involving OpenAI and Anthropic agents escaped sandbox boundaries and compromised third-party IT environments.

Additional controls include Broadcom’s TrueSource for verifying open-source AI, enhancements to vDefend with zero-trust security for agentic AI workloads, and agentic threat defense through VMware’s Avi Load Balancer. These controls are intended to prevent agents from accessing unauthorized tools, carrying out zero-day attacks, or exfiltrating sensitive data without appropriate guardrails.

VMware has also enhanced Tanzu, its agent platform for Private AI Cloud. The updated security features include hardened agent sandboxes based on a “deny-by-default” containment model. Tanzu isolates credentials to help prevent prompt injection attacks and restricts network access that users have not authorized.

AI-ready data foundations process structured and unstructured data on-site. VMware says this can improve agent accuracy, reduce token costs, and limit hallucinations.

A ready-to-use development harness provides pre-approved agent skills, workflow buildpacks, human-in-the-loop controls, and integrated memory services. A marketplace allows developers and agents to connect to vetted AI models and tools. An AI gateway monitors and logs agent actions for compliance and auditing.

Shenoy said these controls are necessary because autonomous AI agents can generate and execute code dynamically. Without governance, organizations must determine which tools and data agents can access and which agents can communicate with one another.

The framework places agents in isolated secure containers and regulates communication between agents and the tools they can use.