VMware Explore: VMware, Nvidia Unveil New Private Generative AI Platform

VMWARE EXPLORE 2023 — During VMware Explore 2023 on Tuesday, VMware and Nvidia announced a new full-stack platform for private generative AI for enterprises.

VMware Private AI Foundation with Nvidia will enable enterprises to customize models and run generative AI applications, including intelligent chatbots, assistants, search and summarization. The platform will be a fully integrated solution featuring generative AI software and accelerated computing from Nvidia, built on VMware Cloud Foundation and optimized for AI.

Private AI Foundation builds on the companies’ decade-long partnership. Their co-engineering work optimized VMware’s cloud infrastructure to run Nvidia AI Enterprise with performance comparable to bare metal. Mutual customers further benefit from the resource and infrastructure management, and flexibility enabled by VMware Cloud Foundation.

VMware intends to release Private AI Foundation in early 2024.

Private AI Foundation will be supported by Dell Technologies, Hewlett Packard Enterprise (HPE) and Lenovo. They will be among the first to offer systems that accelerate enterprise large language model (LLM) customization and inference workloads with Nvidia L40S GPUs, Nvidia BlueField-3 DPUs and Nvidia ConnectX-7 SmartNICs.

Paul Turner, VMware’s vice president of product management for vSphere, said Private AI Foundation is a skewed product from VMware.

VMware’s Paul Turner

“What that means is it’s available through all of our OEM channels, through our reseller channels, through our VAR channels and distributors,” he said. “So all of them will be able to get this product. And then on the MSP side, we’ve got our VMware Cloud Partner Program with more than 2,000 partners as part of that, and this will be available through the MSP program as well. MSP has a different pricing model and how we release it to them, so we’ll work through the timing of it, but we hope that we will have MSP availability of this as well. We expect to have MSP availability and we expect it to be day zero as well.”

Generative AI Part of the ‘Next Big Wave’

Justin Boitano, Nvidia‘s vice president of enterprise and edge computing, said he’s been with Nvidia since the two companies started working together, and Private AI Foundation represents the next step in that work.

Nvidia’s Justin Boitano

“The next big wave that we’re talking about obviously is generative AI, and this requires us to continue on that journey that we started together 10 years ago,” he said. “Generative AI is really the most transformational technology of our lifetimes. A generative AI model can encapsulate any knowledge, the knowledge of the internet, the knowledge of a business or the knowledge of licensed assets. The source knowledge that you start with can be zettabytes of data, but the resulting model that it gets encoded into is much smaller. It’s just gigabytes in size.

“So if we look at the new Meta Llama 2 70 billion-parameter model,” he continued, “that fits into about 140 gigabytes of memory. Now these models are amazing because they provide a natural language interface into a company’s business systems. Any question you ask them can be translated into queries across your business systems to get these real-time responses. So the power is is phenomenal. And so we see AI being infused into every business over the next decade to make people 10 times more productive to help them answer these complex questions about their business faster and more efficiently.”

Generative AI to Boost Global Economy

To achieve business benefits faster, enterprises are seeking to streamline development, testing and deployment of generative AI applications. McKinsey estimates generative AI could add up to $4.4 trillion annually to the global economy.

Private AI Foundation will enable enterprises to harness this capability by customizing LLMs, producing more secure and private models for their internal usage, offering generative AI as a service to their users, and more securely running inference workloads at scale.

Expect the platform to include integrated AI tools so enterprises can run proven models trained on their private data in a cost-efficient manner, according to VMware.

The platform will feature Nvidia NeMo, an end-to-end, cloud-native framework included in Nvidia AI Enterprise, the operating system of the Nvidia AI platform that allows enterprises to build, customize and deploy generative AI models anywhere. NeMo combines customization frameworks, guardrail toolkits, data curation tools and pretrained models to offer enterprises an easy, cost-effective and fast way to adopt generative AI.