Nvidia GTC: Blackwell Platform for Gen AI Steals Show
Nvidia kicked off its annual event Monday with a big focus on generative AI.
Nvidia CEO Jensen Huang on stage at Nvidia GTC, March 18, 2024.NVIDIA
Here are some of the most interesting observations and remarks during Nvidia CEO Jensen Huang's two-hour keynote:
“$100 trillion of the world's industries is represented in this room today. This is absolutely amazing.”
“It’s not about driving down the cost of computing. It's about driving up the scale of computing.”
“We need bigger GPUs. And so, ladies and gentlemen, I would like to introduce you to a very, very big GPU: Blackwell.”
“The likelihood of a super computer running for weeks on end is approximately zero. And the reason for that is because there's so many components working at the same time. The statistic, the probability, of them working continuously is very low. And so we need to make sure that … we checkpoint and restart as often as we can. But if we have the ability to detect a weak chip or a weak node early, we can retire it and maybe swap in another processor. That ability to keep the utilization of the supercomputer high, especially when you just spent $2 billion building it, is super important.”
“We now have the ability to encrypt data — of course at rest, but also in transit. And while it's being computed.”
“All of these capabilities are intended to keep Blackwell fed and as busy as possible.”
“The future is generative, which is the reason why we call it generative AI.”
“The inference capability of Blackwell is off the charts. In fact, it is some 30 times [that of] Hopper.”
“Blackwell is going to be the most successful product launch in our history.”
The world’s largest public cloud computing provider, AWS, soon will offer the new Nvidia Blackwell GPU platform alongside its own generative AI technologies.
For its part, AWS will contribute Nitro System, Key Management Service for security, Elastic Fabric Adapter petabit scale networking, and EC2 UltraCluster hyper-scale clustering.
The companies expect all of those components to deliver the infrastructure and tools for customers pursuing new AI capabilities.
“Nvidia’s next-generation Grace Blackwell processor marks a significant step forward in generative AI and GPU computing,” said Adam Selipsky, AWS CEO.
When combining Nvidia’s tools with AWS’ technologies, he added, “we make it possible for customers to build and run multitrillion parameter large language models faster, at massive scale and more securely than anywhere else.”
Huang agreed.
“Our collaboration with AWS is accelerating new generative AI capabilities and providing customers with unprecedented computing power to push the boundaries of what's possible,” he said.
Not to be outdone, Microsoft, too, is beefing up its work with Nvidia.
Azure will soon feature the GB200 Grace Blackwell processor, just like AWS. Microsoft further will add the Nvidia Quantum-X800 InfiniBand networking to Azure. This will create trillion-parameter foundation models for natural language processing, computer vision, speech recognition and more, the companies said.
“Together with Nvidia, we are making the promise of AI real, helping to drive new benefits and productivity gains for people and organizations everywhere,” said Satya Nadella, chairman and CEO of Microsoft. “The announcements we are making today will ensure customers have the most comprehensive platforms and tools across every layer of the Copilot stack, from silicon to software, to build their own breakthrough AI capability.”
Finally, Microsoft has made its Azure NC H100 v5 VM virtual machine based on the Nvidia H100 NVL platform generally available. It supports mid-range training and inferencing.
“Through our collaboration with Microsoft, we’re building a future that unlocks the promise of AI for customers, helping them deliver innovative solutions to the world,” Huang said.
Of course, Google Cloud, too, is talking up its collaboration with Nvidia.
Just like its peers and rivals, the world’s third-largest public cloud computing provider is adopting the Grace Blackwell AI computing platform. It’s also going to debut the Nvidia DGC Cloud service. Plus, Google Cloud has made Nvidia H100-powered DGXTM Cloud platform generally available and plans to roll out plenty of AI tools and integrations for developers.
“The strength of our long-lasting partnership with Nvidia begins at the hardware level and extends across our portfolio,” said Thomas Kurian, CEO of Google Cloud. “Together with Nvidia, our team is committed to providing a highly accessible, open and comprehensive AI platform for ML developers.”
Huang agreed.
“With expanded infrastructure offerings and new integrations with Nvidia’s full-stack AI, Google Cloud continues to provide customers with an open, flexible platform to easily scale generative AI applications.”
On a slightly different AI-collaboration note, Oracle is expanding its partnership with Nvidia with an eye toward sovereign AI.
In other words, the companies are helping governments and enterprises to run cloud services locally, and within a country’s or organization’s secure premises. Those capabilities come with operational controls for diversifying and boosting economic growth, Oracle and Nvidia said.
“As AI reshapes business, industry and policy around the world, countries and organizations need to strengthen their digital sovereignty in order to protect their most valuable data,” said Safra Catz, Oracle CEO. “Our continued collaboration with Nvidia and our unique ability to deploy cloud regions quickly and locally will ensure societies can take advantage of AI without compromising their security.”
Huang said that Oracle’s cloud applications and infrastructure, combined with Nvidia products, “create the flexibility and security nations and regions require to control their own destiny.”
Oracle debuted its sovereign cloud platform for managed service providers, specifically, in 2022. Oracle and Nvidia said on March 18 that their combined offerings work in the public or private cloud.
Oh, and yes, Oracle, like its competitors, is adding the Blackwell platform to its environment.
Nvidia on March 18 launched dozens of enterprise-grade generative AI microservices, dubbed Nvidia AI Enterprise 5.0.
The company says these will help businesses create and deploy custom applications on their own platforms while retaining full ownership and control of their intellectual property.
AI Enterprise 5.0 includes a range of microservices: NIM for deploying AI models in production and the Nvidia CUDA-X collection of microservices, which includes Nvidia cuOpt. It also comes with the new AI Workbench, a toolkit for developers. Plus, it now supports Red Hat OpenStack.
“Established enterprise platforms are sitting on a goldmine of data that can be transformed into generative AI copilots,” Huang said. “Created with our partner ecosystem, these containerized AI microservices are the building blocks for enterprises in every industry to become AI companies.”
Customers will be able to access NIM microservices from Amazon SageMaker, Google Kubernetes Engine and Microsoft Azure AI, as well as VMware Private AI Foundation. It integrates, too, with AI frameworks including Deepset, LangChain and LlamaIndex. The microservices also are supported on hundreds of systems, including servers and workstations from Cisco, Dell Technologies, Hewlett Packard Enterprise, HP, Lenovo and Supermicro.
In yet another Nvidia partnership announcement, SAP has joined the lineup.
The enterprise resource management vendor also aims to enable customers to take advantage of generative AI across its portfolio of cloud-based platforms. As such, SAP and Nvidia will build SAP Business AI. This will infuse generative AI capabilities into the Joule copilot and other products.
SAP will use Nvidia’s generative AI foundry service to hone LLMs for domain-specific scenarios and deploy applications with Nvidia’s new NIM microservices. These integrated capabilities should be available by the end of 2024.
“Enterprise customers want … technology that delivers real business value,” said Christian Klein, SAP CEO. “[P]artnerships … are at the core of our strategy to invest in technology that maximizes the potential and opportunity of AI for business.”
Huang said SAP is “sitting on a gold mine of enterprise data that can be transformed into custom generative AI agents to help customers automate their businesses.”
Here’s what analyst firm TechAisle’s Anurag Agrawal told us about Nvidia’s news:
“From a channel perspective, AI is expected to impact traditional channel business models profoundly. It will change how businesses operate at all levels, fundamentally altering IT economics and service delivery capabilities. Discussions with IT buyers will change as AI defines new ways businesses can use technology. AI makes IT relevant to non-technical managers, expanding overall opportunity and the buying group that channel partners need to address.
"For channel businesses, gradual change will not adequately respond to AI-driven market trends. Some vendor partners may disintermediate the channel and develop AI-powered sales tools and self-service platforms to establish their leadership position; some customers may see partners that are more in tune with the business options that arise from AI’s impact throughout infrastructure and operations.
"Channel leaders must embrace a ‘sense and respond’ approach that allows them to identify developments that will have a tangible impact on their businesses while avoiding overcommitment to avenues that may seem technically impressive but are misaligned with financial or operational goals. While the promise of robust AI-driven growth is evident, success will be far from automatic.
"Channel organizations will need to ‘up their game’ to address new business decision-makers in terms that reflect an understanding of the operational changes they envision and will need to tie IT solutions to business metrics and outcomes. From the buyer’s perspective, deploying and benefitting from AI is an extraordinarily high-stakes and complex venture. Accordingly, buyers indicate they will seek a coalition of channel suppliers to meet their AI technology and business challenges.”
Here’s what analyst firm TechAisle’s Anurag Agrawal told us about Nvidia’s news:
“From a channel perspective, AI is expected to impact traditional channel business models profoundly. It will change how businesses operate at all levels, fundamentally altering IT economics and service delivery capabilities. Discussions with IT buyers will change as AI defines new ways businesses can use technology. AI makes IT relevant to non-technical managers, expanding overall opportunity and the buying group that channel partners need to address.
"For channel businesses, gradual change will not adequately respond to AI-driven market trends. Some vendor partners may disintermediate the channel and develop AI-powered sales tools and self-service platforms to establish their leadership position; some customers may see partners that are more in tune with the business options that arise from AI’s impact throughout infrastructure and operations.
"Channel leaders must embrace a ‘sense and respond’ approach that allows them to identify developments that will have a tangible impact on their businesses while avoiding overcommitment to avenues that may seem technically impressive but are misaligned with financial or operational goals. While the promise of robust AI-driven growth is evident, success will be far from automatic.
"Channel organizations will need to ‘up their game’ to address new business decision-makers in terms that reflect an understanding of the operational changes they envision and will need to tie IT solutions to business metrics and outcomes. From the buyer’s perspective, deploying and benefitting from AI is an extraordinarily high-stakes and complex venture. Accordingly, buyers indicate they will seek a coalition of channel suppliers to meet their AI technology and business challenges.”
NVIDIA GTC — Nvidia continues to lead the AI races.
At its annual conference Monday in San Jose, California, the maker of semiconductors and GPUs unleashed an avalanche of news. Perhaps the biggest nugget to come out of Nvidia GTC that stands to interest channel partners is the release of the Blackwell platform. Nvidia bills Blackwell as technology that lets organizations do real-time generative AI on trillion-parameter large language models at up to 25-times less cost and energy consumption than its predecessor.
The platform comprises Nvidia’s GB200 Grace Blackwell Superchip and the B100 Tensor Core GPUs. It’s named after David Harold Blackwell — a mathematician who specialized in game theory and statistics, and who was the first Black scholar inducted into the National Academy of Sciences. The new Blackwell architecture succeeds Hopper, launched two years ago.
In the slideshow above, look for insights into Nvidia’s expanded partnerships with Amazon Web Services, Microsoft Azure, Google Cloud and Oracle, all of which will put Blackwell to use. In the meantime, here are some technical details around Blackwell:
Comes with 208 billion transistors.
Supports double the compute and model sizes with new 4-bit floating point AI inference capabilities.
Delivers 1.8TB/s bidirectional throughput per GPU for high-speed communication among up to 576 GPUs.
Features confidential computing to protect AI models, customer data.
“For three decades we’ve pursued accelerated computing, with the goal of enabling transformative breakthroughs like deep learning and AI,” Jensen Huang, founder and CEO of Nvidia, said. “Generative AI is the defining technology of our time. Blackwell is the engine to power this new industrial revolution. Working with the most dynamic companies in the world, we will realize the promise of AI for every industry.”
Nvidia GTC also featured a number of other announcements, the most applicable to channel partners we're covering for you at a high level. Get all the nitty-gritty details at Nvidia’s online newsroom.
Meanwhile, above we feature some of the pithiest takeaways from Huang’s Nvidia GTC 2024 keynote speech, as well as more insight into how AWS, Azure, Google Cloud and other vendors are working more with Nvidia.
About the Author(s)
You May Also Like