As artificial intelligence continues to reshape industries and redefine the boundaries of innovation, Microsoft is proud to announce a leap forward in secure, high-performance computing in our Secret and Top Secret clouds: the integration of NVIDIA H200 Tensor Core GPUs into Azure’s AI infrastructure. This advancement helps unlock unprecedented generative AI capabilities, large language models (LLMs), and high-performance computing (HPC) workloads, while meeting the stringent security and compliance demands of government and regulated industries.

Built for the Future of AI 

The NVIDIA H200 GPU is engineered for today’s generation of AI workloads. With 141 GB of HBM3e memory and 4.8 TB/s of bandwidth, it delivers a 76% increase in memory capacity and a 43% boost in bandwidth over its predecessor, the H100. This enables faster training, larger model support, and more efficient inference, fundamental for applications ranging from real-time language translation to scientific simulations.

The ND H200 v5 series starts with a single VM and eight NVIDIA H200 Tensor Core GPUs, interconnected with 900 GB/s NVLink. ND H200 v5-based deployments can scale up to thousands of GPUs with 3.2Tb/s of interconnect bandwidth per VM. Each GPU within the VM is provided with its own dedicated, topology-agnostic 400 Gb/s NVIDIA Quantum-2 CX7 InfiniBand connection. These connections are automatically configured between VMs occupying the same virtual machine scale set, and support GPUDirect RDMA.

Supported Features:

• Premium Storage: Offers the highest speed Azure Storage can provide to a VM.
• Premium Storage caching: Allows the VM to exceed the underlying disk IOPS limitations through multitier caching technology.
• Generation 2 VMs: Use the new UEFI-based boot architecture rather than the BIOS-based architecture used by Generation 1 VMs, leading to improved boot and installation times.
• Accelerated Networking: Helps enhance networking performance of the VM by enabling Single Root I/O Virtualization (SR-IOV).
• Ephemeral OS Disk: Disks that are created on the local virtual machine storage and not saved to the remote Azure Storage.
• Trusted Launch: Helps protect against advanced and persistent attack techniques through secure boot and other key integrity and trust guarantees.

Real-World Impact 

AI is beginning to deliver tangible benefits across government missions, streamlining operations and enabling new capabilities. Agencies adopting generative AI tools are reporting faster turnaround times for tasks such as grant review and technical content preparation, leading to improved accessibility and efficiency. Document intelligence solutions are helping to reduce manual effort in legal and financial analysis, while predictive analytics and intelligent assistants are enhancing decision-making and transforming how users access and interact with information. Early implementations have demonstrated measurable improvements in workflow speed, data accessibility, and service responsiveness, supporting government teams as they work to better serve citizens.

What’s Next 

Microsoft is actively expanding its GPU capacity across Azure Secret and Top Secret clouds, enabling customers to power their AI workloads. With support for Azure Machine Learning, Microsoft AI Foundry (including powering serverless models like Azure OpenAI and community models from the model catalog), and Azure Kubernetes Service, the ND H200 v5 series integrates seamlessly into existing capabilities.

As we continue to push the boundaries of AI, Microsoft remains committed to delivering secure, scalable, and high-performance infrastructure that empowers every organization to innovate with confidence.

To get started with the newly authorized products now available in Azure for U.S. Government Secret and Top Secret, please contact your Microsoft Account Team. Visit ND-H200-v5 size series – Azure Virtual Machines on Microsoft Learn to discover how to do more for your mission with Microsoft Azure.