You can provision nodes with 1, 2, 4, and 8 RTX PRO 6000 GPU instances, with each GPU providing 96 GB of GDDR7 memory. This launch provides the capability to use a single-node GPU, G7e.2xlarge instance to host powerful open source foundation models (FMs) like GPT-OSS-120B, Nemotron-3-Super-120B-A12B (NVFP4 variant), and Qwen3.5-35B-A3B, offering organizations a cost-effective and high-performing option. This makes it well suited for those looking to improve costs while maintaining high performance for inference workloads. The key highlights for G7e instances include:

Amazon Elastic Compute Cloud (Amazon EC2) G7e instances represent a significant leap in GPU-accelerated inference on the cloud. They deliver up to 2.3x inference performance compared to the previous-generation G6e instances. Each G7e GPU provides 1,597 GB/s bandwidth, doubling the per-GPU memory of G6e and quadrupling that of G5. Networking scales to 1,600 Gbps with EFA on the largest G7e size—a 4x jump over G6e and 16x over G5—unlocking low-latency multi-node inference and fine-tuning scenarios that were previously impractical on G-series instances. The following table summarizes the generational progression at the 8-GPU tier:

With 768 GB of aggregate GPU memory on a single instance, G7e can host models that previously required multi-node setups on G5 or G6e, reducing operational complexity and inter-node latency. Combined with support for FP4 precision using fifth-generation Tensor Cores and NVIDIA GPUDirect RDMA over EFAv4, G7e instances are positioned as the go-to choice for deploying LLMs, multimodal AI, and agentic inference workloads on AWS.

G7e’s combination of memory density, bandwidth, and networking capabilities makes it well suited for a broad range of modern generative AI workloads:

To quantify the generational improvement, we benchmarked Qwen3-32B (BF16) on both G6e and G7e instances using the same workload: ~1,000 input tokens and ~560 output tokens per request. This is representative of document summarization or correction tasks. Both configurations use the native vLLM container with prefix caching enabled.