Amazon's Graviton5 boosts speed and efficiency with new EC2 instances, surpassing Moore's law.

Concept 1: What is a CPU and Why Does It Matter?

The Basics

A CPU (Central Processing Unit) is the "brain" of a computer. It executes instructions — running applications, processing data, and managing tasks.

Why It Matters Here

Amazon builds its own custom CPUs called Graviton, instead of buying off-the-shelf processors. This gives them control over:

• Speed
• Energy efficiency
• Cost
• Security

Key Takeaway

Graviton is Amazon's homegrown CPU, now in its 5th generation, powering cloud servers on AWS.

---

Concept 2: Moore's Law — and Going Beyond It

What is Moore's Law?

Moore's Law is the historical observation that:

The number of transistors on a chip doubles approximately every two years, leading to faster and cheaper computers.

Why "Beyond Moore's Law"?

Simply cramming more transistors is no longer enough. Graviton5 goes further by improving:

• Architecture design (how the chip is organized)
• Core count (more parallel workers)
• Manufacturing process (smaller transistors)
• Memory speed (faster data delivery)

Key Takeaway

Graviton5 doesn't just follow Moore's Law — it combines multiple engineering advances simultaneously.

---

Concept 3: CPU Cores — More Workers, More Work Done

What is a Core?

Think of a CPU core as a worker on an assembly line. More cores = more tasks handled simultaneously.

Graviton5's Improvement

That's a 2x increase in parallel processing power.

But It's Not Just Quantity

Each individual core in Graviton5 is also 25% faster than a Graviton4 core.

Key Takeaway

Graviton5 doubled the workforce AND made each worker 25% more productive.

---

Concept 4: Nanometer Process — Smaller = Better

What Does "Nanometer" Mean?

The nanometer (nm) measurement refers to the size of transistors on a chip.

• Smaller transistors = more fit on the same chip
• More transistors = more computing power
• Smaller also means less energy wasted as heat

Graviton5's Improvement

What This Enables

• Greater circuit density (more transistors packed tightly)
• Faster on-chip communication (signals travel shorter distances)

Key Takeaway

Moving to 3nm lets Graviton5 pack more power into less space while using less energy.

---

Concept 5: Branch Prediction — Thinking Ahead

The Problem

Modern programs contain branches — decision points like:

IF condition is true → do Task A
ELSE → do Task B

The CPU must wait to know which path to take... unless it guesses ahead of time.

What is Branch Prediction?

Branch prediction is the CPU's ability to speculatively guess which path a program will take and start executing it before the decision is confirmed.

• If the guess is correct → time saved ✅
• If the guess is wrong → work is discarded and restarted ❌

Graviton5's Improvement

The Neoverse V3 core (co-designed by Arm and Amazon's Annapurna Labs) has substantially improved branch prediction, leading to:

Up to 30% better performance on real-world applications like databases

Why "Real-World" Matters

Micro-benchmarks (small test loops) don't reflect real applications. Real apps like databases have complex, unpredictable branching — so better branch prediction has a huge real-world impact.

Key Takeaway

Better branch prediction = less wasted work = faster real applications.

---

Concept 6: Memory Hierarchy — Bringing Data Closer

The Problem: DRAM is Far Away

DRAM (main memory/RAM) stores the data a CPU needs, but it's relatively slow to access:

DRAM access time ≈ 100 nanoseconds
CPU speed ≈ 3.3 GHz (1 cycle = ~0.3 nanoseconds)

Therefore: 1 DRAM access = ~330 CPU cycles of waiting!

That's like a chef waiting 5 minutes every time they need an ingredient from the pantry.

The Solution: Cache Memory

Caches are small, ultra-fast memory banks built inside or very close to the CPU. They store frequently used data so the CPU doesn't have to go all the way to DRAM.

The Cache Hierarchy (Fastest → Largest)

Graviton5's Improvement

The L3 cache is 5x larger than the previous Graviton generation.

More cache = more data kept close to the CPU = fewer slow DRAM trips

Key Takeaway

Graviton5's massive cache acts like a large kitchen counter — keeping ingredients (data) within arm's reach instead of making constant trips to the pantry (DRAM).

---

Concept 7: DDR5 Memory — Faster Data Highways

What is DDR5?

DDR5 is the latest generation of DRAM technology. The number after DDR5 (e.g., DDR5-8800) indicates the data transfer speed.

Graviton5's Advantage

• Supports DDR5-8800 — the fastest memory available
• AWS worked directly with DRAM manufacturers to achieve this
• Result: Fastest memory of any processor instances in the cloud

Key Takeaway

Even when the CPU must go to main memory, Graviton5 gets data back faster than any competitor.

---

Concept 8: PCIe — Connecting the CPU to Everything Else

What is PCIe?

PCIe (Peripheral Component Interconnect Express) is the high-speed connection between the CPU and other components like:

• Storage (NVMe SSDs)
• Network cards
• GPUs

Graviton5's Improvement

Graviton5 supports PCIe Gen 6, which doubles the bandwidth of PCIe Gen 5.

Key Takeaway

Faster PCIe means faster communication between the CPU and storage/networking hardware.

---

Concept 9: Chiplet Architecture — Divide and Conquer

What is a Chiplet?

Instead of building one massive single chip (monolithic die), a chiplet architecture breaks the CPU into multiple smaller chips (dies) that work together.

Benefits:

• Easier to manufacture (higher yield)
• Mix different technologies
• Better scalability

Graviton's Evolution

Graviton5's New Design

Each of the 4 chiplets contains:

• 48 cores
• DRAM controllers (built-in, no separate die)
• PCIe controllers (built-in, no separate die)

Graviton5 Chip
├── Chiplet 1: [48 cores + DRAM ctrl + PCIe ctrl]
├── Chiplet 2: [48 cores + DRAM ctrl + PCIe ctrl]
├── Chiplet 3: [48 cores + DRAM ctrl + PCIe ctrl]
└── Chiplet 4: [48 cores + DRAM ctrl + PCIe ctrl]

Die-to-Die Connectivity

The chiplets communicate at 420 GB/s — extremely fast inter-chiplet bandwidth that minimizes delays.

Key Takeaway

Graviton5's integrated chiplet design eliminates bottlenecks from separate controller dies and enables faster, more efficient chip-to-chip communication.

---

Concept 10: NUMA — Smart Memory Organization

What is NUMA?

NUMA (Non-Uniform Memory Access) is a memory architecture where:

• Different parts of the CPU have faster access to nearby memory
• Access to remote memory (on another chiplet) is slower

Why It Matters for Virtual Machines

Graviton5 can be configured into 2 or 4 NUMA regions, allowing:

• Virtual machines (VMs) to be assigned to a specific NUMA region
• VMs of 48 cores or smaller get reduced memory latency
• The L3 cache can be partitioned to match the VM's size

Key Takeaway

NUMA awareness means each virtual machine gets memory that's physically close to its cores — reducing wait times.

---

Concept 11: Virtualization and Virtual Machines (VMs)

What is a VM?

A Virtual Machine is a software-based simulation of a physical computer. One physical server can run many VMs simultaneously, each acting like an independent computer.

Why This Matters

Cloud customers rent VMs (called EC2 instances on AWS). The physical Graviton5 chip must:

• Run multiple VMs at once
• Keep them isolated from each other (security)
• Give each VM its fair share of resources

Key Takeaway

Graviton5's architecture is specifically optimized to run many VMs efficiently and securely.

---

Concept 12: The Nitro System and Security Isolation

What is the Nitro System?

Nitro is Amazon's custom hardware and software system that manages virtualization on AWS. It:

• Offloads virtualization tasks from the CPU
• Enforces strict isolation between VMs

The New: Nitro Isolation Engine

Graviton5 introduces the Nitro Isolation Engine, a dedicated hardware component that:

Key Takeaway

The Nitro Isolation Engine is a purpose-built security enforcer — not general-purpose software doing security as a side job.

---

Concept 13: Formal Verification — Math-Proven Security

What is Formal Verification?

Formal verification is a technique that uses mathematical proofs to demonstrate that a system behaves exactly as intended — not just in tested scenarios, but in all possible scenarios.

Traditional Testing vs. Formal Verification

Why It's Significant

The Nitro Isolation Engine makes AWS the first formally verified cloud hypervisor — meaning its security isolation is mathematically proven, not just tested.

Analogy

• Traditional testing = checking if a bridge holds 100 specific trucks
• Formal verification = mathematically proving the bridge holds any load within its design parameters

Key Takeaway

Graviton5's security isn't just tested — it's mathematically proven, setting a new industry standard.

---

Summary: All Concepts Together

GRAVITON5 IMPROVEMENTS
│
├── MORE CORES (96 → 192) ──────────── More parallel work
├── BETTER CORES (+25%) ─────────────── Each core is faster
├── 3nm PROCESS ─────────────────────── Denser, more efficient chip
├── BRANCH PREDICTION ───────────────── Smarter execution of real apps
├── LARGER L3 CACHE (5x) ────────────── Data closer to CPU
├── DDR5-8800 MEMORY ────────────────── Fastest cloud memory
├── PCIe Gen 6 ──────────────────────── Faster I/O connections
├── NEW CHIPLET DESIGN ──────────────── Integrated, lower latency
├── NUMA OPTIMIZATION ───────────────── Better VM memory locality
└── NITRO ISOLATION ENGINE ──────────── Formally verified security

Net Result

---

Bottom Line: Graviton5 isn't just a faster chip — it's a holistically redesigned system that improves speed, efficiency, memory, connectivity, and security simultaneously, going well beyond what Moore's Law alone would predict.