Cache Coherency

Cache Coherency

Cache Coherency (The “Speed of Light” Problem)

This is the concept that explains why Data-Oriented Technology Stacks (DOTS) and ECS (Entity Component System) exist.

To understand this, you must understand that RAM is slow.
Relatively speaking, for the CPU to fetch data from RAM, it takes “forever.”

The Analogy: The Carpenter

Imagine you are a carpenter (The CPU).

• L1 Cache: Your toolbelt. (Instant access).
• L2/L3 Cache: The toolbox at your feet. (Fast access).
• RAM: The hardware store down the street. (Slow access).

You want to avoid going to the hardware store (RAM) at all costs.

How the Cache works (Cache Lines)

When the CPU asks RAM for a variable (e.g., bullet.speed), it doesn’t just grab that one number. It guesses that you will probably need the data next to it soon. So, it grabs a whole “chunk” of memory (usually 64 bytes)—called a Cache Line—and puts it in your toolbelt (Cache).

The Problem: “Cache Miss”

Imagine you have a List<Enemy> in your game.

• In the Heap: Since you created these enemies at different times, Enemy A is at Shelf #1, Enemy B is at Shelf #9000, and Enemy C is at Shelf #4.
• The Loop: You write a loop to update all enemies.
  1. CPU asks for Enemy A. It goes to the hardware store (RAM), gets Enemy A.
  2. CPU finishes Enemy A.
  3. CPU asks for Enemy B. It checks the toolbelt (Cache). Is Enemy B there? NO. Because Enemy B was at Shelf #9000, not next to Enemy A.
  4. CPU has to go back to the hardware store.

This is a Cache Miss. The CPU sits idle, doing nothing, waiting for data. If you have 10,000 enemies, you are making 10,000 trips to the store. This kills your FPS.

The Solution: “Cache Coherency” (Data Locality)

Instead of scattering enemies randomly in the Heap, you force them to sit right next to each other in memory (like an Array of Structs).

• The Optimized Loop:
  1. CPU asks for Enemy A.
  2. It goes to the store (RAM) and grabs a chunk. Because they are next to each other, this chunk contains Enemy A, B, C, and D!
  3. CPU updates Enemy A.
  4. CPU asks for Enemy B. It checks the toolbelt. YES! It’s already there.
  5. CPU updates B, C, and D instantly without touching RAM.

Real World Example

• Object-Oriented (Bad for CPU):
  class Player { int health; String name; Inventory inv; }
  The data is scattered. The pointer to Inventory points to somewhere else entirely.
• Data-Oriented (Good for CPU):
  struct HealthData { int[] healths; }
  All the health integers for every entity are packed tight in one long array. The CPU can chew through 10,000 health integers instantly because they are contiguous (touching) in memory.