Memory Hierarchy

High Level

1. Registers
   • Register File
   • Managed by: Compiler
   • Scale: KiB
2. Cache
   • Multiple levels (each larger and slower than the last)
   • SRAM
   • Managed by: Hardware
   • Scale: KiB, MiB
3. Main Memory
   • DRAM
   • Managed by: Operating System
   • Scale: GiB
4. I/O Devices
   • Disk storage, flash storage
     • Flash is a middle ground between hard disk and RAM
     • Reads nearly as fast as RAM, but writes a lot slower (still better than HD though)
     • Flash also has a limited number of writes
   • Managed by: Operating System
   • Scale: TiB

Detailed Levels

1. Pipeline registers
2. Register file
3. 1st Level Cache (on chip)
4. 2nd Level Cache (on same MCM as CPU)
5. (more cache levels)
6. Physical memory (mounted on same board as CPU)
7. Virtual Memory (on hard disk, same enclosure as CPU)
8. Disk files (on hard disk, same enclosure as CPU)
9. Network-accessible disk files (same building as CPU)
10. Backup/archive system (same building as CPU)
11. Data warehouse (same planet as CPU)

• 1, 3-7 are usually made invisible to (even assembly) programmers
• 2 is invisible to high level languages

Physical

Memory

1. Static Random Access Memory (SRAM)
   • Used in the cache (the buffer within the processor)
   • Faster and less dense than DRAM → More expensive
2. Dynamic Random Access Memory (DRAM)
   • Main/primary memory
3. Nonvolatile Memory
   • Secondary memory
   • Stores data between runs
   • Either flash (faster, more expensive, and longer life) or magnetic disk (not that)

Stacking

• Vertical Stacking (3D)
  • DRAM is stacked on top of the chip
  • Has cooling issues
• Interposer Stacking (2.5D)
  • DRAM is stacked next to the chip
  • Both are on top of an interposer
  • Interposer communicates between the two

Locality

• Temporal locality
  • Items accessed recently are likely to be accessed again
  • ex: instructions in a loop
• Spatial locality
  • Items near those accessed recently are likely to be accessed soon
  • ex: array data
• How Memory hierarchy uses this
  • Store everything on disk
  • Copy recently accessed (and nearby) items from disk to main memory
  • Copy more recently accessed (and nearby) items from main memory to cache
  • Cache hit means that accessed data is present in a faster level
    • Hit ration = hits / accesses
    • Miss ration = misses / accesses = 1 - hit ratio

Dependability Virtual Machines