← Back to list
Computing & Embedded
#성능분석#연산강도#메모리대역폭#HPC#컴퓨트바운드
Last updated · 2026-11-16

The Roofline Model and Performance Analysis

1. Overview

A. Definition

The Roofline Model is a visual performance model that plots the theoretically attainable maximum performance of a compute kernel on given hardware as two upper bounds (rooflines) over a log–log graph whose horizontal axis is Arithmetic Intensity (floating-point operations per byte). Proposed in 2009 by Samuel Williams, Andrew Waterman, and David Patterson of UC Berkeley and LBNL in a CACM paper, it lets one diagnose, in a single picture, whether performance is limited by compute capability (peak FLOP/s) or by memory bandwidth (peak bandwidth).

The first question an engineer must answer when starting performance optimization is: "is this code slow because there is too much computation, or because it cannot move data fast enough?" The two causes call for opposite prescriptions. If compute is the bottleneck, one must raise arithmetic throughput with SIMD, FMA, and more cores; but if data movement is the bottleneck, adding compute units is useless, and one must cut memory traffic through cache reuse, data locality, and compression. The Roofline model gives this question a quantitative coordinate. Placing the kernel's arithmetic intensity on the x-axis and the two hardware ceilings as rooflines immediately reveals the nature of the bottleneck and the room for optimization, by showing which part of the roof the kernel sits below.

The Roofline is powerful because it is not a mere benchmark number but a device that overlays the physical limits of hardware and the characteristics of software on the same plane. Peak FLOP/s and peak bandwidth are constants fixed once the chip is chosen, while arithmetic intensity follows once the algorithm is chosen. By making the two worlds meet on one graph, one can reason with evidence about "what is the best this algorithm can do on this hardware" and "is there room to go higher through optimization, or has it already hit the roof?" From a professional engineer's viewpoint, the Roofline is a first-order diagnostic tool that backs performance investment decisions with engineering numbers — compute vs. memory expansion, algorithm redesign vs. hardware replacement.

B. Background and Necessity

The Roofline emerged in the late 2000s, when single-core clock gains hit the power wall and the shift to multicore began. Core counts rose, but memory bandwidth did not grow as fast, so the gap between compute capability and data-supply capability (the so-called memory wall) widened. As a result, many real kernels fell into a state of idling compute units while waiting on memory, and a single number like "what percentage of peak FLOP/s are we using" could not explain why they were slow or what to fix.

The need for the Roofline has only grown over time. Accelerators like GPUs, NPUs, and TPUs dramatically raised arithmetic throughput, but memory bandwidth grew relatively less, so workloads with low arithmetic intensity increasingly suffer bandwidth-bound behavior that fails to exploit the accelerator's potential. For example, the token-generation (decode) phase of a large language model (LLM) must read model weights from memory for each token when batch size is small, so arithmetic intensity is a very low 1–2 FLOP/Byte, and performance ends up tied to memory bandwidth even on accelerators rated at hundreds of TFLOP/s. In such situations the Roofline explains at a glance "why an expensive accelerator delivers only a few percent of peak," and shows which axis prescriptions like batching, quantization, and KV-cache compression move — which is why it has become a common language for AI inference and training optimization beyond HPC.

2. Core Concepts — Arithmetic Intensity and Two Ceilings

The overall structure of the Roofline model is as follows. Two ceilings provided by the hardware (peak compute performance and peak memory bandwidth) meet one characteristic of the kernel (arithmetic intensity) to determine "attainable performance."

flowchart TB
  HW["Hardware traits"] --> PEAK["Peak compute<br/>(FLOP/s, flat roof)"]
  HW --> BW["Peak memory bandwidth<br/>(Byte/s, slanted roof)"]
  SW["Kernel traits"] --> AI["Arithmetic intensity<br/>(FLOP/Byte)"]
  PEAK --> MODEL["Attainable performance<br/>= min(peak, bandwidth x intensity)"]
  BW --> MODEL
  AI --> MODEL
  MODEL --> DIAG{"Kernel position<br/>vs. ridge point"}
  DIAG -->|"left"| MEM["Memory bound<br/>(bandwidth limits)"]
  DIAG -->|"right"| COMP["Compute bound<br/>(compute limits)"]
  style PEAK fill:#e8f0fe,stroke:#2f6fed,stroke-width:2px
  style BW fill:#e6f4ea,stroke:#188038,stroke-width:2px
  style MEM fill:#fce8e6,stroke:#d93025,stroke-width:2px
  style COMP fill:#fef7e0,stroke:#f29900,stroke-width:2px

The model's core formula is attainable performance P = min(P_peak, B × I), where P_peak is the hardware's peak compute (FLOP/s), B is peak memory bandwidth (Byte/s), and I is arithmetic intensity (FLOP/Byte). When intensity is low, the B × I term is small and performance is tied to bandwidth (slanted roof); when intensity is high, B × I exceeds P_peak and performance is tied to compute (flat roof). The point where the two terms are equal, i.e. I = P_peak / B, is called the ridge point; to its left is memory bound, to its right compute bound.

Arithmetic intensity (also called operational intensity) is defined as the number of floating-point operations performed per byte exchanged with DRAM. The key is that the denominator is not "total data size" but "the traffic that actually moved to and from memory." For the same algorithm, increasing data reuse through cache blocking (tiling) reduces memory traffic, raising arithmetic intensity and shifting the kernel rightward on the graph to a higher performance ceiling. In other words, arithmetic intensity is not a fixed algorithmic constant but a design variable that moves with the implementation, and this is what makes the Roofline a tool that points toward optimization directions, not just diagnosis.

The Roofline uses log–log axes for a reason. Arithmetic intensity ranges thousands-fold, from 0.1 to hundreds of FLOP/Byte depending on the kernel, and performance is also broadly distributed, from tens of GFLOP/s to hundreds of TFLOP/s. Linear axes cannot fit this wide range on one screen, and the bandwidth-bound region becomes a curve hard to read by eye rather than a straight line. On log axes, "P = B × I" appears as a line of slope 1 (the slanted roof) and the peak compute ceiling as a horizontal line, so the two roofs form a clean kinked line and the ridge point and kernel position are intuitively readable. The choice of axes itself is a design decision that governs the model's legibility.

The graph below shows a typical Roofline curve and the positions of representative kernels. The horizontal axis is arithmetic intensity (FLOP/Byte), the vertical axis attainable performance (GFLOP/s), and the slanted region (bandwidth bound) kinks into the flat region (compute bound) at the ridge point.

{
  "type": "line",
  "data": {
    "labels": ["0.1", "0.25", "0.5", "1", "2", "4", "8", "16", "32", "64"],
    "datasets": [
      { "label": "Roofline ceiling (bandwidth 100GB/s, peak 1000GFLOP/s)", "data": [10, 25, 50, 100, 200, 400, 800, 1000, 1000, 1000], "borderColor": "#2f6fed", "tension": 0.1, "fill": false },
      { "label": "SpMV (AI=0.25, memory bound)", "data": [null, 25, null, null, null, null, null, null, null, null], "borderColor": "#d93025", "pointRadius": 7, "showLine": false },
      { "label": "Stencil (AI=2, memory bound)", "data": [null, null, null, null, 200, null, null, null, null, null], "borderColor": "#f29900", "pointRadius": 7, "showLine": false },
      { "label": "Dense GEMM (AI=32, compute bound)", "data": [null, null, null, null, null, null, null, null, 1000, null], "borderColor": "#188038", "pointRadius": 7, "showLine": false }
    ]
  },
  "options": {
    "plugins": { "legend": { "position": "bottom" }, "title": { "display": true, "text": "Roofline graph: attainable performance vs. arithmetic intensity (ridge point AI=10)" } },
    "scales": {
      "x": { "title": { "display": true, "text": "Arithmetic intensity (FLOP/Byte, log)" }, "type": "logarithmic" },
      "y": { "title": { "display": true, "text": "Attainable performance (GFLOP/s, log)" }, "type": "logarithmic" }
    }
  }
}

3. Reading the Roofline Graph — Ridge Point and Kernel Placement

The first step in using the Roofline in practice is to locate the ridge point. The ridge point I* = P_peak / B summarizes the "character" of that hardware. For example, a hypothetical chip with 1,000 GFLOP/s peak compute and 100 GB/s memory bandwidth has a ridge point at 10 FLOP/Byte. This means "only kernels performing 10 or more operations per byte can fully use the compute, and below that they are tied to memory." The further right the ridge point lies (the more bandwidth is relatively scarce versus compute), the more kernels fall into the memory-bound region. Modern GPUs and AI accelerators have such large arithmetic throughput that ridge points reach 10 to several hundred FLOP/Byte, so most ordinary kernels naturally become memory bound.

Placing representative kernels makes the model's usefulness clear. Sparse matrix–vector multiply (SpMV) performs 2 multiply-add operations per nonzero while reading at least 8 bytes (value + index + vector access), so its arithmetic intensity is an extremely low 0.1–0.25 FLOP/Byte, clinging to the far left of the slanted roof as a classic memory-bound case. Stencil operations have some reuse, so their arithmetic intensity is a moderate 1–4 and often still left of the ridge point. By contrast, dense matrix multiply (dense GEMM), when tiling n×n blocks, has O(n³) operations and O(n²) memory traffic, so arithmetic intensity rises to tens or hundreds of FLOP/Byte in proportion to block size, hitting the flat roof as compute bound. This placement difference explains both why the large GEMMs of deep-learning training achieve high accelerator peak utilization, and why small-batch LLM inference does not.

Computing arithmetic intensity by hand makes the model more concrete. Consider AXPY (y = a·x + y), a representative memory-bound kernel. Per element there are 2 FLOP (one multiply, one add), and memory moves 12 bytes in single precision: read x (4B), read y (4B), write y (4B). So arithmetic intensity is 2 ÷ 12 ≈ 0.17 FLOP/Byte, and on a hypothetical chip with a ridge point of 10 FLOP/Byte it can use only about 1.7% of peak compute (= 0.17 × 100 GB/s = 17 GFLOP/s / 1,000 GFLOP/s) — an extremely memory-bound case. This figure nails down, in one line of arithmetic, the conclusion that "AXPY gains nothing from more compute units; bandwidth alone determines its performance." In the same way, dense GEMM has arithmetic intensity roughly proportional to block size B (O(B³) operations / O(B²) traffic), so the larger the tile held in cache, the more it crosses the ridge point into compute bound.

How "close to the roof" a kernel is also matters. If the measured performance point lies just below the roof, there is almost no more to squeeze from that hardware (the bottleneck is a physical limit); if it falls far below the roof, implementation inefficiency (poor vectorization, cache misses, branching, load imbalance) keeps it under the ceiling, so there is large room for code optimization. This "gap between the point and the roof" tells you the upper bound of optimization ROI.

4. Extensions — Ceilings and Hierarchical Roofline

The basic Roofline draws only a single peak-compute line and a single bandwidth line, but real chips have several tiers of performance bound. The process of adding these as ceilings for finer diagnosis is as follows.

flowchart LR
  START["Profile kernel<br/>measure FLOP and memory traffic"] --> CALC["Compute arithmetic intensity<br/>I = FLOP / Byte"]
  CALC --> PLOT["Mark performance point on Roofline"]
  PLOT --> CHECK{"Judge position<br/>vs. ridge point"}
  CHECK -->|"memory bound"| MOPT["Data reuse, tiling<br/>compression, prefetch"]
  CHECK -->|"compute bound"| COPT["Vectorization, FMA<br/>ILP, more parallelism"]
  MOPT --> CEIL["Re-evaluate vs. higher ceiling"]
  COPT --> CEIL
  CEIL --> DONE{"Near the roof?"}
  DONE -->|"no"| PLOT
  DONE -->|"yes"| STOP["Physical limit reached<br/>reconsider hardware/algorithm"]
  style MOPT fill:#fce8e6,stroke:#d93025,stroke-width:2px
  style COPT fill:#fef7e0,stroke:#f29900,stroke-width:2px
  style STOP fill:#e6f4ea,stroke:#188038,stroke-width:2px

The compute ceiling represents instruction-level optimization stages. Without vectorization (SIMD) one is capped at the low ceiling of scalar peak; without FMA (fused multiply-add) at a ceiling about half of that. If a kernel's performance point sits at such a lower ceiling, no amount of memory tuning helps, and one must first secure vectorization, FMA, and instruction-level parallelism (ILP). The bandwidth ceiling represents data-supply optimization stages. Ignoring NUMA locality or omitting software prefetch and streaming stores drops effective bandwidth below half of peak, sinking the slanted roof downward.

More refined variants are the hierarchical roofline and the cache-aware roofline model (CARM). Since the memory hierarchy is split into L1, L2, LLC, and HBM/DRAM with different bandwidths at each stage, one draws separate arithmetic intensity and bandwidth roofs for each level. For example, an A100 GPU has HBM2e bandwidth of about 1.5–2.0 TB/s, while L2 and shared-memory bandwidth is several times higher. The same kernel may be memory bound by "DRAM-based arithmetic intensity" yet compute bound by "L2-based intensity," and the hierarchical roofline separates which level is the true bottleneck. This analysis lets one quantitatively prescribe "if tiling makes the working set fit in L2, HBM traffic disappears and the kernel shifts right (to higher performance)."

5. Comparison and Application Cases

Comparing the Roofline with other performance-analysis techniques clarifies its place. The key is "on which axis the bottleneck is seen" and "whether hardware bounds are drawn explicitly."

Aspect Roofline model Amdahl's law Profiler (hotspot) analysis
Axis viewed Arithmetic intensity vs. performance (bandwidth/compute) Parallel fraction vs. core count Execution time per function/line
Question answered Is the bottleneck compute or memory How much speedup from parallelization Where is time spent
Hardware limits States peak FLOP/s and bandwidth Not handled directly Not handled directly
Prescription Raise reuse or vectorization Shrink serial section Remove hotspots
Limitation Underweights latency, branching, imbalance Ignores memory bottleneck Unaware of position vs. ceiling

The key insight from this comparison is that the three techniques are complementary, not exclusive. When a profiler finds "which kernel spends the most time," the Roofline tells you "whether that kernel is bound by memory or compute and how much room remains to the ceiling," and Amdahl's and Gustafson's laws predict "how much total response time shrinks when that kernel is parallelized and optimized." Practical performance engineering weaves the three in sequence.

Consider concrete application cases. First, in optimizing an HPC fluid-dynamics stencil code, the Roofline showed the kernel clinging near the bandwidth roof (arithmetic intensity ~2), and rather than add compute units the team chose to raise reuse through temporal blocking, lifting arithmetic intensity and gaining several-fold performance. Second, in LLM inference serving, the Roofline explained that batch-1 decode is memory bound at arithmetic intensity ~1–2, and quantitatively justified why continuous batching (bundling multiple requests to raise weight reuse → rightward shift) and quantization (cutting weight bytes → higher bandwidth efficiency) are effective. Third, commercial tools like Intel Advisor and NVIDIA Nsight Compute auto-generate Rooflines, plotting hundreds of kernels' performance points on one screen and prioritizing the kernels farthest from the roof (with the most optimization room) for attack first.

6. Deep Dive — Recent Trends and Tools

The Roofline has steadily expanded alongside hardware evolution since 2009, and has become a core tool again in the age of AI accelerators. First is the maturing of automated measurement tools. Early on, peak performance and kernel arithmetic intensity were estimated by hand, but now the Empirical Roofline Toolkit (ERT) measures realistically attainable bandwidth and compute roofs with microbenchmarks, and Intel Advisor and NVIDIA Nsight Compute automatically collect FLOP and memory traffic via hardware counters to draw per-kernel Rooflines. This enables diagnosis on "realistically attainable roofs" rather than "theoretical peaks."

Second are roofs multilayered by tensor cores and mixed precision. Modern GPUs have greatly differing peak compute for FP64, FP32, FP16/BF16, INT8, and FP8. For instance, NVIDIA A100 is about 19.5 TFLOP/s for FP32 but about 312 TFLOP/s for tensor-core FP16 — more than 10× higher — so several tiers of flat roof appear by precision. The ridge point also shifts far right together (peak↑ ÷ bandwidth ≈ constant), yielding the structural insight that the lower the precision, the higher the arithmetic intensity needed to fully use the accelerator. This explains, via the Roofline, why mixed precision and quantization are directly tied to performance in deep learning.

Third is a renewed focus in the memory-centric era. Trends like HBM3, CXL-based memory expansion, and Processing-in-Memory all target the memory-bound problem of "compute to spare but data undeliverable." The Roofline provides a common yardstick for evaluating how much such new technologies push each axis (the slope of the slanted roof = bandwidth). Moreover, LLM-serving optimization research (FlashAttention's memory-traffic reduction, KV-cache compression, speculative decoding, etc.) can almost all be interpreted as attempts to "escape memory bound by raising arithmetic intensity or cutting bandwidth demand," and the Roofline has become the standard frame for explaining their effects.

7. Considerations and Implications

Points to keep in mind when applying and judging the Roofline model from a professional engineer's perspective:

  • Recognize and supplement the model's assumptions and blind spots. The basic Roofline assumes compute and memory fully overlap, so the slower of the two dominates. In reality, however, unhidden memory latency, branch mispredictions, synchronization and load imbalance, and power/thermal throttling pull performance below the roof. Thus use the Roofline to see "the ceiling and the nature of the bottleneck," and examine latency and imbalance factors together with profilers and trace analysis. Before concluding "it hasn't hit the roof, so it's a code problem," one must verify the room for latency hiding and the accuracy of measurement (how FLOP and bytes are counted).

  • Use it as the basis for investment decisions. The ridge point and kernel distribution directly answer the procurement and design question "should we buy more compute or more memory?" If most workloads cluster left of the ridge point (memory bound), the marginal utility of adding cores and accelerators is low, and high-bandwidth memory, memory expansion (HBM, CXL), or improving the algorithm's arithmetic intensity yields higher ROI. Conversely, if compute bound dominates, low-precision compute and dedicated accelerators are the answer. The Roofline presents this trade-off in numbers, backing TCO judgments for hardware and cloud-instance selection.

  • Take it as a rudder for algorithm and implementation redesign. Since arithmetic intensity is not a fixed value but a variable that moves with implementation, the Roofline offers, in design language, "what to fix to move the performance point in which direction." Tiling, fusion, and recompute cut memory traffic to push rightward; vectorization and FMA push upward. Defining a performance target not as "vaguely faster" but as a coordinate like "cross the ridge point into compute bound" makes the stopping condition of optimization work clear.

  • View prospects and related technologies together. As AI and HPC converge on the same hardware and precision and memory hierarchy become multilayered, one must diagnose with multiple rooflines by precision and by level, not a single roof. The Roofline is completed when woven with Amdahl's and Gustafson's laws (scaling limits), memory hierarchy and NUMA (data supply), and power-efficiency (performance/watt) analysis. In particular, extending to a "performance/watt roofline" from a sustainability and cost perspective lets one judge even energy efficiency within the same frame, so its importance as a decision tool for green computing and large-scale AI infrastructure design is expected to grow further.

References


In one line: The Roofline model is a visual performance model that plots a kernel's arithmetic intensity (FLOP/Byte) on the horizontal axis and peak compute and memory bandwidth as two roofs, diagnosing at a glance whether performance is compute bound or memory bound; the kernel's position relative to the ridge point and its gap from the roof quantify optimization direction, room, and investment decisions, and in the age of mixed precision, hierarchical memory, and LLM inference it has expanded to multiple roofs to become the common diagnostic language of HPC and AI acceleration.