CXL (Compute Express Link)

1. Overview

CXL (Compute Express Link) is an open interconnect standard that links heterogeneous computing elements — CPUs, GPUs, accelerators, memory expanders — over the PCIe physical layer while maintaining cache coherency at low latency.

As AI and HPC (high-performance computing) workloads explode, the data center bottleneck is no longer raw compute but memory bandwidth, capacity, and sharing efficiency. In a traditional server, DRAM is physically bound to each CPU socket's DDR channels, so memory left idle on one node cannot be borrowed by another — a structural problem known as stranded memory. Hyperscaler analyses report that a substantial fraction of server memory (10–25% depending on the study) sits idle this way, and when memory accounts for 40–50% of server cost, this represents enormous capital waste.

The root cause is the premise that "memory is tied to the CPU." PCIe, as a general-purpose I/O link, has scaled its bandwidth but cannot guarantee cache coherency, so an accelerator wishing to share CPU memory had to resort to cumbersome DMA copies and software synchronization. CXL targets exactly this point: it reuses the PCIe physical and electrical layers while adding a cache-coherency protocol, enabling a structure where "accelerators and CPUs view the same memory as a single address space." In short, CXL emerged along three axes: ① the coherency demand driven by pervasive heterogeneous acceleration, ② the need to expand memory capacity and bandwidth, and ③ resource disaggregation and TCO (total cost of ownership) reduction through memory pooling.

Examined more closely, the limits of the old structure appear in both directions. The first is rigidity of capacity and bandwidth. The number of DDR channels is physically fixed by the CPU socket, so even when you want to add memory you cannot exceed the per-socket channel and slot limits, and bandwidth is tied in lockstep, scaling only proportionally. The second is the impossibility of sharing. For server B to use server A's spare memory requires a separate path such as RDMA over the network, which pushes memory access latency into the microsecond range — effectively turning "memory" into "storage." CXL breaks through both limits while preserving load/store memory semantics, which is why it is regarded not as a mere connection spec but as a paradigm shift in data center memory architecture.

A defining trait of CXL is its openness and backward compatibility. Launched in 2019 under Intel's leadership, the CXL Consortium subsequently absorbed rival standards such as Gen-Z and OpenCAPI to become the de facto industry standard, and every higher version guarantees full compatibility with lower ones. Data centers can thus preserve their existing PCIe ecosystem investment while migrating to CXL incrementally. This openness, aligned with the demands of hyperscalers, telcos, and financial data centers that seek to avoid single-vendor lock-in, is why CXL has come to be accepted as a common infrastructure language transcending the walls of any one CPU or GPU maker.

2. Overall CXL Architecture and Protocols

CXL dynamically multiplexes three sub-protocols over a single physical link. The figure below shows CXL's overall stack.

graph TD
    subgraph Host["Host (CPU)"]
        CPU["CPU core + cache"]
        HA["Home Agent"]
        CTRL["CXL controller"]
    end
    subgraph Protocol["CXL protocols (multiplexed)"]
        IO["CXL.io (discovery/config/DMA)"]
        CACHE["CXL.cache (accelerator to host memory coherency)"]
        MEM["CXL.mem (host to device memory access)"]
    end
    subgraph PHY["Physical layer"]
        FLIT["Flit-based transport"]
        PCIE["PCIe PHY (electrical/physical reuse)"]
    end
    subgraph Device["CXL device"]
        ACC["Accelerator (GPU/NPU)"]
        MEMEXP["Memory expander (DRAM/CXL)"]
    end
    CPU --> HA --> CTRL
    CTRL --> IO
    CTRL --> CACHE
    CTRL --> MEM
    IO --> FLIT
    CACHE --> FLIT
    MEM --> FLIT
    FLIT --> PCIE
    PCIE --> ACC
    PCIE --> MEMEXP

A. CXL.io — functionally equivalent to the PCIe transaction layer, it handles device discovery, enumeration, register access, interrupts, and DMA — the basic I/O. CXL.io is the mandatory "skeleton" every CXL device must support; boot-time initialization and configuration run entirely over it. Using only CXL.io makes a device essentially indistinguishable from a PCIe device, and the benefits of cache coherency emerge only in the next two protocols.

B. CXL.cache — this protocol lets a device (accelerator) cache host (CPU) memory while maintaining coherency. A GPU or NPU pulls a region of CPU memory into its own cache to compute on, and any changes are automatically synchronized with the CPU cache through a MESI-family coherency protocol. Developers can therefore handle data as if sharing pointers, without explicit copy/sync code, greatly improving the productivity of heterogeneous parallel programming. For example, if a recommender system keeps a large embedding table in CPU memory and the NPU references it via CXL.cache, the repeated overhead of PCIe DMA copies can be eliminated.

C. CXL.mem — conversely, this protocol lets the host (CPU) directly access (load/store) memory attached to a device as part of its own physical address space. This is the core of memory expansion and pooling. From the CPU's view, DRAM connected via CXL.mem looks like a slightly slower NUMA node, and the operating system recognizes it as a separate memory tier for page placement. CXL memory's access latency is reportedly about twice that of local DDR (tens to ~100ns additional), making it suited to bandwidth- and capacity-sensitive workloads that are less latency-sensitive.

That three protocols coexist on one link is a crucial design point. An AI accelerator (Type 2), for instance, is enumerated over CXL.io at boot and configuration, caches input tensors from host memory coherently via CXL.cache, and exposes its own HBM/DRAM to the host via CXL.mem. Because protocols switch by moment and purpose over the same physical path, complex data-movement patterns can be served by a single connection without needing multiple separate links — this is CXL's structural elegance.

How CXL reuses PCIe at the physical layer also deserves note. CXL reuses PCIe's electrical signaling, connectors, and form factors as-is, but at link initialization it goes through alternate protocol negotiation to decide whether to use the slot in PCIe mode or CXL mode. To reduce latency it also transmits in fixed-size Flit (flow control unit) units rather than PCIe's variable-length packets (TLPs); CXL.cache/mem, being latency-sensitive, use a small 68-byte Flit path, while CXL.io, where bandwidth matters, uses larger Flits — optimizing the path accordingly. This design philosophy of "layering coherency atop proven PCIe assets" lets CXL reuse existing server board designs almost unchanged without dedicated wiring, a decisive factor that greatly accelerated ecosystem adoption.

According to the combination of these three protocols, CXL devices are classified into three types. The table below summarizes the differences, but the essence is "which direction of coherency is needed." Type 1 is a compute accelerator (NIC, smartNIC) that uses host memory, requiring CXL.cache; Type 2 is a GPU/FPGA that has its own memory and shares bidirectionally with the host, using all three protocols; Type 3 is a pure memory expander centered on CXL.mem.

Type	Protocols used	Representative devices	Memory characteristics
Type 1	CXL.io + CXL.cache	smartNIC, compute NIC	Caches host memory
Type 2	CXL.io + CXL.cache + CXL.mem	GPU, FPGA, AI accelerator	Bidirectional host/device memory sharing
Type 3	CXL.io + CXL.mem	Memory expansion/pooling device	Provides device memory to host

3. Version Evolution and the Memory Pooling Architecture

CXL evolved rapidly in a short span, with each version keeping pace with the bandwidth gains of PCIe generations. 1.0/1.1 (2019) focused on single-node accelerator and memory expansion; 2.0 (2020) introduced switches to enable pooling, in which multiple hosts share one memory pool. 3.0 (2022) doubled bandwidth to 64GT/s on PCIe 6.0 and added multi-level switching, memory sharing, and the fabric concept; 3.1 (announced November 2023) reinforced PBR (Port Based Routing) switches, direct device-to-device P2P communication, and trusted execution environment (TEE) security, extending to a disaggregation architecture beyond the rack.

It is important here to distinguish pooling from sharing. Pooling allocates one memory resource exclusively to several hosts over time (owned by only one host at a time), whereas sharing lets multiple hosts access the same region concurrently with coherency. Sharing came into its own in CXL 3.0 along with the Dynamic Capacity Device (DCD) and Global Fabric Attached Memory concepts. The figure below shows the resource-disaggregation structure of CXL switch-based memory pooling.

flowchart LR
    H1["Host A (CPU)"]
    H2["Host B (CPU)"]
    H3["Host C (CPU)"]
    SW["CXL switch (Fabric Manager)"]
    M1["Memory module 1 (DRAM)"]
    M2["Memory module 2 (DRAM)"]
    M3["Memory module 3 (CXL/expansion)"]
    H1 -->|CXL.mem| SW
    H2 -->|CXL.mem| SW
    H3 -->|CXL.mem| SW
    SW -->|dynamic allocation| M1
    SW -->|dynamic allocation| M2
    SW -->|dynamic allocation| M3
    FM["Management software: capacity reallocation"]
    FM -.control.-> SW

Version-by-version bandwidth evolution is also a key design variable. CXL 1.1/2.0, on PCIe 5.0 at 32GT/s, provides about 64GB/s (unidirectional) over x16 lanes, while 3.0/3.1, on PCIe 6.0 at 64GT/s, doubles this to roughly 128GB/s. However, PCIe 6.0 introduces PAM4 modulation and FEC (forward error correction), so balancing the latency increase from error correction against Flit design is important. In capacity terms, if 2.0's single-level switch handles pools of tens of units, 3.0's multi-level switching scales to fabrics of hundreds to thousands of nodes — so version and topology must be chosen to match deployment scale and target utilization.

In this structure, the Fabric Manager acts as the orchestrator that dynamically detaches and attaches memory-module capacity according to each host's demand. For instance, if a nightly batch job on host A demands a large amount of memory, the Fabric Manager allocates idle modules to A and reclaims them after the job to reassign to another host. This is the core value of memory disaggregation: reducing stranded memory lowers total installed memory and raises utilization. Microsoft Azure's research prototype "Pond" showed that CXL pooling could save roughly 9–10% of DRAM demand, which at hyperscale translates into hundreds of millions of dollars in savings.

4. Comparison: CXL vs. Traditional PCIe, NVLink, NUMA

To understand CXL's position, comparison with adjacent technologies is needed. On the surface they all look like "fast connections," but the problem each solves and its coherency model differ fundamentally.

Category	PCIe	CXL	NVLink	Traditional NUMA (QPI/UPI)
Cache coherency	None	Yes (asymmetric)	Yes	Yes (symmetric)
Physical layer	PCIe	PCIe reuse	Dedicated	Dedicated socket-to-socket
Openness	Open	Open	Proprietary (NVIDIA)	Proprietary (CPU vendor)
Primary use	General I/O	Memory expansion/pooling/accelerators	GPU-to-GPU high-speed link	CPU socket-to-socket
Scope	Within node	Node to rack fabric	GPU cluster within node	Socket-to-socket

The difference from PCIe hinges on the presence of coherency. With PCIe alone, an accelerator using CPU memory requires software to manually manage cache flushes and DMA, whereas CXL guarantees this automatically in hardware. One practical implication is that CXL's coherency is asymmetric: the host plays the coherency "home" role and the device follows, which lowers device design complexity and eases interoperability across vendors. NVIDIA's NVLink, by contrast, specializes in symmetric high-bandwidth GPU-to-GPU connection and delivers overwhelming bandwidth inside AI training clusters, but has the limitation of a proprietary ecosystem. In real AI data centers, therefore, a complementary combination of NVLink between GPUs and CXL for memory expansion/pooling has realistically taken hold.

Another point to note is that CXL and PCIe are not competitors but closer to an inclusion relationship. CXL does not replace PCIe but selectively layers coherency atop it, so the same slot accommodates both PCIe devices (NVMe SSDs, general NICs) and CXL devices (memory expanders) depending on the situation. This lets data center operators flexibly reallocate slot resources and lowers investment risk. Conversely, NVLink and proprietary socket interconnects demand dedicated wiring and specific-vendor silicon, so in flexibility terms CXL's open standard is judged advantageous over the long run.

Comparison with NUMA is also important. Traditional UPI/QPI-based multi-socket NUMA is limited to 2–8 sockets and tied to the CPU vendor, whereas CXL expands the memory tier far more flexibly through switches and fabrics. In practice the Linux kernel exposes CXL memory as an additional NUMA node and, via a tiering mechanism, automatically moves hot pages to local DDR and cold pages to CXL memory, minimizing the latency penalty.

5. Deep Dive: Practical Application in AI Data Centers and Latest Trends

The area where CXL draws the most attention is large-scale AI infrastructure. LLM (large language model) inference and training require hundreds of GB to several TB of parameters and KV cache, and while HBM (high-bandwidth memory) offers excellent bandwidth, its capacity and cost limits are stark. Here CXL does not replace HBM but serves to expand the lower end of the memory hierarchy. That is, in a hierarchy of GPU HBM (tier 1) → CPU local DDR (tier 2) → CXL expansion memory (tier 3), offloading infrequently used parameters or long-context KV cache to CXL memory compensates for the shortage of costly HBM capacity. Samsung and SK hynix have launched CXL-based memory expansion modules (such as CMM-D), offering hundreds of GB to TB-class capacity in a PCIe slot form factor.

Application cases are also spreading in the database and analytics domains. In-memory databases and large OLAP (online analytical processing) engines see performance plummet when the working set spills to disk upon exceeding local DRAM; placing CXL expansion memory as a second tier of the buffer pool delays disk spill and keeps throughput stable. Compared with microsecond-scale disk/network access, CXL memory at hundreds of nanoseconds is over an order of magnitude faster, effectively filling the "middle tier" between capacity and latency.

From the memory-semiconductor industry's perspective, CXL is a strategic turning point. Memory vendors had been confined to competing on commodity parts under the DDR spec, but CXL offers an opportunity to broaden into high-value memory solutions combining controllers, firmware, and security features. As of 2024–2026 CXL is still in an early commercialization stage: with server CPUs supporting CXL 1.1/2.0 — Intel Sapphire Rapids/Emerald Rapids, AMD Genoa — becoming widespread, adoption is beginning, centered on Type 3 memory expansion. Since fully leveraging pooling and sharing presupposes a mature CXL 2.0/3.0 switch ecosystem and Fabric Manager software, the realization of a fully disaggregated data center is expected to unfold gradually.

Seen in concrete numbers, the effect becomes clearer. Serving an LLM of 70B–175B parameters in FP16 means weights alone reach 140–350GB, and in multi-user inference the KV cache occupies tens to hundreds of GB more. When a single GPU's HBM is on the order of 80–192GB, attaching several TB of expansion memory to the CPU side via CXL for model sharding and cache offload lets you accommodate longer contexts and more concurrent users without adding GPUs. This is a typical design pattern that raises the utilization of expensive accelerators through a division of labor where "GPUs handle compute, CXL handles capacity."

The evolution on the security side is also noteworthy. When memory is shared and moved across multiple hosts, the risk of one tenant's data being exposed to another grows. Accordingly, CXL 3.1 strengthens CXL IDE (Integrity and Data Encryption) and TEE (trusted execution environment) integration to support data encryption and integrity on the link plus tenant isolation. This is a prerequisite for safely using CXL pooling in multi-tenant clouds, and can be seen as a case of extending zero-trust principles down to the memory fabric layer.

6. Considerations and Implications

From a professional engineer's perspective, the following should be judged comprehensively when reviewing CXL adoption.

• Designing the latency-capacity trade-off: CXL memory has about twice the access latency of local DDR, so it is unsuited to latency-sensitive workloads (OLTP, real-time trading) and should be selectively applied to bandwidth- and capacity-centric in-memory analytics, AI inference KV cache, database buffer pool expansion, and the like. The quality of the memory-tiering policy and the hot/cold page classification algorithm determines effective performance.

• Ecosystem maturity and an incremental migration strategy: Since CXL 2.0/3.0 switches and Fabric Managers are still early-stage, a realistic roadmap is to resolve capacity bottlenecks with Type 3 memory expansion for now and expand pooling and sharing in phases through PoCs (proofs of concept). To avoid specific-vendor lock-in, open-standard compliance and interoperability verification should be included in procurement criteria.

• Economic analysis from TCO and sustainability perspectives: Because memory pooling reduces stranded memory and thereby cuts installed volume and power simultaneously, investment feasibility should be quantified not only in capital expenditure (CapEx) but also operating expenditure (OpEx) and carbon emissions (ESG). That said, a total-cost analysis reflecting the added cost of switches and controllers and the complexity of software operations is needed.

• Security, isolation, and operational governance: In multi-tenant environments, memory sharing entails side-channel and data-remanence risks, so CXL IDE/TEE-based encryption and memory sanitization policies should be mandated, and the Fabric Manager's privileges and access control should be designed on zero-trust principles. Availability design that evaluates the blast radius of shared memory on many hosts upon failure should proceed in parallel.

• Integration with related technologies: CXL delivers value less in isolation than when combined with HBM, NVLink, RDMA, Kubernetes resource scheduling, and the like. Especially in AI infrastructure, it is advisable to proactively establish a memory and connectivity architecture blueprint that clearly divides the roles of HBM (bandwidth), CXL (capacity), and NVLink (GPU-to-GPU).

• OS and software stack readiness: CXL's value is not realized by hardware alone; it requires a kernel version supporting memory tiering, hot-page promotion, and NUMA balancing, along with Fabric Manager and orchestration software. An adoption roadmap therefore needs a comprehensive plan covering not just hardware procurement but OS upgrades, observability, and the upskilling of operations staff.

References

• CXL Consortium, "Compute Express Link 3.1 Specification Release" (2023.11), https://jotup.co/node/2659156
• Design & Reuse, "CXL 3.1: What's Next for CXL-based Memory in the Data Center", https://www.design-reuse.com/blog/55790-cxl-3-1-what-s-next-for-cxl-based-memory-in-the-data-center/
• SNIA, "CXL Expanded Capabilities" (CMSS23), https://www.snia.org/sites/default/files/cmss/2023/SNIA-CMSS23-Rudoff-CXL-Expanded-Capabilities.pdf

---

In one line: CXL is an open standard that adds cache coherency atop the PCIe physical layer to link CPUs, accelerators, and memory as if in a single address space; through its three protocols — CXL.io/cache/mem — and Type 1–3 devices it realizes memory expansion, pooling, and sharing, serving as a core infrastructure technology that resolves the stranded-memory problem and capacity bottleneck of AI data centers.