GPU Virtualization and Sharing Technologies (vGPU, MIG, MPS, Time-Slicing)

1. Overview

GPU virtualization is the umbrella term for technologies that abstract a single physical GPU so that multiple virtual machines, containers, or processes can safely share it, or conversely bundle several GPUs into what looks like one logical resource. In the narrow sense it refers to the partitioning techniques that let several users share one GPU, implemented along three axes: temporal, spatial, and hardware partitioning.

GPU virtualization rose to prominence because of a structural gap between accelerator demand and resource efficiency. As demand for generative AI, deep-learning training, inference, and HPC exploded, the GPU became the most expensive and most supply-constrained resource in the data center. The acquisition cost of a single NVIDIA H100 runs into the tens of thousands of dollars, and once power and cooling are added, even 1% of idle capacity translates directly into cost. Yet in practice it is rare for a single workload to fill a GPU to 100%. Workloads such as interactive inference services, notebook-based data analysis, and small test runs in CI pipelines often use only 10–30% of the GPU's compute units (SMs) and memory, leaving the rest idle. Filling this gap of "low utilization of an expensive resource" is the primary reason GPU-sharing technology exists.

A second driver is the demand for multi-tenancy and isolation. In public clouds and shared internal clusters, different teams or customers divide the same GPU, yet one tenant's work must not peek into another tenant's memory or erode its performance. In other words, isolation that lets tenants "share while not interfering with one another" is a precondition. GPU virtualization techniques diverge precisely by where they strike a compromise between the two conflicting goals of utilization and isolation. They form a spectrum ranging from time-slicing, which nearly abandons isolation to chase utilization, to MIG, which isolates fully at the hardware level but fixes the partition granularity.

A third driver is the asymmetry with CPU virtualization. CPU virtualization has matured over decades so that a hypervisor can freely partition and overcommit cores and memory, but a GPU is a special device with thousands of cores (SMs), dedicated memory, and its own scheduler, so a general-purpose hypervisor cannot handle it directly. GPU vendors therefore came to provide dedicated virtualization means at the driver, firmware, and hardware layers, and the results are the time-slicing, MPS, MIG, and vGPU discussed below.

How much this gap translates into cost is revealed by simple arithmetic. If a GPU node that costs 40,000 dollars a year in acquisition and operation combined stays at an average utilization of 20%, roughly 32,000 dollars' worth of compute per year is effectively thrown into the void. If the same node is divided among four workloads via sharing technology to raise average utilization to 70%, the available compute grows 3.5-fold without any additional hardware investment. In AI clusters of hundreds to thousands of GPUs, this utilization gap leads to differences in capital and operating costs on the order of billions of won, so GPU sharing takes on the character of a financial decision rather than a mere technical choice.

2. Classification of Sharing Methods and the Overall Structure

The best framework for understanding GPU-sharing techniques is "what is used as the basis for division." A GPU's resources divide broadly into compute resources (SMs, Streaming Multiprocessors) and memory resources (VRAM, memory bandwidth, L2 cache), and sharing techniques are classified by whether these are used alternately over time or carved up spatially for simultaneous use. The figure below places the four representative techniques on the axes of temporal, spatial, and hardware partitioning.

flowchart TB
  ROOT["GPU sharing technologies"]
  ROOT --> T["Temporal partitioning"]
  ROOT --> S["Spatial partitioning"]
  ROOT --> H["Hardware partitioning"]
  T --> TS["Time-Slicing<br/>run in turns via context switching"]
  T --> VG["vGPU (default mode)<br/>hypervisor time-slices"]
  S --> MPS["MPS (Multi-Process Service)<br/>kernels from many processes run together"]
  H --> MIG["MIG (Multi-Instance GPU)<br/>physically partitions SM, memory, cache"]
  H --> VGM["MIG-backed vGPU<br/>hardware isolation + VM assignment"]
  TS -.->|Weak isolation, high utilization| G1["overcommit-oriented"]
  MIG -.->|Strong isolation, fixed granularity| G2["multi-tenancy-oriented"]

Temporal sharing is a method in which multiple jobs occupy one GPU in turns over short time slices. The principle is the same as CPU round-robin scheduling: at any instant only one job uses the GPU, but to the human eye they appear to run concurrently. Each switch incurs a context-switch cost, and while one job holds the GPU the others wait, so latency grows. Memory is also not physically divided, so if one job hogs VRAM excessively another may fail with OOM (Out of Memory). In exchange for weak isolation, it is simple to implement and free to overcommit, making it suitable for development and test environments that prioritize utilization above all.

Spatial sharing is a method in which multiple jobs divide the GPU's compute units at the same instant. To keep thousands of SMs from sitting idle, when one job uses only some SMs the leftover SMs are filled by another job's kernels. The representative implementation, MPS, belongs here. It achieves higher throughput than temporal sharing, but memory space and faults are still not fully separated.

Hardware partitioning physically carves up the interior of the GPU so that each slice has its own SMs, its own memory, its own memory controllers, and its own L2 cache slice. MIG is representative: a partitioned instance effectively behaves like a small, independent GPU, so performance interference (noisy neighbor) and memory intrusion are blocked at the source. Isolation is the strongest, but because the partition boundaries are fixed in hardware, flexibility is lower.

3. Principles and Architecture of the Main Techniques

The four techniques give different answers to the single question "where do you compromise between utilization and isolation." Below we examine the principles in order from weak to strong isolation. Each technique's differences become clear when compared by the kinds of isolation it provides (memory isolation, performance isolation, fault isolation).

A. Time-Slicing

Time-slicing is the most basic form of sharing, in which the GPU's built-in scheduler runs multiple CUDA contexts in turns over time slices. When one job's kernel execution finishes or its time slice expires, the GPU saves execution state such as registers and the program counter, restores the next job's state, and continues execution. This switch is on the order of a few to tens of microseconds, but when switching is frequent the cumulative overhead grows.

The decisive limitation of time-slicing is that there is almost no isolation. Because memory is not physically separated, two jobs share the same VRAM pool, and one job's memory leak or over-occupancy spreads into another job's failure. Furthermore, if one job runs a long kernel, another job's latency grows unpredictably, producing performance interference. When you configure time-slicing in Kubernetes, you can advertise one physical GPU as, say, 4 "replicas" and assign them to 4 pods; this does not mean memory is quartered, only that there are 4 scheduling slots. Thus if you allocate beyond the total memory, OOM results all the same.

Time-slicing is nonetheless widely used because it has no hardware constraints (it works on every GPU) and its configuration is simple. In environments where "availability and cost matter more than isolation"—an internal Jupyter notebook cluster used by dozens of people, a CI runner that uses the GPU only intermittently—it is actually reasonable to oversubscribe one GPU to 8–16 jobs to fill the idle time.

When setting the oversubscription factor, however, the ground should be the fraction of jobs active at once. If 16 people keep notebooks open but on average only 2–3 are actually running cells, an 8x oversubscription is safe, but the moment everyone runs a large training job simultaneously, context switches surge and VRAM is instantly exhausted. Therefore, the operating principle that reduces failures is to apply time-slicing only to workloads with "low peak concurrency and easy retries," and to attach a memory cap as well.

B. MPS (Multi-Process Service)

MPS is a sharing method that lets CUDA kernels submitted by multiple processes run simultaneously (spatially) within a single GPU context. If time-slicing runs them "in turns," MPS runs them "together." Originally CUDA could not run kernels from different processes concurrently and serialized them; MPS merges the processes' kernels into one server context and executes them concurrently at the SM level, raising utilization by stacking small workloads that a single process cannot fully fill the GPU with.

Why MPS raises throughput is explained by the arithmetic of SM occupancy. If some inference kernel uses only 30% of the GPU's SMs, then when run alone the remaining 70% of the compute units do nothing during that time. If three processes place the same kernel together via MPS, the three kernels fill different SMs and occupancy rises to as much as 90% in theory, so the same hardware processes nearly three times as many requests. Under time-slicing the three jobs would take turns, leaving 70% empty in each interval; the difference between "in turns" and "together" lies precisely in this reclamation of idle SMs.

The benefit of MPS is that there is no context-switch cost and that small kernels fill leftover SMs to raise throughput. Also, since the Volta generation, features were added to cap each client to a ratio of compute resources (e.g., 50% of the GPU) along with a degree of memory protection. Fundamentally, however, MPS has weak fault isolation. Because it shares a single GPU context, if one process raises a fatal error (e.g., a memory access violation) it can affect the other processes bound to the same MPS server. Memory protection is also closer to a software boundary than full hardware isolation. MPS is therefore most effective in single-tenant environments where throughput matters more than isolation, such as a multi-process inference service among mutually trusted parties (same team, same job). For instance, when placing several small latency-sensitive inference models on one GPU, running them concurrently with MPS can lower each request's tail latency while raising overall throughput.

C. MIG (Multi-Instance GPU)

MIG is a hardware-level GPU partitioning technology introduced in the NVIDIA Ampere generation (A100, A30) and carried into the Hopper (H100, H200) and Blackwell (B200) generations. It carves one GPU into up to 7 independent instances (GPU Instances), each with its own set of SMs, its own VRAM, its own memory bandwidth, its own L2 cache slice, and its own memory controllers. That is, the partition lines are drawn in silicon rather than by a software scheduler, so one instance's work cannot physically reach another instance's memory or performance. This is the basis for the full memory, performance, and fault isolation that MIG provides.

flowchart LR
  subgraph GPU["Physical GPU (e.g., H100 80GB)"]
    direction TB
    subgraph I1["GI 3g.40gb"]
      SM1["SM slices x3"]
      M1["VRAM 40GB"]
    end
    subgraph I2["GI 2g.20gb"]
      SM2["SM slices x2"]
      M2["VRAM 20GB"]
    end
    subgraph I3["GI 1g.10gb"]
      SM3["SM slice x1"]
      M3["VRAM 10GB"]
    end
    L2["L2 cache and memory controllers are also partitioned per slice"]
  end
  I1 --> T1["Tenant A (training)"]
  I2 --> T2["Tenant B (inference)"]
  I3 --> T3["Tenant C (notebook)"]

MIG's division unit is provided as fixed combinations called profiles. Taking the H100 80GB as an example, you can choose only predetermined sizes from 1g.10gb (the smallest slice, up to 7) through 2g.20gb, 3g.40gb, and 7g.80gb (the whole GPU), and the slice size is denoted by the combination of "number of compute slices (g)" and "memory capacity (gb)." Profile names and possible combinations differ by SKU; for example, the H100 94GB product uses different memory notations such as 1g.12gb and 7g.94gb. Because the granularity is thus fixed in hardware, you cannot create an arbitrary size such as a "7.5GB memory instance," and changing the partition configuration carries the constraint that you must drain all jobs on that GPU and reconfigure.

MIG is a core means of multi-tenant clouds. For example, if you carve one H100 into seven 1g.10gb instances, you can place seven small inference/experiment workloads fully isolated from one another, so that one tenant's runaway does not affect the others. The reason a cloud provider can sell "1/7 of a GPU" as a product, or an enterprise can establish billing and quota schemes that allocate GPU slices by team, is likewise thanks to this hardware isolation.

Another strength of MIG is the predictability of performance. Because a partitioned instance has dedicated memory bandwidth and cache, its share of performance is guaranteed to be constant no matter what the neighboring instances do. This matters especially for inference services that must promise an SLA: under time-slicing or MPS latency sways with the neighbor's load, but under MIG the variance of tail latency is small, so billing and SLA design become simpler. Conversely, because the partition granularity is fixed, fragmentation can arise where "one slice is free but a job larger than it waits," so capacity planning that designs the profile configuration to match the workload distribution in advance is needed.

D. vGPU (Virtual GPU)

vGPU is a technology that divides a physical GPU among multiple virtual machines (VMs) at the hypervisor layer, used mainly in VM-based virtualization environments (VMware vSphere, KVM, Citrix, etc.) and in VDI (virtual desktops) and cloud IaaS. A vGPU driver is installed in each VM, and a manager that cooperates with the hypervisor sits on the physical GPU to mediate the GPUs requests of the VMs. Default vGPU assigns the GPU to VMs in turns by time-slicing, but it guarantees separation on the memory side by fixing the framebuffer (VRAM) size allocated to each VM with a profile. vGPU is distinguished from the preceding three techniques by being a licensed commercial software stack.

Noteworthy is MIG-backed vGPU, which combines vGPU with MIG. In this configuration, instances hardware-partitioned by MIG are assigned to each VM as vGPUs, so you obtain even the hardware isolation of compute performance—which time-sliced vGPU cannot provide—at the VM granularity. In other words, it is a way to take "the operational convenience of VM virtualization + the strong isolation of MIG" at once, and is preferred in virtualized clouds with strict inter-tenant isolation requirements, such as finance and the public sector.

4. Comparison of Techniques and Selection Criteria

The difference among the four techniques ultimately comes down to which isolation is guaranteed in hardware, and how much flexibility and generality is given up in exchange. The table below is an aid to comparison; "why such a difference arises" should be understood from the principles in the preceding sections.

Category	Time-Slicing	MPS	MIG	vGPU (default)
Partition axis	Time	Space	Hardware	Time (can combine MIG)
Memory isolation	None	Weak (SW)	Strong (HW)	Yes (profile)
Performance isolation	None	Partial	Strong (HW)	Weak
Fault isolation	None	Weak	Strong	Per VM
Partition granularity	Free (slot count)	Free	Fixed (profile)	Fixed (profile)
Hardware requirement	Every GPU	Most GPUs	A100–B200 etc.	Supported GPU + license
Representative use	Dev/test overcommit	Single-tenant inference	Multi-tenancy billing	VDI, VM cloud

A common misconception is the expectation that "sharing always raises overall throughput." In reality, if a workload already saturates the GPU, sharing only adds switching and contention overhead and actually slows things down. The premise under which sharing pays off is that each job fails to fill the GPU and leaves slack in SMs and memory; without this slack the right answer is dedicated allocation, not partitioning. Therefore, before adopting sharing, you must profile the actual SM occupancy and VRAM usage of representative workloads to confirm whether there is room to share.

The key to read from this table is the trade-off running vertically, "the stronger the isolation, the lower the granularity and generality." Time-slicing divides freely on any GPU but has no isolation, while MIG has perfect isolation but demands the fixed frame of profiles and specific-generation hardware. The choice therefore starts from "does my workload share a trust boundary or not." If the jobs are trusted work from the same team, it is better to maximize utilization with time-slicing/MPS; if it is multi-tenancy where different customers and teams mix, you must secure hardware isolation with MIG (or MIG-backed vGPU).

In practice these are sometimes combined hierarchically. For example, after carving an H100 with MIG into two 3g.40gb plus one 1g.10gb, you layer MPS again on top of the inference MIG instance to run several small models under spatial partitioning. This makes possible a "two-tier structure of isolation and efficiency" that isolates the outer boundary strongly in hardware while raising utilization again among trusted internal workloads. In fact, there are reports that one inference platform raised GPU utilization several-fold with a MIG+MPS combination versus single-GPU-occupancy, so combination design is a key lever for cost reduction.

5. Deep Dive: Kubernetes Integration and the Shift to DRA

The field where GPU sharing actually operates is mostly Kubernetes. Traditionally Kubernetes has handled GPUs with the Device Plugin model. When the node's NVIDIA device plugin advertises an extended resource such as nvidia.com/gpu, the scheduler assigns a whole GPU according to the pod's resource request. Adding time-slicing configuration on top lets you advertise one GPU as several replica slots, and MIG can be exposed with single (all instances the same profile) or mixed (different profiles blended) strategies. However, the device-plugin model was designed to treat GPUs as "whole units," and its limitation of not being able to natively express sharing, per-workload configuration, topology requirements, and capability constraints was pronounced.

What emerged to overcome this limitation is DRA (Dynamic Resource Allocation). DRA is a model in which, when a workload declares via a ResourceClaim "what capability of resource it needs and how much," a driver dynamically allocates and configures the GPU to meet that need, positioning itself as the successor architecture to the device plugin. The related API (resource.k8s.io/v1) reached GA in Kubernetes 1.34, and NVIDIA's DRA driver lets you declaratively configure sharing methods such as time-slicing and MPS on this framework (some features are still provided via alpha-stage feature gates). Note, however, that you cannot use the DRA stack and the existing device-plugin stack simultaneously in one cluster, so the operator must choose one of the two.

Another axis of recent trends is virtual memory oversubscription and KV cache sharing. In large language model inference, model weights and the KV cache eat into VRAM, so software layers that use Unified Memory and the CUDA Virtual Memory Management (VMM) API to overcommit GPU memory, or that let multiple inference requests share KV cache pages (e.g., page-level attention cache management), are developing rapidly. That is, "static isolation" centered on hardware partitioning (MIG) and "dynamic sharing/overcommit" centered on software are maturing together in both directions, and DRA can be seen as an attempt to unify the two under a declarative interface.

In addition, at the vGPU layer, the billing/license model and mode choice (compute-oriented vCS, workstation vWS, etc.) enter as variables of operational design. When evaluating sharing technology, you must weigh not only pure performance and isolation characteristics but also license cost, supported hardware generation, and the maturity of hypervisor and Kubernetes integration together to reach an optimum from the total-cost-of-ownership (TCO) perspective.

6. Considerations and Implications

Adoption strategy (branching by workload nature): The starting point of GPU-sharing design is the "trust boundary." Trusted batch and development workloads from the same team should maximize utilization with time-slicing/MPS, while multi-tenancy that mixes external customers and other organizations must secure hardware isolation with MIG. Latency-sensitive real-time inference, for which performance interference is fatal, is safest avoiding time-slicing and using dedicated MIG instances or MPS cap settings. Rather than making a single technique the company-wide standard, applying sharing policy differentially by workload type is advantageous for both cost and stability. In practice, separating the node pools into "MIG isolation pool / time-slicing shared pool / dedicated pool" and routing workloads to the appropriate pool by namespace and label makes operations simpler.
Trade-offs (isolation vs. utilization vs. flexibility): The three goals cannot be maximized simultaneously. MIG buys isolation but pays a flexibility loss in fixed profiles and reconfiguration cost (all jobs must be drained), while time-slicing gains flexibility and utilization but gives up isolation. An organization must quantify "the scale of harm on isolation failure (data leakage, SLA violation)" and "the cost value of 1 percentage point of utilization" and explicitly decide where to compromise. In particular, beware the paradox in which chasing utilization alone and overcommitting excessively surges OOM and tail latency, so that effective throughput actually drops.
Observability and operations (visibility of shared environments): As sharing grows, it becomes hard to find the cause of bottlenecks and interference unless you can see "which job uses how much on which slice." Collect GPU utilization (SM occupancy), VRAM usage, memory bandwidth, and per-instance power with means such as DCGM at the slice level, and in multi-tenancy connect them to per-tenant quotas and billing. A plain "GPU utilization 90%" cannot distinguish whether the SMs are actually working or merely occupying memory, so KPIs for shared environments must be more granular.
Outlook and related technologies: Sharing technology evolves intertwined with Kubernetes DRA, CUDA virtual memory management, KV cache sharing in inference servers, and memory-expansion technologies such as GPU interconnects (NVLink, NVSwitch) and CXL. Going forward, static hardware partitioning and dynamic software overcommit are expected to converge under a declarative interface (DRA), making elastic GPU pools that "automatically partition and reclaim according to the requested capability" commonplace. From a professional engineer's perspective, the key competency becomes designing GPU resource management governance that spans isolation, utilization, billing, and observability, rather than being locked into a particular vendor's technique.

References

NVIDIA, "Multi-Instance GPU (MIG) User Guide — Supported MIG Profiles." https://docs.nvidia.com/datacenter/tesla/mig-user-guide/
NVIDIA, "Multi-Process Service (MPS)." https://docs.nvidia.com/deploy/mps/
NVIDIA, "Virtual GPU Software Documentation." https://docs.nvidia.com/vgpu/
Kubernetes, "Dynamic Resource Allocation." https://kubernetes.io/docs/concepts/scheduling-eviction/dynamic-resource-allocation/
NVIDIA, "DRA Driver for NVIDIA GPUs." https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/dra-intro-install.html

In one line: GPU virtualization is a spectrum of technologies that compromise the conflicting goals of low utilization of an expensive GPU and multi-tenant isolation via temporal partitioning (time-slicing), spatial partitioning (MPS), hardware partitioning (MIG), and VM partitioning (vGPU); it carries the trade-off that the stronger the isolation, the more granularity and flexibility is lost, and it is evolving toward static isolation and dynamic sharing converging declaratively through Kubernetes DRA and CUDA virtual memory.