Active Learning

1. Overview

A. Definition

Active Learning is a paradigm of semi-supervised learning in which the learner itself selects the "most informative" samples from a vast pool of unlabeled data, queries an expert (an oracle, usually a human labeler) for their labels, and iteratively improves the model from the responses. Its core premise is that "labels are expensive and scarce," and its goal is data efficiency: reaching a target performance with the minimum labeling cost.

Traditional supervised learning (passive learning) has labelers annotate data that was randomly sampled or collected exhaustively, then trains once on all of it. Active learning, by contrast, weaves training and labeling into a single closed loop rather than treating them as separate preprocessing stages. Because the model decides for itself "which sample am I most confused about right now, or which one, if known, would shift the decision boundary the most?" and queries only those, the number of labels needed to reach the same performance can be several to dozens of times smaller than with random collection. In other words, the essence of active learning is to redefine the value of learning in terms of "what you look at" rather than "how much you look at."

B. Background and necessity

The bottleneck of modern machine learning is shifting from algorithms or computation to high-quality labels. Model architectures have been leveled upward by pretrained models and open source, but domain labels must still be attached by hand, and that cost dominates. Segmenting a lesion in a medical image requires a specialist to mark it pixel by pixel, and boxing the object boundaries in a 3D lidar point cloud for autonomous driving takes tens of minutes per frame. Exhaustive labeling is unrealistic in cost, time, and expert availability, and here the idea "if we cannot label everything anyway, let us label the most valuable samples first" is the starting point of active learning.

The second axis amplifying the need is extreme data skew. In domains where the target of interest is rare—fraudulent transactions, rare diseases, semiconductor process defects—it is hard even to encounter the minority class through random sampling. Active learning intensively queries ambiguous samples near the decision boundary, i.e., regions where the minority class is likely to cluster, so it allocates the label budget efficiently under [[class-imbalance]]. The third axis is preference-data collection in the era of foundation models and LLMs. Because human comparative evaluation in RLHF or preference alignment is very expensive, the strategy of picking the prompt–response pairs the model is most uncertain about and asking a human is itself an application of active learning. This topic is closely linked to [[class-imbalance]], [[semi-supervised-learning]], [[data-labeling]], and [[rlhf]].

C. Core properties that characterize active learning

For active learning to work, several premises and design variables must line up. If these elements are missing, active learning easily becomes over-engineering that only adds complexity without benefit, so the items below should be diagnosed first before adoption.

• Asymmetry of labeling cost: it is most effective when data collection is cheap but labeling is expensive (a large unlabeled pool exists). If labels are free, the incentive for active learning disappears.
• Query strategy: by what criterion is "informativeness" defined? The choice among uncertainty, committee disagreement, expected model change, diversity, and so on determines performance.
• Query scenario: whether you pick from the whole pool (pool-based), judge on the fly from a flowing stream (stream-based), or synthesize inputs to ask (membership query synthesis).
• Batch budget: whether you query one item at a time or query hundreds in a bundle. Deep learning makes batch queries essential because of retraining cost.
• Oracle quality: humans do not always give the right answer (noise, subjectivity). A design that assumes oracle error is necessary.

2. Operating structure and closed loop of active learning

Active learning repeats, until a stopping condition is met, the cycle of building a model from a small initial set of labels → scoring the unlabeled pool → picking the most informative samples → querying a human → adding the labels to the training set and retraining. This closed loop is the backbone of active learning, and the structure diagram below summarizes the whole flow.

flowchart TB
  U["Unlabeled data pool<br/>(large, low-cost)"] --> S["Query strategy<br/>(informativeness scoring)"]
  L0["Small initial label set"] --> M["Train current model"]
  M --> S
  S --> Q["Select most informative samples<br/>(top k)"]
  Q --> O["Query the oracle (human labeler)"]
  O --> A["Acquire labels, add to training set"]
  A --> M
  M --> C{"Stopping condition?<br/>(budget, performance saturation)"}
  C -->|"No"| S
  C -->|"Yes"| R["Deploy final model"]
  style R fill:#e8f0fe,stroke:#2f6fed,stroke-width:2px
  style S fill:#fff4e5,stroke:#e8a33d,stroke-width:2px

The most important design decision in this loop is the query scenario. The three scenarios diverge according to "in what form you encounter the unlabeled data."

• Pool-based sampling: scores the entire fixed, large unlabeled pool every round and picks the top samples. Most real-world work, where unlabeled data is already gathered (medical-image archives, collected driving logs), falls here, and it is the most widely used.
• Stream-based selective sampling: as samples flow in one at a time, it judges on the spot "is it worth asking about this now?" and discards or queries it. Used when the pool cannot be stored wholesale, as in sensor streams or edge environments.
• Membership query synthesis: the learner generates inputs to ask about, but the generated inputs can take a form humans cannot interpret, so practical application is limited.

The three scenarios are chosen in interplay with storage and compute constraints. Pool-based must infer over the entire pool every round, so if the pool is hundreds of millions of items the scoring itself becomes the bottleneck and approximation or candidate sampling is needed, whereas stream-based cannot revisit past data but imposes a small memory burden. Most enterprise environments already accumulate logs and images in a data lake, so pool-based is the standard, but places needing immediate judgment, such as a real-time quality-inspection line, suit stream-based.

The second key is the batch-mode design. Because retraining deep learning on every single added item is excessively costly, each round queries hundreds to thousands of items in a bundle. But simply taking "the top k by uncertainty" causes almost identical samples to be selected in bulk. For example, if 500 similar blurry photos are all ambiguous, labeling only a few of them provides enough information. Therefore batch queries must remove redundancy by considering both informativeness and diversity among samples. This is why mixed strategies such as BADGE and Core-set, described later, emerged.

The batch design also hides a trade-off between the number of rounds and the batch size. Splitting into small batches and running many rounds lets you query more accurately with the latest model each round, but retraining and labeling round-trip costs rise; taking large batches is operationally convenient, but the model is not updated within a round, increasing the risk of redundantly selecting similar samples. In practice the batch size is set by considering both the labeling organization's processing cycle (e.g., one batch per day) and the retraining cost, with rules of thumb such as taking 5–10% of the total label budget as one batch.

The third design axis is the stopping criterion. Because active learning cannot run indefinitely, the stopping criteria are label-budget exhaustion, saturation (a plateau) of validation performance, or the point where the marginal performance gain from one batch falls below a threshold. If no stopping condition is specified, overinvestment occurs as the label budget keeps pouring into a region of almost no utility, so a system that monitors the slope of the performance-versus-label curve must be put in place at the operational-design stage.

3. Types and principles of query strategies

Query strategies divide broadly, according to "how informativeness is defined," into uncertainty-based, committee-based, expected-impact-based, and representativeness/diversity-based. None is a cure-all; the standard is to combine them to fit the data characteristics and model type.

A. Uncertainty Sampling

The most intuitive and widely used strategy queries the samples the model is least confident about. The intuition is simple: a sample the model is already confident about offers little to learn even when labeled, but a sample sitting right on the decision boundary and confusing it 50:50 has a single label that moves the boundary greatly. Three representative ways quantify uncertainty. Least confidence picks the sample with the lowest top-class probability, margin sampling picks the sample with the smallest gap between the first- and second-ranked class probabilities, and entropy picks the sample with the largest entropy of the whole class distribution. In binary classification the three are effectively the same, but in multiclass, entropy better reflects the ambiguity of the whole distribution and margin better reflects the contest between the top two candidates.

Let us see the difference with concrete numbers. In a three-class problem, suppose sample X has predicted probabilities (0.50, 0.49, 0.01) and sample Y has (0.40, 0.30, 0.30). Least confidence slightly prefers Y since both top probabilities are similar (0.50 vs 0.40), while margin, because X's first–second gap (0.01) is smaller than Y's (0.10), considers X more ambiguous. Entropy, on the other hand, considers Y more ambiguous since its probability is spread more evenly across the three classes. That is, "are the two candidates tight at the boundary (margin)?" and "is the distribution scattered overall (entropy)?" are different questions, and if the detection goal is "resolving confusion pairs" margin suits it, while if it is "reducing overall ambiguity" entropy suits it. The subtle practical point is that even for the same "uncertainty," the choice of formula changes the query result.

The weakness of uncertainty sampling is that it is vulnerable to noise and outliers. Extreme values with label errors or out of distribution keep confusing the model, so they are repeatedly selected, yet labeling them does not help generalization. Also, when the model's probability estimates are overconfident (especially in deep learning), the uncertainty itself is distorted. Thus in deep learning, methods that calibrate predictive uncertainty with Monte Carlo dropout (MC Dropout) or ensembles and then score (e.g., BALD, Bayesian Active Learning by Disagreement) are used. More fundamentally, one must distinguish the epistemic uncertainty of the model "mistakenly thinking it does not know what it does know" from the aleatoric uncertainty of the data itself; since active learning can only reduce the former (resolvable with more data), a calibration device that filters out samples with large aleatoric noise determines the strategy's completeness.

B. Query-by-Committee (QBC)

This strategy forms a committee of several different hypotheses (models) and queries the samples on which the committee members' predictions diverge the most. The principle is explained by version-space reduction. On a sample where the members agree, the conclusion is the same whichever model you pick, so there is nothing to learn, but on a sample where the members split, knowing its label greatly shrinks the "set of hypotheses that can survive." The degree of disagreement is measured by vote entropy or the KL divergence between each member's distribution and the average predictive distribution. The committee is formed by bagging/boosting ([[ensemble-learning]]) or by different initializations and hyperparameters, and in deep learning ensemble and dropout samples effectively play the committee's role.

Query-by-committee is historically the strategy that laid the theoretical foundation of active learning; under the ideal condition where the version space halves with each query, one can show that the number of queries needed decreases on a logarithmic scale. Real models do not fully satisfy this ideal, but the intuition that "disagreement is information" still holds, carried today into deep ensembles and Bayesian approximations.

QBC's practical advantage lies in being less swayed by a particular model's bias. A single model's uncertainty inherits that model's inductive bias as-is, but a heterogeneous committee exposes one another's blind spots as if by cross-validation. However, the cost of training and maintaining a committee every round is added, and if the members are too similar (highly correlated) the disagreement signal weakens and effectively degenerates into uncertainty sampling. Therefore securing diversity among members is the crux of QBC's success, and mixing different feature subsets (feature bagging) or models of different structures is recommended.

C. Expected-impact-based (Expected Model Change, Error Reduction)

This strategy directly estimates "how greatly the model changes if this sample is labeled." Expected Gradient Length (EGL) picks the one with the largest expected magnitude of the gradient that would occur when the sample is added. Going a step further, Expected Error Reduction picks the sample that, when added with a hypothetical label, reduces the most the expected error (or uncertainty) over the entire remaining unlabeled set. It is theoretically the most principled, but because it assumes retraining over every candidate × every possible label, its computational cost is enormous, making it hard to use for large-scale or deep learning without approximation.

The insight this family gives is that "being uncertain does not necessarily mean useful." Even if the model is confused, if the sample's actual impact on the decision boundary is slight, or the region is already sufficiently learned from similar samples, the marginal value of the label is low. Expected-impact-based strategies are the most faithful to the goal in that they aim directly at the "expected effect of learning" rather than "ambiguity." In practice, variants that use only the last-layer gradient instead of full retraining, or that lower cost with influence functions or Fisher-information approximations, are employed.

D. Representativeness and diversity-based

Whereas the preceding strategies focus on "boundary ambiguity," this family makes the selected samples represent and cover the whole data distribution. Chasing only uncertainty produces sampling bias where samples cluster in a particular region, so one picks evenly by dividing the distribution with clustering, or optimizes, as in the Core-set technique, so that "the selected samples cover the entire unlabeled set within a radius r." One of the most successful mixed strategies in practice, BADGE, applies k-means++ to the gradient embeddings of the model's last layer to achieve informativeness (gradient magnitude) and diversity (mutually distant samples) at once.

Another strength of representativeness-based strategies is that they ride directly on the quality of the representation space. If there are well-aligned embeddings from self-supervised pretraining, distance and density in that space directly mean "how much of the distribution is covered," so clustering and core-set strategies work far better. Conversely, in the early cold-start phase where the representation is poor, the uncertainty signal cannot be trusted, so many practical pipelines adopt a two-phase design that switches from embedding-based diversity in phase 1 to uncertainty in phase 2. The table below compares the characteristics of each strategy. Still, the table is only a summary; the basis for the choice should be found in the principles in the prose above.

Strategy	Definition of informativeness	Compute cost	Strengths	Weaknesses
Uncertainty sampling	Proximity to boundary (confidence, margin, entropy)	Low	Simple, fast, easy to implement	Vulnerable to noise, overconfidence, bias
Query-by-committee (QBC)	Disagreement among members	Medium	Model-agnostic, theoretical grounding	Cost of forming and maintaining a committee
Expected impact (EGL, EER)	Expected model change, error reduction	Very high	Principled, greedily optimal	Impractical for large-scale or deep learning
Representativeness, diversity	Distribution coverage, redundancy removal	Medium	Strong at batch and bias mitigation	Weak at reflecting boundary ambiguity

4. Comparison with passive learning and similar paradigms

To understand the value of active learning, what you compare it with matters. The comparison below focuses not on a mere list but on why the differences arise.

flowchart LR
  subgraph PASSIVE["Passive learning"]
    P1["Random/exhaustive collection"] --> P2["Batch labeling"] --> P3["Train once"]
  end
  subgraph ACTIVE["Active learning"]
    A1["Small initial labels"] --> A2["Informativeness query"] --> A3["Selective labeling"] --> A4["Iterative retraining"]
    A4 --> A2
  end
  style ACTIVE fill:#e8f0fe,stroke:#2f6fed

Passive learning randomly draws data and labels all of it. It has the advantage of simple implementation and no distribution bias, but it spends the same cost even on easy samples the model already knows well, so label efficiency is low. In typical benchmarks it is often reported that active learning reaches the same accuracy with 30–60% fewer labels than random. That said, one must be clear that "if data is already sufficient and labels are cheap," passive learning is safer and faster. Active learning pays off against the investment only when labeling cost dominates.

Let us gauge the benefit with simple numbers. In a problem where random collection needs 100,000 items to reach the target accuracy, if active learning yields a 40% reduction it reaches the same performance with 60,000 items. At a per-item label price of 1,000 won, that is a direct saving of 40 million won, and in domains where the per-item price is tens of thousands of won, as in specialist reading, the scale of saving is far larger. Of course, net profit comes only after subtracting the added costs of query-strategy computation, iterative retraining, and loop operation, so in problems with low label prices and abundant data this balance can reverse. The adoption decision must rest on this break-even analysis.

Semi-supervised learning uses unlabeled data differently. Semi-supervised learning has the model attach pseudo-labels by itself to unlabeled data and use them, whereas active learning asks a human for real labels. The two do not conflict but are complementary, so combining active and semi-supervised—self-labeling confident samples semi-supervisedly and asking a human only for ambiguous ones—maximizes data efficiency. Self-supervised learning pretrains a representation without labels and is a complement that eases the cold-start problem by providing active learning with a "good initial feature space." The difference from reinforcement learning is the nature of the feedback: reinforcement learning learns a "policy" from reward signals, while active learning picks "data" via label queries.

Meanwhile active learning has two structural pitfalls. The first is the cold start: if the initial model is too weak, the very judgment of "what is informative" is wrong and only odd samples get queried. So it is recommended to cover the distribution broadly with a diversity-based approach in early rounds and switch to uncertainty-based once the model is somewhat settled, a hybrid schedule. The second is sampling bias: if the training set gathered by a particular strategy distorts the real distribution, that model's evaluation may be over- or under-estimated. Therefore the evaluation set must always use a fixed test set secured randomly, independent of the active-learning loop.

A third, often-overlooked pitfall is model dependency. Data gathered by active learning is a set of "samples that particular model was ambiguous about at that time," so if you later swap in a model of an entirely different structure, that data may not be optimal. In other words, a dataset curated by active learning is implicitly coupled with the model at collection time and has lower reusability. To mitigate this, keep the collecting model as general and strong a representation as possible (self-supervised-pretraining-based), or keep part of the strategy model-agnostic and diversity-based to preserve the data's general value.

5. Advanced: active learning in the era of deep learning and foundation models

Traditional active-learning theory developed on the premise of shallow models like SVM and logistic regression, but deep learning poses three new challenges. First, deep learning makes per-item retraining unrealistic, so batch queries are essential and the diversity consideration seen earlier becomes central. Second, deep learning's softmax probabilities tend to be overconfident, distorting uncertainty estimation, so techniques that separate epistemic uncertainty via Bayesian approximation, like MC Dropout, deep ensembles, and BALD, have developed. Third, training a huge model from scratch every round with a small number of labels is unstable, so the combination of first securing a representation with self-supervised pretraining and then picking only fine-tuning samples with active learning has become standard.

A recent research trend also draws attention to active fine-tuning. When adapting a huge pretrained model to a specific domain, instead of labeling all data, fine-tuning only the few samples the model is most uncertain about during domain transfer yields a large performance gain even with few labels. This is especially powerful when combined with few-shot and efficient fine-tuning ([[lora]]) techniques that adapt with only a few samples, and it becomes a central axis of the modern adaptation strategy of "a huge model + a small, well-chosen set of domain labels."

The biggest change is the advent of foundation models and LLMs. Now, instead of humans, "AI-assisted labeling," in which a powerful model proposes label candidates and the human only reviews, has become reality. In this setup active learning is reinterpreted as a human-in-the-loop allocation policy of "auto-approve what the model is confident about, let a human review only the ambiguous." Also, because human comparative evaluation is very expensive in RLHF and preference alignment, active preference learning, which queries the response pairs the reward model is most uncertain about, draws attention as a key technique for reducing alignment-data cost.

Going a step further, active learning converges with the weak supervision and data-centric AI currents. One cheaply creates a large volume of noisy labels with labeling functions or heuristics, then has humans refine only the samples the model is most uncertain about or where the labeling functions conflict. This meets the data-centric view of "systematically improving the data rather than changing the model," and active learning serves as the engine that sorts the improvement targets in order of maximum value.

Industrial applications are also abundant. The autonomous-driving data engine is representative: the closed loop of selecting, from vast driving logs collected on the road, only the "edge-case" scenes the model failed on or was confused by, labeling them, and retraining, is itself large-scale active learning. In medical imaging, specialist reading cost is extremely high, and many studies report cutting the reading burden several-fold by querying only slices with high uncertainty. In manufacturing defect detection, intensively querying a few defect samples overcomes extreme imbalance. In financial fraud detection, sending only ambiguous transactions to investigators allocates investigation resources efficiently. What these cases share is that "label and review resources are finite and expensive," and active learning functions as the operating principle that allocates those resources to the most valuable samples.

That said, one must not conclude that deep active learning always beats random. Some benchmark studies have reported that, under conditions with strong data augmentation and self-supervised pretraining, active learning's benefit is not much different from random sampling, or even lags behind depending on the strategy. That is, active learning's benefit is sensitive to the combination of data, model, augmentation, and budget, and prudence is needed to first verify with an independent benchmark whether "it actually benefits our problem." Because of this uncertainty, the importance of a reproducible evaluation protocol (same initial set, same test set, averaged over multiple seeds) is emphasized.

6. Considerations and implications

Active learning is, before being an algorithmic technique, an operating system in which the labeling organization, budget, quality, and governance interlock. From the professional-engineer viewpoint, one must weigh not only model accuracy but also benefit against label investment, bias control, human–model collaboration design, and sustainability of continuous operation. The key issues to check at adoption are as follows.

• Feasibility and ROI judgment: active learning is not free. Query-strategy computation, iterative retraining, labeling infrastructure, and loop operation incur engineering cost. Net profit comes only when labeling cost is the dominant share of the total and the unlabeled pool is large enough. If labels are cheap or data is already sufficient, passive learning is more reasonable, and adoption should be decided after quantitatively weighing this trade-off.
• Sampling bias and evaluation reliability: a training set gathered by active learning is by design skewed in distribution, so "evaluating" the model on this data distorts performance. Always validate with a random fixed test set independent of the loop, and in the cold-start phase use diversity-based strategies to secure distribution coverage and prevent bias accumulation.
• Oracle quality and human-in-the-loop design: human labelers make errors from fatigue, subjectivity, and variation in expertise. One must design redundant labeling and consensus, labeler-reliability estimation, and an escalation path to experts for ambiguous samples, and the query UX (how many at a time, what context to present) governs label quality and throughput. In the era of AI-assisted labeling, where to draw the boundary between auto-approval and human review is the key lever of quality and cost.
• Stopping condition and operational governance: when to stop the loop (performance saturation, budget exhaustion, drop in marginal informativeness) must be defined in advance to prevent overinvestment. Also, to respond to post-deployment distribution drift, the active-learning loop should be extended into a continual-operation system, and leaving a record of which samples were queried and why secures data lineage and audit traceability ([[data-lineage]]), making it defensible from the regulatory and quality standpoint.
• Situational fit of strategy choice: rather than relying on a single strategy, a design with staged transitions—diversity/representativeness early, uncertainty after stabilization, and a BADGE-type mixed strategy for batch queries—is the most robust in practice. Because a strategy's effect varies with the model and data characteristics, it is advisable to first measure the label-efficiency curve per strategy with a small-scale pilot.
• Data reusability and asset value: because data curated by active learning may be coupled to the model at collection time and have lower reusability, one should mix in some model-agnostic diversity strategy and leave the collection history and rationale as metadata to manage it as a reusable data asset of the organization. This is a strategic consideration that connects to MLOps and data governance in the long run.

References

• Burr Settles, "Active Learning Literature Survey," University of Wisconsin-Madison, CS Technical Report 1648. https://minds.wisconsin.edu/handle/1793/60660
• Ash et al., "Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds (BADGE)," ICLR 2020. https://arxiv.org/abs/1906.03671
• Sener & Savarese, "Active Learning for Convolutional Neural Networks: A Core-Set Approach," ICLR 2018. https://arxiv.org/abs/1708.00489
• Gal et al., "Deep Bayesian Active Learning with Image Data (BALD)," ICML 2017. https://arxiv.org/abs/1703.02910

---

In one line: Active learning is a closed loop in which the learner itself picks the most informative unlabeled samples to query a human and retrain, combining uncertainty, committee, expected-impact, and diversity strategies as the situation demands—a data-efficiency-centered semi-supervised paradigm that aims to reach target performance at minimum labeling cost.