AI Fairness and Algorithmic Bias Mitigation
1. Overview
A. Definition
AI Fairness is the property by which a machine learning model's predictions and decisions do not unfairly disadvantage particular groups on the basis of legally or ethically protected attributes such as sex, age, race, disability, or region, and Bias Mitigation is the set of activities that identify and statistically/technically correct the causes of such unfairness across the data, training, and output pipeline.
Fairness is a different axis of quality from "a model with high accuracy." A hiring-screening model with 95% average accuracy is still socially unacceptable if its errors concentrate on one sex. In other words, fairness is the problem of exposing the per-group performance gap that the aggregate metric hides; it is the perspective that asks "for whom does the model err?"
Algorithmic bias does not mean only intentional discrimination. Rather, most bias is unintentional bias in which social inequality accumulated in historical data is learned by the model as-is, the sample lacks representativeness, or a proxy variable correlated with a protected attribute causes indirect discrimination. Thus bias mitigation should be approached not as "weeding out bad developers" but as engineering control of structural defects in data and models.
B. Background and Necessity
First, AI decision-making has rapidly expanded into high-stakes domains such as hiring, credit scoring, insurance underwriting, medical diagnosis, criminal justice, and welfare eligibility. In these domains a wrong decision directly affects an individual's livelihood, health, or liberty, so average accuracy alone cannot justify a service — the "distribution of errors" must also be explained.
Second, real harm cases have accumulated. ProPublica's 2016 analysis that the recidivism-risk tool COMPAS assigned excessive false positives to Black defendants, a case in which a major IT firm's hiring AI disclosed in 2018 systematically penalized female applicants, and research (Gender Shades) showing that several facial-recognition systems had markedly higher error rates for women of color all elevated algorithmic discrimination into a social agenda.
Third, regulation is turning fairness into an obligation. The EU AI Act (in force 2024) classifies hiring, credit, education, and similar uses as high-risk AI and requires data quality, bias checking, and documentation. Domestically as well, the Personal Information Protection Commission's rules on the right regarding automated decisions and the "AI Framework Act" (promulgated 2025, expected to take effect 2026) are institutionalizing trust and fairness requirements. The legislative timeline, which may change, is described in general terms, but the professional engineer must be able to translate it into requirements, controls, and evidence.
C. Goals of Securing Fairness
The first goal of fairness activity is group equity — managing the gaps in acceptance rate, error rate, and opportunity between protected and non-protected groups within an acceptable range. The second goal is individual consistency, the principle that similar individuals should receive similar decisions. The third goal is transparency and accountability — leaving, in a documented and verifiable form, which fairness criterion was chosen and why and which trade-off was accepted.
2. Sources of Algorithmic Bias and the Lifecycle
Bias does not arise all at once at a single stage; it seeps in throughout the lifecycle from data collection to operation. Mitigation strategy must therefore be designed not at a single point but as defense in depth. The structure below shows both the points where bias enters and the corresponding control points.
flowchart TB
subgraph SRC["Bias entry paths"]
H["Historical bias (past social inequality)"]
R["Sampling bias (lack of representativeness)"]
L["Label bias (subjective labels)"]
P["Proxy bias (proxy correlation)"]
F["Feedback loop (output re-enters as data)"]
end
H --> DATA["Training data"]
R --> DATA
L --> DATA
P --> DATA
DATA --> MODEL["Model training"]
MODEL --> OUT["Prediction / decision"]
OUT --> USER["Users / society"]
USER -. "retraining data" .-> F
F --> DATA
DATA --> C1["Pre-processing control"]
MODEL --> C2["In-processing control"]
OUT --> C3["Post-processing control"]
A. Historical Bias and Sampling Bias
Historical bias occurs when the data itself accurately reflects reality but that reality is already unequal. For example, if hiring in a given job family was male-centered over the past decade, a model trained on that data wrongly learns "maleness" as a signal of competence. Here the data is "not wrong," but used as-is it reproduces discrimination.
Sampling/representation bias occurs when a particular group's data is scarce or low in quality. In facial recognition, if light-skinned, male images are over-sampled, the error rate surges for dark-skinned, female faces. In the Gender Shades study some commercial systems had error rates below 1% for light-skinned men but exceeding 30% for dark-skinned women — a textbook result of sampling bias.
B. Label Bias and Proxy Bias
Label bias arises when the ground-truth label embeds human subjectivity and prejudice. Labeling a "high-performing employee" by past managers' subjective evaluations makes the prejudice inherent in those evaluations the target variable directly. Proxy bias is the phenomenon in which, even without using a protected attribute directly, a variable strongly correlated with it (residential ZIP code, alma mater, commute distance, etc.) indirectly mediates discrimination. The key reason that "fairness through unawareness" — removing the protected-attribute column — fails is precisely this proxy variable; in practice a single ZIP code can act as a powerful proxy for race and income.
C. Feedback Loop Bias
Feedback loop bias is when, in operation, the model's decisions become data again and amplify bias. In predictive policing, concentrating patrols in a particular area increases that area's arrest records, and the increased records in turn reinforce the prediction of a "crime hotspot," creating a vicious cycle. In recommender systems too, exposure yields clicks and clicks again determine exposure, progressively excluding niche tastes and minority groups. Feedback loops are invisible in static evaluation, so continuous monitoring during operation is essential.
These five bias sources are not independent; they combine and amplify. When minority-group data is scarce due to sampling bias, the model's uncertainty for that group grows, uncertain predictions lead to conservative decisions (rejection), and those decisions re-enter via the feedback loop so that the gap accumulates. Bias diagnosis must therefore be not a single-cause search but a structural analysis that follows data lineage to trace "what entered at which stage."
3. Fairness Metrics and Measurement
Fairness does not fix to a single "definition." Depending on what is regarded as fair, different and sometimes mutually incompatible mathematical criteria exist. The flow below shows the entire fairness evaluation/mitigation pipeline.
flowchart LR
A["Define protected attributes / groups"] --> B["Select reference metric"]
B --> C["Compute per-group metrics"]
C --> D{"Gap within threshold?"}
D -- "No" --> E["Apply mitigation technique"]
E --> F["Retrain / recalibrate"]
F --> C
D -- "Yes" --> G["Document fairness / model card"]
G --> H["Operational monitoring / drift watch"]
H -. "gap recurs" .-> C
A. Group Fairness Metrics
The three most widely used group-fairness metrics are as follows. First, Demographic Parity is the criterion that the positive-prediction rate must be equal between protected and non-protected groups, expressed as P(Ŷ=1│A=a) = P(Ŷ=1│A=b). Second, Equal Opportunity is the criterion that the probability of being predicted positive among actual positives (true positive rate, TPR) must be equal across groups. Third, Equalized Odds is the stricter criterion that both the true positive rate and the false positive rate (FPR) must be equal across groups.
In practice the Disparate Impact Ratio (DIR), derived from the U.S. EEOC's "80% rule," is also widely used. It is the minority group's acceptance rate divided by the majority group's acceptance rate, and a value below 0.8 is typically regarded as a red flag for unfairness. For example, if the male pass rate is 50% and the female pass rate is 30%, the DIR is 0.6 — a clear case of disparate impact.
| Metric | Core condition | Suitable situation | Limitation |
|---|---|---|---|
| Demographic parity | Equal positive-prediction rate | Opportunity allocation (hiring, ads) | May ignore real qualification differences |
| Equal opportunity | Equal true positive rate | Positives that must not be missed (disease screening) | FPR gap uncontrolled |
| Equalized odds | Equal TPR / FPR | Justice, credit where both error directions matter | Very strict, accuracy loss |
| Disparate impact ratio | Acceptance ratio ≥ 0.8 | Legal discrimination screening | Arbitrariness of the threshold |
B. Individual Fairness and Counterfactual Fairness
Whereas group fairness looks at the "group average," Individual Fairness is the principle that "similar individuals should receive similar decisions," with the definition of an appropriate similarity metric being the core challenge. Counterfactual Fairness, meanwhile, uses causal inference to ask "if only this person's protected attribute had changed, would the decision have differed." If the decision changes, that attribute acted unfairly. This can capture even indirect discrimination through proxy variables, making it theoretically powerful, but it carries the practical constraint that building an accurate causal graph is difficult.
C. The Impossibility Theorem of Fairness
Fairness metrics often cannot be satisfied simultaneously. According to the impossibility theorem formalized by Kleinberg, Chouldechova, and others, when the base rate differs between groups, it is generally impossible to satisfy calibration, equal true positive rate, and equal false positive rate all at once. That is, fairness is not a pure technical problem but one that involves a value judgment of "which fairness to prioritize." The professional engineer must not hide this trade-off but make it explicit so that stakeholders can choose.
4. Bias Mitigation Techniques
Bias mitigation divides into three categories by point of application: pre-processing, in-processing, and post-processing. None is always superior; they are combined according to data accessibility, model modifiability, and regulatory requirements.
A. Pre-processing Techniques
Pre-processing corrects the data itself, with the advantage of being model-independent and transparent. As representative techniques, reweighing assigns larger weights to under-represented group samples to balance them. Resampling oversamples the minority group (e.g., SMOTE) or undersamples the majority. Learning Fair Representations learns a latent representation that preserves task information while removing protected-attribute information. For instance, if female approval cases are scarce in loan data, reweighing amplifies their influence so the model learns that pattern adequately.
The limitation of pre-processing is that one cannot fully control how the downstream model uses the data. Also, if proxy variables are intricately entangled, simple reweighing alone cannot remove indirect discrimination.
B. In-processing Techniques
In-processing integrates the fairness constraint directly into the model's objective function. Fairness regularization adds a per-group gap penalty term to the loss function to optimize accuracy and fairness together. Adversarial debiasing places, separately from the predictor, an "adversary that tries to guess the protected attribute from the output," and trains the predictor so the adversary fails, thereby removing sensitive information from the output. Constrained optimization imposes a fairness condition such as equalized odds as a constraint and maximizes accuracy.
In-processing enables the most fundamental correction but requires access to the model structure, and convergence stability and computational cost can grow. Its application to external pre-trained models (API-based LLMs, etc.) is limited. The intuition of adversarial debiasing is this: if one can guess an applicant's sex with high accuracy from the model's output (or intermediate representation) alone, that means sex information is still leaking into the output. Training the predictor so that the adversary's sex-estimation accuracy converges to chance level (about 50% for a binary attribute) removes the trace of sensitive information from the output while preserving the original task performance.
C. Post-processing Techniques
Post-processing adjusts the output or decision threshold of an already-trained model. Threshold adjustment applies a different decision threshold per group to equalize the true positive rate. Calibration post-processing corrects the per-group score distribution. Post-processing has the great advantage that it does not retrain the model, so it applies even to black-box models and is low-cost. However, using different thresholds per group can spark "reverse discrimination" controversy or legal problems, and it leaves the root cause untouched.
| Category | Point of application | Advantage | Limitation | Representative techniques |
|---|---|---|---|---|
| Pre-processing | Before training | Model-independent, transparent | Weak downstream control | Reweighing, resampling, fair representation |
| In-processing | During training | Fundamental correction | Needs model access, higher cost | Adversarial debiasing, fairness regularization |
| Post-processing | At inference | Black-box applicable, low cost | Reverse discrimination, root cause unresolved | Threshold adjustment, calibration |
As a practical case, when a bank measured its loan model's DIR at 0.65, an approach that balances the data with pre-processing reweighing, adds an in-processing fairness constraint, and then applies per-group calibration in operation — a three-stage combination — raising the DIR above 0.85 while holding the overall accuracy loss to within 2 percentage points is typical. The key is not a single technique but a combination across the whole lifecycle.
5. In Depth: Bias and Standards Trends in the Generative AI Era
The fairness discussion for traditional classification and regression models enters a new phase with large language models (LLMs) and generative AI. Because LLMs are trained on web-scale unrefined corpora, they absorb social stereotypes about sex, race, religion, and occupation as-is. Representative is representational harm — associating "nurse" with a woman and "CEO" with a man, or generating harmful/demeaning expressions toward specific groups. This is hard to reduce to numeric metrics and requires combining red-teaming, bias benchmarks (BBQ, StereoSet, etc.), and human evaluation.
Mitigation means change too. Pre-training data filtering, alignment via reinforcement learning from human feedback (RLHF) and Constitutional AI, output control through system prompts and guardrails, and limiting the basis via retrieval augmentation (RAG) are combined. However, excessive filtering can cause over-refusal on certain topics or yet another bias, so the balance between fairness and usefulness again becomes a challenge.
On the standards/regulation side, the NIST AI RMF (2023) presents fairness and bias management as one of the trustworthiness characteristics and, in a separate publication, provides a taxonomy of systemic, statistical, and human biases. ISO/IEC TR 24027 addresses bias evaluation of AI systems, and the AI management system standard ISO/IEC 42001 (2023) includes bias control as a management-system requirement. The EU AI Act legally mandates, for high-risk AI, the adequacy of training, validation, and test datasets and bias checking. Thus fairness is moving from "recommendation" to "auditable control."
6. Considerations and Implications
First, the choice of fairness criterion is a matter of business, legal, and ethical consensus. The professional engineer must not let developers arbitrarily decide among demographic parity, equal opportunity, and equalized odds; the decision-making structure should be designed so that stakeholders choose based on domain risk (minimizing false negatives for disease screening, equal opportunity for ad allocation, etc.) and regulatory requirements. Clearly conveying that, due to the impossibility theorem, "achieving all fairness simultaneously" is impossible is also the engineer's duty.
Second, the accuracy–fairness trade-off must be managed quantitatively. It is normal for a fairness constraint to lower overall accuracy somewhat; what matters is documenting whether that loss is within a reasonable range and whose benefit the loss serves. An effective approach is to present an accuracy–fairness Pareto curve so decision-makers see the trade-off with their own eyes and choose.
Third, fairness is an object of continuous monitoring during operation, not a one-off check. Due to data drift and feedback loops, a model that was fair at deployment can become unfair over time. Therefore one must periodically compute per-group performance and DIR and build an MLOps pipeline (including model cards and a fairness dashboard) that retrains/recalibrates on threshold breach.
Fourth, linkage with governance and accountability is needed. Fairness activity gains effectiveness when combined with data lineage, model cards, impact assessment (DPIA / AI impact assessment), appeal procedures, and human-in-the-loop oversight. In particular, aligned with the trend of institutionalizing the right to explanation and objection regarding automated decisions, fairness-check results should be left as audit evidence and designed to allow external verification.
Fifth, do not overlook proxy variables and intersectionality. Since removing protected attributes alone cannot prevent indirect discrimination, proxy correlations must be analyzed actively, and the evaluation design must reflect that bias can be amplified at the intersection of attributes — such as "dark skin × female" — rather than at a single attribute.
References
- NIST, "Towards a Standard for Identifying and Managing Bias in Artificial Intelligence (SP 1270)": https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.1270.pdf
- NIST, "AI Risk Management Framework (AI RMF 1.0)": https://www.nist.gov/itl/ai-risk-management-framework
- ISO/IEC 42001:2023 (AI management system): https://www.iso.org/standard/81230.html
- EU Artificial Intelligence Act: https://artificialintelligenceact.eu/
- IBM AI Fairness 360 (AIF360): https://aif360.res.ibm.com/
In one line: AI fairness is a quality axis that controls the performance gap between protected groups; one must identify historical, sampling, label, proxy, and feedback-loop bias across the whole lifecycle, choose a domain-appropriate criterion among possibly mutually incompatible fairness metrics, combine pre-processing, in-processing, and post-processing techniques, and back it with continuous operational monitoring and governance.