IM Engineer Study Notes
← Back to list
AI & Data
#AI윤리#AI거버넌스#XAI#공정성#131회
Last updated · 2026-09-26

AI Ethics and Governance Model

1. Overview

A. Definition

The ethics principles to be observed in the process of developing and applying AI, and the governance model for effectively managing and regulating AI. Its purpose is the realization of Trustworthy AI.

The reason AI ethics has become so important is that AI's decisions no longer stay in trivial areas such as recommendation ads but have entered areas that determine people's lives, such as hiring, lending, healthcare, and criminal justice. Past the era in which good performance was enough, "whether this AI discriminates against a particular group (fairness), whether it can explain why it made such a decision (transparency), and who takes responsibility when it goes wrong (accountability)" have become the preconditions for adopting AI.

Here, if ethics principles prescribe 'what must be observed,' the governance model provides the execution system of 'how organizations and society enforce and manage those principles.' With only principles and no governance, it stops at a declaration (ethics washing); with only governance and no principles, it loses direction. The two are therefore not separate concepts but must be understood as a front-and-back relationship of "values (principles)" and "the management structure that continuously realizes those values (governance)."

B. Background and Necessity

As generative AI became popular and anyone could use powerful AI, problems of bias, misinformation (hallucination), copyright, and privacy surfaced in society. Models once handled only by a few experts are now placed in the hands of hundreds of millions, and the ripple effect of misuse and abuse has grown accordingly. As incidents such as fraud using deepfakes, disputes over copyright infringement in training data, and unauthorized learning from personal information followed one after another, the normative gap of "technology races ahead while norms cannot keep up" became a social problem.

Accordingly, as regulations such as the EU AI Act (in force in 2024) and Korea's AI Basic Act were strengthened, the proactive management of AI issues came to be directly tied to corporate survival. This is because a violation of regulations may incur enormous fines (the EU AI Act, up to 7% of global turnover), and an AI service that has lost trust is shunned by the market. In other words, AI ethics and governance are now not "a choice of good companies" but "a prerequisite for a sustainable business."

2. Major AI Ethics Principles (Overall Structure)

The core principles of AI ethics do not exist independently but are intertwined to form a single trust system. The structure diagram below shows the relationships among the principles that support Trustworthy AI.

flowchart TB
  E["AI Ethics Principles(Trustworthy AI)"] --> F["Fairness/Non-discrimination"]
  E --> T["Transparency/Explainability(XAI)"]
  E --> A["Accountability/Safety"]
  E --> P["Privacy/Human-centeredness"]
  T -. provides verification basis .-> F
  A -. attributes responsibility .-> T
  style E fill:#e8f0fe,stroke:#2f6fed,stroke-width:2px

Fairness is preventing discrimination by removing bias from data and algorithms. Bias is usually the result of social prejudice that already existed in the training data being reflected and amplified in the model, so it must be inspected from the data stage. In fact, the AI hiring tool that Amazon scrapped in 2018 learned from past male-dominated hiring data and evaluated female applicants unfavorably — a representative case showing that "data is value."

Transparency and explainability (Transparency·XAI) is presenting the basis for why the AI made such a judgment, and it becomes the precondition for verifying fairness. This is because if the basis of a judgment cannot be known (a black box), even whether discrimination occurred cannot be confirmed. Explainable techniques such as LIME and SHAP support this.

Accountability is clarifying the subject responsible for the result. So that the excuse "the algorithm did it" does not work when an AI's decision goes wrong, it prescribes in advance who among developer, operator, and deployer bears what responsibility. Safety and robustness is preventing malfunctions and adversarial attacks, including designing the model to withstand threats such as attacks that subtly manipulate the input to deceive the model or prompt injection. Privacy and human-centeredness is protecting personal information and guaranteeing that a human intervenes in the final judgment (Human-in-the-loop), evolving toward parallelizing data protection and learning with technologies such as differential privacy and federated learning.

These principles also conflict. For example, using a simple model to raise explainability lowers accuracy (transparency vs. performance), and strengthening personal-information protection reduces training data, making fairness verification harder (privacy vs. fairness). Therefore, in practice, rather than mechanically maximizing all principles, finding a trade-off between principles according to the service's risk level and context becomes the substantive task of governance.

Principle Content
Fairness Remove data/algorithm bias, prevent discrimination
Transparency/Explainability Disclose the basis of judgment (XAI), comprehensibility
Accountability Clarify the subject responsible for results
Safety/Robustness Prevent malfunctions and adversarial attacks
Privacy/Human-centeredness Protect personal information, maintain human oversight

3. AI Governance Model (Architecture)

The governance model is a layered structure that moves ethics principles into actual organizational operation. Starting from the top-level value (principles), it passes through organization, process, and technology to lead to regulatory compliance, forming a closed loop in which operational results are fed back into the principles.

flowchart LR
  P["Ethics Principles"] --> O["Organization/Committee(CAIO)"]
  O --> PR["Impact Assessment/Risk Management"]
  PR --> M["Monitoring/Audit"]
  M --> R["Regulatory Compliance"]
  M -. improvement feedback .-> P
  style PR fill:#e8f0fe,stroke:#2f6fed

At the top are principles and policies (AI ethics standards, internal guidelines), and an organization and system (AI ethics committee, Chief AI Officer CAIO) is formed to execute them. Without an organization, principles remain merely documents, so establishing an executing entity with responsibility and authority is the starting point of governance.

Below that, processes (AI impact assessment, risk classification/management, audit) turn. In particular, AI impact assessment is a procedure that diagnoses in advance the risks an AI will pose to individuals and society before a service is launched, corresponding to the AI version of a privacy impact assessment (PIA). Technology and operations (XAI, MLOps monitoring, bias/drift surveillance) support this, continuously monitoring model performance degradation (model drift) and new bias even after deployment. Only when the whole is aligned with regulatory response (EU AI Act, NIST AI RMF, ISO/IEC 42001) do the organization's own standards and external regulatory requirements operate as a single system without conflict.

Layer Composition
Principles/Policy AI ethics standards, internal policy
Organization/System AI ethics committee, responsible officer (CAIO)
Process AI impact assessment, risk classification/management, audit
Technology/Operations XAI, MLOps monitoring, bias surveillance
Regulatory Response EU AI Act (risk-based), NIST AI RMF, ISO 42001

For this layered structure to actually work, each layer must be organically connected. If only the principles are excellent but there is no organization, they are not executed; even with an organization, without an impact-assessment process risks are not filtered out in advance; and even with a process, without post-deployment monitoring the bias that arises over time is missed. For example, the UK's 2020 university-admissions grade-prediction algorithm affair — in which the algorithm reflected schools' past grades and gave unfavorable scores to students in low-income areas, yet with poor prior impact assessment and human review, mass protests led to the policy being withdrawn — is a case in point. It shows that the whole can collapse if even a single layer of governance is missing.

Therefore, mature governance establishes these layers not as an organizational chart on paper but as an operating cycle that actually turns. Only when regular AI ethics committee deliberations, mandatory impact assessments for new services, and constant monitoring of deployed models with feedback of the results into policy settle into a single routine does governance gain effectiveness.

4. Comparison of Domestic and International Regulations/Standards

To understand AI governance, one must know the differences in the approaches of representative regulations and standards. They take different directions of "legal enforcement (hard law)" and "voluntary framework (soft law)," and the difference stems from the purpose of each system.

Category Nature Core Approach
EU AI Act Legal regulation (mandatory) Risk-based 4-tier classification, strong obligations for high-risk AI
NIST AI RMF Voluntary framework Risk-management guidance of the 4 functions Govern-Map-Measure-Manage
ISO/IEC 42001 Certification standard Build/certify an AI Management System (AIMS)
Korea's AI Basic Act Legal regulation Regulate high-impact AI, balance promotion and regulation

The reason the EU AI Act is a strong regulation that imposes fines upon violation is that the EU places the protection of fundamental rights as its top value and regulates the very conditions of market entry. In contrast, the reason NIST AI RMF is a voluntary framework with no enforcement is that the US, worried about hindering innovation, prefers a flexible risk-management guide. ISO 42001 standardizes this into the form of a management system that organizations can be certified in, providing a means for companies to prove their reliability externally. In this way, even the same "AI governance" varies in strength and form depending on the regulatory philosophy.

5. Deep Dive: Governance Issues in the Generative AI Era and the EU AI Act Risk Classification

The spread of generative AI has given rise to new issues hard to address with existing governance. First, the generality problem. Since a single foundation model is used for countless purposes such as translation, coding, and consultation, it is hard to specify "risk by purpose" in advance. The EU AI Act's separate provision for general-purpose AI (GPAI) is for this reason.

Second, transparency obligations are being strengthened. The trend requires putting watermarks and marks on content created by generative AI (deepfakes, AI-generated text) to distinguish it from what humans made. Third, on the issue of data provenance and copyright, the legitimate securing of training data and the disclosure of a training-data summary are becoming points of contention.

The EU AI Act classifies such risks into risk-based 4 tiers for regulation. This classification, an implementation of the proportionality principle that "only what carries large risk is strongly regulated," is frequently cited in Professional Engineer answers.

Risk Tier Example Regulation
Unacceptable Social scoring, real-time remote biometric identification Prohibited in principle
High Hiring, lending, healthcare, judicial AI Conformity assessment, documentation, human oversight obligatory
Limited Chatbots, deepfakes Transparency (disclosing it is AI) obligation
Minimal Spam filters, game AI No regulation (voluntary)

This approach of imposing heavier obligations the greater the risk is a practical compromise that filters out only dangerous uses while avoiding the innovation chilling that arises when all AI is regulated uniformly. However, one must note that even the same technology falls into a different tier depending on the context in which it is used. For example, facial recognition is minimal risk when used to create game characters, but corresponds to unacceptable when used for real-time surveillance in public places. In other words, the object of regulation is not "the technology itself" but "the use and context in which that technology is placed," which shows that AI governance is not technology control but social risk management.

As likely exam directions: ① a question asking to diagram AI ethics principles and the governance model as a layered structure and explain the role of each layer, ② a question comparing the EU AI Act, NIST AI RMF, and ISO 42001 and discussing the difference from Korea's AI Basic Act, and ③ a practical question asking to design a governance system when introducing high-risk AI in a particular industry (healthcare, finance) are strong candidates. When writing an answer, not stopping at listing principles but developing "conflict and balance between principles," "embedding at the design stage," and "continuous monitoring" from the Professional Engineer perspective is the high-score strategy.

6. Considerations and Implications

  1. Embedding from the design stage (Responsible AI by Design) is the key. Because responding after a problem erupts costs far more to recover trust, bias, explainability, and safety must be put in as requirements from early development. Prior design is always cheaper than a post-hoc audit.
  2. A balance of self-regulation (ethics) and external regulation (law) is needed. A risk-based approach that strongly regulates high-risk uses (high-risk AI) and leaves low risk to voluntary means harmonizes innovation and safety. Over-regulation kills innovation, and under-regulation loses trust.
  3. Maintaining human oversight (Human-in-the-loop) is the last safeguard. Especially in high-risk areas, it must be designed so that AI does not monopolize the final decision and humans review and take responsibility. One must also guard against humans uncritically accepting AI decisions due to automation bias.
  4. Continuous monitoring and feedback are essential. Since AI's performance and fairness can degrade due to changes in data distribution even after deployment (model drift), governance must be carried through to the operational stage with MLOps-based constant surveillance and a retraining system.
  5. Securing global regulatory conformity is important. Because multinational services must satisfy differing regulations such as the EU AI Act and Korea's AI Basic Act simultaneously, designing compliance to the strictest standard (highest common denominator) is efficient.

References

In one line: AI ethics handles the principles of fairness, transparency, accountability, privacy, and human-centeredness, the governance model executes them through the layers of principles → organization → impact assessment → monitoring → regulatory compliance, and it realizes Trustworthy AI through the EU AI Act's risk-based regulation, embedding at the design stage, and human oversight.