Governing AI Development

Domain III asks: how do you govern the people, processes, and decisions that build AI systems? Governance must be embedded at every stage of development — not retrofitted after deployment.

The Core Principle: AI governance is not a compliance checkbox at the end — it is a continuous practice woven through every development decision. The earlier governance is applied, the cheaper and more effective it is. A bias caught in the data collection phase costs far less to fix than one discovered after deployment at scale.

AI Development Lifecycle — Governance at Every Stage

Stage 1

Problem Definition

Define intended use, scope, affected parties; assess need for AI impact assessment

Stage 2

Data Collection

Establish legal basis; assess data provenance, representativeness, consent

Stage 3

Data Preparation

Apply pre-processing bias controls; data quality checks; document Datasheet for Dataset

Stage 4

Model Training

Apply in-processing fairness constraints; version control; document training decisions

Stage 5

Evaluation & Testing

Fairness/bias testing; red teaming; adversarial testing; document Model Card

Stage 6

Deployment

Human oversight controls; shadow deployment; post-processing bias controls; incident response plan

Stage 7

Monitoring & Maintenance

Monitor for model drift; feedback loop management; periodic re-evaluation; retirement planning

Four Governance Domains in AI Development

🗄️

Data Governance

Controlling how training data is collected, assessed for quality and bias, documented, and managed throughout its lifecycle. Garbage in = garbage out.

🧠

Model Development Controls

Documentation standards (Model Cards, Datasheets), testing requirements, privacy/security by design, and version control for AI models.

👁️

Human Oversight

The mechanisms — HITL, HOTL, HIC — through which humans remain in control of AI decision-making at different levels of automation.

🔍

Accountability & Auditing

Red teaming, algorithmic audits, third-party assessments, and the documentation trails that enable AI systems to be evaluated, explained, and held accountable.

📋 Domain III: What Gets Tested on the AIGP Exam

Topic Area	Key Concepts
Data governance for AI	Data quality dimensions, data provenance, data lineage, training data documentation (Datasheets for Datasets)
Bias identification & mitigation	Pre-processing, in-processing, post-processing approaches; fairness metrics; demographic parity
Model documentation	Model Cards — intended use, performance across groups, limitations; technical documentation
Privacy & security by design	7 Privacy by Design principles; differential privacy; federated learning; data minimization in AI
Human oversight	HITL vs HOTL vs HIC; meaningful human review; override capabilities
Testing & evaluation	Fairness testing, adversarial testing, red teaming, shadow deployment, A/B testing
Supply chain & third-party AI	Open source risks, model provenance, vendor governance, inherited bias
Model drift	Data drift vs concept drift; monitoring triggers; retraining thresholds

Data Governance for AI

AI models are only as good as the data they are trained on. Data governance for AI encompasses the policies, practices, and controls that ensure training data is high quality, representative, legally compliant, and well-documented.

Why Data Governance is Different for AI: Traditional data governance focuses on operational data. AI data governance adds unique challenges: training data can encode historical discrimination, data collected for one purpose may not be valid for training, and the relationship between data quality and model behavior is often opaque.

Data Quality Dimensions

Accuracy

Data correctly reflects the real-world entities or events it represents.

e.g., Labels on medical images match actual diagnoses

Completeness

All required data is present; missing values are identified and addressed.

e.g., No missing demographic fields in training set

Consistency

Data is uniform across sources; no contradictory values for the same entity.

e.g., Same address format used across all records

Representativeness

Training data reflects the diversity of the real-world population the model will serve.

e.g., Facial recognition trained on all skin tones

Timeliness

Data is current and relevant; outdated data can cause concept drift.

e.g., Fraud patterns from 2015 may not reflect today's methods

Validity

Data conforms to defined formats, ranges, and business rules.

e.g., Age values within plausible human range

Bias in Training Data — Three Mitigation Stages

Stage 1 · Before Training

Pre-Processing

Intervene in the training DATA before the model sees it. Modify the dataset to reduce bias at its source. This is the earliest and often most effective intervention.

Techniques: Resampling (oversample underrepresented groups), reweighting (assign higher loss weight to minority examples), data augmentation (add synthetic samples), relabeling (correct mislabeled data)

Stage 2 · During Training

In-Processing

Modify the ALGORITHM or training OBJECTIVE to incorporate fairness constraints. The model learns to balance accuracy with fairness during optimization.

Techniques: Adversarial debiasing (train adversary to minimize ability to predict protected attribute), fairness constraints in loss function, regularization terms penalizing disparate impact

Stage 3 · After Prediction

Post-Processing

Adjust the model's OUTPUTS or DECISION THRESHOLDS after predictions are made. The model itself is unchanged — corrections applied to what users see.

Techniques: Threshold adjustment per group (equalized odds), output calibration, reject option classification (human review for borderline cases)

Fairness Metrics — Key Definitions

Metric	Definition	Exam Hook
Demographic Parity	The positive outcome rate is equal across demographic groups regardless of qualifications	Also called "statistical parity" — equal selection rates
Equal Opportunity	Among qualified individuals, the true positive rate is equal across groups	Focuses on not missing qualified people from any group
Equalized Odds	Both true positive rate AND false positive rate are equal across groups	Strongest fairness constraint — hardest to satisfy simultaneously
Individual Fairness	Similar individuals receive similar predictions regardless of group membership	Contrast with group fairness — focuses on like-for-like treatment
Calibration	Among individuals predicted at 70% probability, 70% actually have that outcome — across all groups	Recidivism tool COMPAS controversy: calibrated but racially biased in error rates

📜 Data Provenance & Lineage

Data Provenance: The origin, source, and chain of custody of training data — where it came from, how it was collected, what transformations were applied, and whether the collection was lawful and consented.

Data Lineage: The end-to-end flow of data through the AI pipeline — from ingestion through transformations, feature engineering, and into the model. Enables traceability and auditability.

Both are required for AI accountability — if a model produces a biased output, data provenance and lineage enable the governance team to trace the bias back to its source.

📦 AI Supply Chain Risk

AI systems increasingly rely on third-party components: pre-trained foundation models, open-source libraries, external datasets, and API-based AI services. Each component introduces supply chain risk.

Key risks: Inherited bias from upstream training data; unknown provenance of pre-trained weights; security vulnerabilities in dependencies; intellectual property and licensing issues with open-source models; vendor lock-in limiting future governance control.

Governance response: AI Bill of Materials (AI-BOM) documents all components; vendor due diligence assessments; contractual data governance obligations for third-party providers.

Model Development Controls

Good AI governance requires standardized documentation, built-in privacy and security protections, meaningful human oversight, and rigorous testing before any model reaches users.

Documentation as Governance: Model Cards and Datasheets for Datasets are not just paperwork — they are the primary mechanism for communicating AI system characteristics to deployers, regulators, and affected communities. Without them, accountability is impossible.

Model Cards vs Datasheets for Datasets

Dataset Documentation

Datasheet for Datasets

Gebru et al. (2018/2021) — inspired by electronics component datasheets

Documents a TRAINING DATASET — who created it, how it was collected, what it contains, its limitations, and recommended/prohibited uses.

Motivation — why was this dataset created?
Composition — what is in it? Size? Demographics?
Collection process — how was data obtained?
Preprocessing — what cleaning was applied?
Uses — recommended and prohibited use cases
Distribution — how is it licensed and shared?
Maintenance — who maintains it going forward?

Model Documentation

Model Card

Mitchell et al. (2019, Google) — standardized reporting for trained models

Documents a TRAINED MODEL — its intended uses, performance across demographic groups, limitations, and ethical considerations.

Model details — type, training approach, version
Intended use — primary use cases and out-of-scope uses
Factors — relevant demographic or contextual factors
Metrics — performance measures used
Evaluation data — what data was used to test?
Quantitative analyses — performance disaggregated by group
Ethical considerations and caveats

Human Oversight Tiers

Human-in-the-Loop (HITL)

Human approves each decision before it is executed

The AI generates a recommendation or output, but a human reviews and approves or rejects it BEFORE any action is taken. Highest level of human control; slowest throughput.

Use case: AI-assisted medical diagnosis — radiologist reviews AI output before clinical decision; loan applications where AI scores but human approves.

Human-on-the-Loop (HOTL)

AI acts autonomously; human monitors and can intervene

The AI executes decisions automatically at speed, but a human monitors the outputs in real time and can override or pause the system. Balances speed with oversight.

Use case: Autonomous trading system — human monitors P&L and can kill switch; content moderation AI — human reviews flagged edge cases after initial automated action.

Human-in-Command (HIC)

Human retains ultimate strategic control over AI systems

Humans maintain overarching authority to modify, retrain, shut down, or redesign AI systems at the organizational or societal level. Not about individual decisions — about systemic control.

Use case: Board-level AI governance policy; national AI oversight authority; ability to require decommissioning of a harmful AI system.

Privacy by Design — 7 Foundational Principles

Developed by Dr. Ann Cavoukian, former Ontario Privacy Commissioner. Embedded in GDPR and referenced in AI governance frameworks globally.

Proactive Not Reactive — Preventative Not Remedial

Anticipate and prevent privacy risks before they occur. Privacy governance addresses problems before they arise, not after a breach.

Privacy as the Default Setting

Users should receive maximum privacy protection automatically, without any action required. Privacy is the default, not something users must opt into.

Privacy Embedded into Design

Privacy is built into the system architecture and business practices — not added on as an afterthought or a bolt-on feature after the fact.

Full Functionality — Positive-Sum, Not Zero-Sum

Privacy does not require sacrificing security or functionality. Both can coexist. "False dichotomies" (privacy vs. security) are rejected.

End-to-End Security — Lifecycle Protection

Data is securely retained and then securely destroyed when no longer needed. Security protections span the entire data lifecycle, not just collection.

Visibility and Transparency

All stakeholders can verify that privacy practices operate as stated. Open to independent verification. No hidden agendas.

Respect for User Privacy — Keep It User-Centric

User interests remain paramount. Strong privacy defaults, appropriate notice, and empowering individuals to exercise their rights.

Privacy-Preserving AI Techniques

🔒 Differential Privacy

Mathematical guarantee that the model's outputs are not significantly affected by any single individual's data. Noise is added during training to prevent inference of individual records.

Trade-off: More privacy = less accuracy. The privacy budget (ε) controls this balance.

🌐 Federated Learning

Model training occurs locally on individual devices or nodes. Only model updates (gradients) — not raw data — are shared with a central server. Raw personal data never leaves its source.

Use case: Keyboard prediction on mobile phones (Google Gboard); medical AI across hospitals without sharing patient records.

🎭 Data Anonymization & Pseudonymization

Anonymization: Irreversibly removes identifiers — data can no longer be linked to individuals. Removes from GDPR scope (if truly achieved).

Pseudonymization: Replaces direct identifiers with pseudonyms. Re-identification still possible with additional data. Still subject to GDPR.

Accountability & Auditing

How do you verify that an AI system is behaving as intended, is fair, and does not cause harm? This tab covers the testing and evaluation practices that make AI systems accountable — before and after deployment.

Accountability Requires Evidence: You cannot claim your AI is fair, safe, or compliant without testing it. The governance practices in this tab — red teaming, algorithmic auditing, testing regimes — are what transform governance principles into verifiable claims.

Pre-Deployment Testing

Red Teaming for AI Systems

What Red Teaming Is

Structured adversarial testing by a dedicated team attempting to find failures, harms, and vulnerabilities
Simulates how bad actors or edge-case users might misuse the system
Tests for both safety failures (harmful outputs) and security failures (prompt injection, model extraction)
Required for GPAI with systemic risk under the EU AI Act
Can be internal, external, or a combination (e.g., third-party red team firms)

Red Team Focus Areas for AI

Jailbreaking: bypassing safety filters through prompt manipulation
Prompt injection: inserting instructions that override system prompts
Bias elicitation: prompts that reveal discriminatory outputs
Hallucination: inducing confident false statements
Privacy leakage: extracting training data or PII from the model
Harmful content: generating dangerous, illegal, or hateful outputs

Testing Types for AI Systems

⚖️

Fairness / Bias Testing

Evaluates whether model outputs are equitable across demographic groups. Uses fairness metrics (demographic parity, equalized odds). Required for high-risk AI under EU AI Act.

⚔️

Adversarial Testing

Applies adversarial inputs designed to cause model failures — adversarial examples that fool image classifiers, or adversarially crafted text prompts. Tests robustness.

🪞

Shadow Deployment

New model runs in parallel with the existing system (or offline) receiving live traffic but NOT serving responses. Outputs are compared to identify performance gaps before going live.

🔬

A/B Testing

Two versions of the model serve different user segments. Outcomes compared to determine which version performs better on defined metrics — including fairness metrics.

🔄

Regression Testing

Verifies that updates or retraining do not degrade previously working functionality. AI-specific concern: model updates can unexpectedly change behavior on subpopulations.

🔮

Out-of-Distribution Testing

Tests model behavior on inputs outside its training distribution — edge cases, unusual combinations, inputs from populations not well-represented in training data.

Algorithmic Auditing

🔎 Types of Algorithmic Audits

First-party (internal): Conducted by the organization that built or deploys the AI. Lowest independence but deepest access to the system.

Second-party (regulatory): Conducted by a regulator or government authority. Required for high-risk AI under the EU AI Act (conformity assessment) and financial sector AI (model risk management).

Third-party (independent): Conducted by external auditors with no stake in the outcome. Highest independence; growing field of algorithmic audit firms.

Effective audits require: access to training data, model architecture, testing results, documentation, and deployment context — not just the model output alone.

🏦 Model Risk Management (SR 11-7)

The US Federal Reserve's Supervisory Guidance on Model Risk Management (SR 11-7, 2011) established principles for governing AI/ML models in financial institutions — predating the AI governance field by a decade.

Key SR 11-7 principles: Model validation by independent parties; conceptual soundness assessment; outcome analysis; ongoing monitoring; model inventory.

These financial sector principles have strongly influenced AI governance frameworks broadly — the emphasis on independent validation, documentation, and ongoing monitoring appear throughout NIST AI RMF and ISO/IEC 42001.

📊 Model Drift — Monitoring Post-Deployment

Drift Type	What Changes	Example	Governance Response
Data Drift	Distribution of input data changes from training data	Fraud patterns change; demographics of users shift	Monitor input distributions; set drift thresholds; trigger review
Concept Drift	Relationship between inputs and the target outcome changes	COVID changed what "healthy" baseline looks like; economic shift changes credit risk	Monitor outcome metrics; compare predictions vs actual outcomes; scheduled retraining
Prediction Drift	Model's output distribution changes without input distribution change	Model begins approving more loans without clear cause	Monitor output distributions; alert on threshold breach
Label Drift	Ground-truth labels themselves change in meaning or prevalence	Regulatory change redefines what counts as a "suspicious transaction"	Update ground truth; re-evaluate; consider full retraining

Practice Quiz — Domain III

Test your knowledge of AI development governance. Select the best answer for each question.

Question 1 of 10

A data scientist notices that a hiring AI underperforms for women candidates. To fix this, they decide to add more female candidate records to the training set and assign higher loss weights to underrepresented examples. This is an example of which bias mitigation approach?

APost-processing — adjusting the model's output thresholds after predictions are made

BIn-processing — modifying the model's training algorithm to incorporate fairness constraints

CPre-processing — modifying the training data before the model is trained (resampling and reweighting)

DTransfer learning — using a pre-trained model and fine-tuning it on a more representative dataset

Adding records (resampling/data augmentation) and assigning higher loss weights to underrepresented groups are both PRE-PROCESSING techniques — they intervene in the training data BEFORE the model is trained. Post-processing adjusts outputs after prediction. In-processing modifies the learning algorithm itself during training.

Question 2 of 10

A "Model Card" is a standardized documentation artifact that primarily serves to:

ADescribe the provenance, collection method, composition, and recommended uses of a training dataset

BProvide structured, transparent information about a trained model's intended use, performance across groups, limitations, and ethical considerations

CDocument an organization's compliance with ISO/IEC 42001 AI Management System requirements

DRecord the results of a red team security and adversarial testing assessment

A Model Card (Mitchell et al., 2019) documents a TRAINED MODEL — its intended use cases, performance disaggregated by demographic groups, limitations, and ethical considerations. Option A describes a "Datasheet for Datasets" (Gebru et al.) — a common exam trap. They are companion documents that serve different purposes.

Question 3 of 10

Which of the following best describes Privacy by Design Principle #2: "Privacy as the Default Setting"?

APrivacy protections are activated only after a user explicitly requests them through account settings

BThe maximum level of privacy protection is provided automatically without any action required from the user — privacy is the default, not an opt-in

CPrivacy compliance defaults to the legal requirements of the jurisdiction where the organization is headquartered

DPrivacy protections are applied by default only to sensitive categories of data such as health or financial information

"Privacy as the Default Setting" means users get the maximum privacy protection AUTOMATICALLY — they don't need to take any action to protect their privacy. It's already protected. This is the opposite of "opt-in privacy" where users must actively request protection. This principle is also reflected in GDPR's data minimization and purpose limitation requirements.

Question 4 of 10

Human-in-the-Loop (HITL) differs from Human-on-the-Loop (HOTL) in which fundamental way?

AHITL requires a human to review and approve each AI decision BEFORE it is executed; HOTL allows the AI to act autonomously while humans monitor and can intervene after the fact

BHITL is required for all high-risk AI systems under the EU AI Act; HOTL is only required for limited-risk systems

CHITL uses a committee of reviewers; HOTL assigns oversight responsibility to a single designated person

DHITL applies during the model training phase; HOTL applies after the model has been deployed to production

HITL = human approval BEFORE each decision (highest control, lowest speed). HOTL = AI acts first, human monitors in real time and can interrupt or override. The key difference is WHEN human review happens — before the decision (HITL) or after the AI acts but with ability to intervene (HOTL). Human-in-Command (HIC) is the third tier: strategic/organizational level control over AI systems.

Question 5 of 10

"Datasheets for Datasets" (Gebru et al.) were introduced to address which core AI governance challenge?

ADocumenting the legal compliance status of AI systems under applicable regulations

BRecording the performance accuracy and F1 scores of trained models across benchmark datasets

CProviding transparency about the provenance, composition, collection process, and appropriate use of training datasets to promote accountability

DCertifying that datasets meet the quality standards required by ISO/IEC 42001:2023

Datasheets for Datasets (Gebru et al., inspired by electronics component datasheets) were introduced to document TRAINING DATA — covering who created it, how it was collected, what it contains, what preprocessing was applied, and what uses are recommended or prohibited. They address the lack of transparency about training data that makes AI bias and misuse hard to investigate.

Question 6 of 10

An AI developer uses an adversarial debiasing technique that simultaneously trains the main model and an adversary network that attempts to predict protected attributes from model outputs. This is an example of which bias mitigation approach?

APre-processing — modifying training data before the model is trained

BIn-processing — modifying the learning algorithm during training to incorporate fairness constraints

CPost-processing — adjusting model output thresholds after predictions are generated

DTransfer learning — using domain adaptation to reduce inherited bias from a pre-trained model

Adversarial debiasing modifies the TRAINING PROCESS ITSELF — it is an in-processing technique. The adversary network is trained simultaneously with the main model, penalizing the main model when the adversary can infer protected attributes. This happens during training, not before (pre-processing) or after (post-processing).

Question 7 of 10

What does "model drift" refer to, and why is it a governance concern?

AA deliberate modification of a model's parameters by an adversary to manipulate its outputs for malicious purposes

BThe gradual degradation of model performance over time as real-world data distributions change from the training data distributions — a governance concern because a model that was fair and accurate at launch may become neither over time

CThe migration of an AI model from one cloud provider to another, requiring re-validation of all governance controls

DThe process by which a model's outputs gradually become more similar to those of competing models due to training on similar datasets

Model drift refers to performance degradation as the world changes in ways the training data no longer reflects. Data drift = the input distribution changes (e.g., new fraud patterns). Concept drift = the relationship between inputs and the target outcome changes (e.g., economic conditions change what makes someone a credit risk). Governance concern: a model certified as fair at deployment may cause disproportionate harm months later without anyone noticing.

Question 8 of 10

In the context of AI development governance, "red teaming" is best described as:

AA dedicated monitoring team that tracks AI system performance in production and alerts on anomalies

BA structured adversarial testing process where a team attempts to identify failures, harms, vulnerabilities, and misuse potential in an AI system before it is deployed

CA regulatory audit conducted by government compliance authorities on deployed high-risk AI systems

DA process of retraining a model using adversarially generated examples to improve its robustness against attacks

Red teaming is structured adversarial PRE-DEPLOYMENT testing — a dedicated team simulates bad actors and edge cases to expose the AI system's failure modes before real users encounter them. It tests for jailbreaking, prompt injection, hallucination, harmful outputs, bias elicitation, and privacy leakage. Required for high-capability GPAI under the EU AI Act.

Question 9 of 10

A company uses "shadow deployment" to evaluate a new AI model before replacing the existing one. What does this testing approach involve?

ARunning the new model in complete secrecy so competitors cannot replicate the approach

BSplitting live traffic equally between the old and new model to compare performance in real conditions (A/B testing)

CRunning the new model in parallel with the live system — receiving real traffic but NOT serving responses to users — so its outputs can be compared to the existing system's outputs without affecting users

DDeploying the new model only to internal employees before rolling it out to external customers

Shadow deployment runs the new model alongside the live system, receiving real inputs but generating outputs that are NOT shown to users — they are captured for comparison. This reveals how the new model would have behaved on real traffic without any risk of harm to users. It's distinct from A/B testing (B), where both models serve real users.

Question 10 of 10

An organization uses a third-party foundation model as the basis for its AI product. From a governance perspective, what is the PRIMARY concern about "inherited bias" in this context?

AThe organization may not have a valid software license for the third-party model

BBiases present in the foundation model's training data or architecture may persist in the organization's fine-tuned product, making the organization responsible for harms it did not directly cause

CThe organization cannot claim its product is AI-powered if it is based on a third-party model

DThe third-party model provider will retain ownership of all outputs generated by the fine-tuned version

Inherited bias means that when you fine-tune or deploy a pre-trained foundation model, its upstream biases travel with it into your product. The organization deploying the model typically bears responsibility for its outputs — regardless of whether the bias originated in third-party training data. This is why AI supply chain governance, vendor due diligence, and testing for inherited bias are essential governance practices.

0/10

Questions correct — review explanations above

Memory Hooks

Mental models and mnemonics for the Domain III concepts most likely to appear on exam day.

🗂️

Pre / In / Post — The Bias Mitigation Ladder

Think of bias mitigation as three places to intervene: PRE-training (fix the data), IN-training (fix the algorithm), POST-prediction (fix the output). Earlier = more effective and less invasive.

Mnemonic: "Fix Data early, Fix Algorithm during, Fix Output last — PIP"

📄

Model Card vs Datasheet — Who Documents What

Datasheet = documents the DATASET (training data). Model Card = documents the MODEL (trained system). Same word pairing as the thing it documents.

Mnemonic: "Datasheet → Data. Model Card → Model." (Gebru made the sheet; Mitchell made the card)

👁️

HITL / HOTL / HIC — The Oversight Ladder

Human In the Loop = human approves each decision BEFORE. Human On the Loop = AI acts, human watches and can stop. Human In Command = strategic control at the organizational level.

Mnemonic: "In the loop = IN before action. On the loop = ON-call to stop it. In command = IN charge of the whole system."

🛡️

Privacy by Design — P + D = Proactive Default

The two most-tested PbD principles: Proactive (prevent, don't react) and Default (maximum privacy without user action). The other 5 flow from these two foundational ideas.

Mnemonic: "PbD starts with P and D — Proactive Default. Then Embed, Positive-sum, End-to-end, Visible, User-centric."

📊

Data Drift vs Concept Drift

Data Drift = the INPUT distribution changes (the world you're observing has shifted). Concept Drift = the RELATIONSHIP between inputs and the outcome changes (what the inputs mean has shifted).

Mnemonic: "Data drift = different people/things. Concept drift = same things, different meaning."

🎯

Shadow vs A/B — Who Gets the Output

Shadow deployment: New model sees real inputs but its outputs go NOWHERE (hidden). Compare offline. A/B testing: Both models serve REAL users — each user gets one model's output. Shadow = zero user risk; A/B = some user exposure.

Mnemonic: "Shadow hides its outputs. A/B shows both to users."

🔄 Quick Reference: Documentation vs Testing vs Oversight

Artifact / Practice	Type	When Applied	Key Purpose
Datasheet for Datasets	Documentation	Data collection/prep (Stage 2–3)	Transparency about training data provenance and characteristics
Model Card	Documentation	Post-training evaluation (Stage 5)	Transparency about trained model performance, limitations, and ethics
DPIA	Assessment	Before processing begins (Stage 1)	Evaluate high risks to individuals' rights under GDPR Art 35
Red Teaming	Testing	Pre-deployment (Stage 5)	Adversarially identify failures, harms, and vulnerabilities
Shadow Deployment	Testing	Pre-deployment (Stage 5–6)	Compare new model to existing on real traffic without user exposure
HITL / HOTL	Oversight	Deployment & operation (Stage 6–7)	Maintain human control over AI decision-making
Drift Monitoring	Monitoring	Post-deployment (Stage 7)	Detect performance degradation from changing data or concepts

Flashcards & Study Advisor

Tap any card to flip it. Use the advisor panel for targeted study guidance by topic.

Flashcards — Domain III Key Concepts

Bias Mitigation

What are the 3 bias mitigation stages and when does each intervene?

tap to reveal

Answer

Pre-processing = fix training DATA before training. In-processing = constrain ALGORITHM during training. Post-processing = adjust OUTPUTS after prediction.

Documentation

Model Card vs Datasheet for Datasets — what does each document?

tap to reveal

Answer

Datasheet for Datasets (Gebru et al.) = documents TRAINING DATA (provenance, composition, collection). Model Card (Mitchell et al.) = documents the TRAINED MODEL (intended use, performance by group, limitations).

Human Oversight

HITL vs HOTL vs HIC — what distinguishes each tier?

tap to reveal

Answer

HITL = human approves EACH decision BEFORE action. HOTL = AI acts autonomously; human monitors and can intervene. HIC = human holds strategic/organizational control over AI systems.

Privacy by Design

Name the first 2 (most tested) Privacy by Design principles.

tap to reveal

Answer

1) Proactive not reactive — prevent privacy harms before they occur. 2) Privacy as default — maximum protection automatically, no user action required.

Model Drift

Data drift vs Concept drift — what is the difference?

tap to reveal

Answer

Data drift = input distribution changes (different people/patterns than training). Concept drift = the RELATIONSHIP between inputs and the outcome changes (same inputs now mean something different).

Testing

What is shadow deployment and how does it differ from A/B testing?

tap to reveal

Answer

Shadow deployment: new model receives real inputs but outputs are NOT shown to users — captured for offline comparison. Zero user risk. A/B testing: both models serve real users and their outputs are compared. Some user exposure.

Privacy Tech

What is federated learning and what privacy problem does it solve?

tap to reveal

Answer

Training occurs LOCALLY on devices/nodes — only model updates (gradients) are shared centrally, never raw personal data. Solves: training on sensitive data (medical, financial) without centralizing it. Used in Google Gboard, medical AI research.

AI Supply Chain

What is "inherited bias" when using a third-party foundation model?

tap to reveal

Answer

Biases encoded in the foundation model's training data or architecture carry into fine-tuned products. The deploying organization typically bears responsibility for output harms — even if the bias originated upstream. Requires vendor due diligence and bias testing.

Master the Full AIGP Deck on FlashGenius

All 4 domains. Hundreds of flashcards. Spaced repetition to make it stick.

Unlock Full Flashcard Deck on FlashGenius →

Study Advisor

Data Governance

Bias Mitigation

Model Documentation

Oversight & Privacy

Testing & Auditing

Data Governance — Exam Focus Points

Data quality dimensions most tested: accuracy, completeness, representativeness, and timeliness. Know how each affects model fairness and reliability.
Representativeness is the dimension most specific to AI — training data must reflect the diversity of the real-world population the model serves.
Distinguish data provenance (where did this data come from?) from data lineage (how did data flow through the pipeline?). Both enable traceability.
AI supply chain risk is a growing exam focus — using third-party models, open source libraries, and external datasets all introduce inherited bias, security vulnerabilities, and governance gaps.
Know what an AI Bill of Materials (AI-BOM) is — a complete inventory of all components in an AI system, analogous to a software bill of materials (SBOM).
Training data collected for one purpose cannot simply be reused to train an AI model without a legal basis under GDPR's purpose limitation principle (Art 5).

Bias Mitigation — Exam Focus Points

The three-stage framework (pre/in/post) is the most tested topic in Domain III. Know the stage name, when it intervenes, and 2–3 techniques for each.
Pre-processing techniques: resampling (oversample underrepresented groups), reweighting (higher loss weight for minority examples), data augmentation (add synthetic samples).
In-processing techniques: adversarial debiasing (adversary tries to predict protected attribute — main model is penalized when it succeeds), fairness regularization in the loss function.
Post-processing techniques: threshold adjustment (different decision thresholds by group to achieve equalized odds), calibration adjustment.
Know the fairness metric definitions: demographic parity (equal selection rates), equalized odds (equal TPR and FPR across groups), individual fairness (similar treatment for similar individuals).
The COMPAS recidivism tool controversy is a canonical case study: the tool was calibrated (accurate on aggregate) but had racially disparate false positive rates — illustrating that multiple fairness metrics can conflict.

Model Documentation — Exam Focus Points

The most common exam trap: confusing Model Cards and Datasheets for Datasets. Remember: Model Card → trained MODEL. Datasheet → training DATASET.
Model Cards (Mitchell et al., Google, 2019): document intended use, out-of-scope uses, factors, metrics, evaluation data, and performance disaggregated by group.
Datasheets for Datasets (Gebru et al., 2018): document motivation, composition, collection process, preprocessing, uses, distribution, and maintenance.
Both are referenced in the EU AI Act's technical documentation requirements for high-risk AI and in ISO/IEC 42001.
System Cards (Meta) extend Model Cards to multi-component AI systems with multiple models and interfaces — an evolution beyond single-model documentation.
Documentation is not a one-time activity — both Model Cards and Datasheets must be updated when models are retrained or datasets are modified.

Oversight & Privacy — Exam Focus Points

HITL, HOTL, and HIC are frequently tested — know the precise definition and a concrete use case for each.
HITL: human approval BEFORE each AI decision. Slowest, highest control. Example: radiologist reviewing AI scan results before clinical action.
HOTL: AI acts first, human monitors in real time and can interrupt. Example: autonomous trading with a human kill-switch watching P&L.
The EU AI Act's human oversight requirement for high-risk AI generally requires HITL or HOTL — not just HIC.
For Privacy by Design, the two most-tested principles are #1 (Proactive) and #2 (Privacy as Default). Know all 7 in order.
Federated learning and differential privacy are the two most-tested privacy-preserving AI techniques. Federated learning = train locally; differential privacy = add mathematical noise to prevent individual inference.
Differential privacy's privacy budget (ε): lower ε = more privacy, less accuracy. Higher ε = less privacy, more accuracy. This is a fundamental trade-off.

Testing & Auditing — Exam Focus Points

Red teaming = structured adversarial testing BEFORE deployment. Focus areas: jailbreaking, prompt injection, bias elicitation, hallucination, privacy leakage, harmful content.
Shadow deployment vs A/B testing: shadow = new model hidden, zero user exposure. A/B = both models serve real users. Shadow is lower risk; A/B provides real behavioral data.
Know the three types of algorithmic audits by party: first-party (internal, deep access), second-party (regulatory), third-party (independent, highest credibility).
SR 11-7 (Federal Reserve, 2011) is AI governance's oldest ancestor — model validation, independent review, ongoing monitoring. Know that it predates modern AI governance but heavily influenced it.
Model drift monitoring: data drift = input distribution changes; concept drift = outcome relationship changes. Both require monitoring thresholds and retraining triggers.
Technical debt in AI governance: shortcuts during development (skipped documentation, inadequate testing) compound into governance failures post-deployment — harder and costlier to fix than if addressed early.