AI Controls, Monitoring & Incidents

Effective AI governance requires more than policies — it demands operationalized controls, continuous performance monitoring, and structured incident response capabilities.

    Auditor's Lens: For every AI control you encounter, ask three questions: Is it designed effectively? (does it address the risk?) Is it operating effectively? (is it actually working?) Is there evidence? (can it be tested?) A control that exists on paper but lacks evidence of operation is an audit finding.
  

Three Lines of Defense — Applied to AI

First Line — AI Development & Business Units

Owns: AI product teams, model owners, data scientists, business process owners

Responsible for day-to-day operation, implementation of AI controls, and identification of risks within the AI system. Owns the AI system and is accountable for its performance and compliance. Implements preventive and detective controls directly.

AI examples: Model validation testing before deployment, data quality checks in the pipeline, access controls on training data, human-in-the-loop review queues, bias testing reports

Second Line — Risk Management & AI Governance

Owns: AI risk function, Chief AI Officer, compliance team, AI governance committee

Provides oversight, policy, and standards for AI risk management. Does NOT own the AI system. Monitors whether the first line is managing risks effectively and reports to senior management. Sets risk appetite and governance frameworks.

AI examples: AI risk register maintenance, AI policy and standards, ongoing monitoring dashboards, fairness threshold reviews, vendor AI risk assessments, escalation protocols for model incidents

Third Line — Internal Audit

Owns: Internal audit function, independent assurance providers

Provides independent assurance to the board and senior management on the effectiveness of AI governance, risk management, and controls. Does not implement controls or manage risk — assesses whether they are adequate and effective.

AI examples: AI audit engagements, testing of model validation processes, reviewing AI policy compliance, assessing bias testing completeness, evaluating incident response capabilities

AI Control Types

🛡️

Preventive

Stop the risk before it occurs

Controls designed to prevent AI failures or misuse from happening in the first place. Most cost-effective when risks are high.

AI examples: Pre-deployment model validation gates, data quality thresholds that block model training, access controls on training data, bias testing requirements before production release

🔍

Detective

Identify when something has gone wrong

Controls that identify AI failures, anomalies, or policy violations after they occur. Essential for AI systems that can degrade silently.

AI examples: Performance monitoring dashboards, drift detection alerts, bias metric tracking, anomaly detection on model outputs, audit logs for AI decisions

🔧

Corrective

Fix the issue after detection

Controls that remediate identified AI failures or anomalies. Bring the system back to an acceptable state of performance or compliance.

AI examples: Model rollback procedures, forced retraining on updated data, automated output suppression when confidence below threshold, incident remediation workflows

↔️

Compensating

Mitigate residual risk when primary controls fail

Alternative controls used when primary controls are absent or insufficient. Reduce residual risk to an acceptable level through alternative means.

AI examples: Mandatory human review for all high-stakes AI decisions when explainability is limited; manual sampling and review when automated monitoring is unavailable

Control Design vs. Operating Effectiveness — The Audit Distinction

Assessment Type	Question Asked	Testing Approach	AI Example
Design Effectiveness	Is the control designed well enough to address the risk?	Walkthrough, inspection of control documentation, inquiry	Review the model validation policy — does it require bias testing for demographic parity?
Operating Effectiveness	Is the control actually operating as designed, consistently?	Re-performance, observation, sampling of evidence	Pull 25 model deployment records — was bias testing completed and documented for each?
Design gap	—	The control doesn't address the right risk	Policy requires accuracy testing but not fairness testing — bias risk is unaddressed
Operating gap	—	The control is well-designed but not followed	Policy requires bias testing but 12 of 25 deployments have no testing documentation

AI Control Design & Key Control Areas

The critical control domains within AI systems that auditors assess — from data and model controls to human oversight and access management.

    Control Mapping: For each AI risk type, there should be one or more mapped controls. If a risk has no corresponding control, that is a control gap — not just a risk finding. AAIA exams test whether candidates can identify both the risk AND the missing control.
  

Human-in-the-Loop (HITL) Controls

Human oversight for consequential AI decisions

Oversight Control

Three HITL Patterns

Human-in-the-loop: Human reviews every AI decision before it takes effect (high oversight, low throughput). Human-on-the-loop: AI acts autonomously but humans monitor and can intervene (medium oversight). Human-out-of-the-loop: Fully automated; human only reviews at exception or audit (low oversight — high-risk for consequential decisions).

When HITL is Required

GDPR Article 22 requires human review for solely automated decisions with significant effects (credit, employment, insurance). HITL is also required whenever: explainability is insufficient to justify an automated decision; the AI system is operating in a novel scenario; or model confidence is below the defined threshold.

⚡ Exam Hook: "Human-on-the-loop" is NOT the same as "human-in-the-loop." A system where a human CAN intervene but isn't required to review every decision may not satisfy GDPR Article 22 for consequential decisions. Know the three patterns and their appropriate use cases.

Model Validation Controls

Pre-deployment and ongoing model quality assurance

Preventive Control

Pre-Deployment Validation

Performance testing: Accuracy, precision, recall, F1 on held-out test data. Robustness testing: How model behaves with adversarial, out-of-distribution, or edge-case inputs. Fairness testing: Demographic parity, equal opportunity across protected groups. Explainability review: Can decisions be explained? Are LIME/SHAP outputs reasonable?

Change Management for AI

Model updates (retraining, architecture changes, threshold changes) must go through formal change management. Key controls: version control for models AND training data, regression testing after updates, documented approval for production deployment, rollback capability. Model cards updated on each version release.

⚡ Exam Hook: AI change management is uniquely complex — even a "minor" retraining on new data can fundamentally change model behavior. Auditors should verify that any model update, including automated retraining pipelines, triggers a validation and approval workflow, not just software change management.

Key AI Control Areas — Audit Checklist

Control Area	Key Controls to Audit	Evidence to Request
Data Quality	Data quality gates, completeness checks, representativeness assessment, data lineage tracking	Data quality reports, pipeline test logs, lineage documentation
Model Validation	Performance testing, fairness testing, robustness testing, independent review of validation results	Model validation reports, test datasets, approval records
Access Control	Least-privilege access to training data and model parameters, separation of duties (developer ≠ validator)	Access control logs, user access reviews, permission matrices
Change Management	Version control for models and data, change approval workflow, regression testing, rollback procedures	Change tickets, version history, deployment approvals
Human Oversight	HITL review queues, override capability, escalation thresholds, decision audit trails	Review queue logs, override records, SLA compliance reports
Monitoring	Performance dashboards, drift alerts, bias metric tracking, anomaly detection	Monitoring dashboards, alert logs, incident tickets from alerts
Explainability	LIME/SHAP outputs available for decisions, explanation quality review, regulatory disclosure capability	Explanation samples, disclosure templates, stakeholder comprehension testing

AI Monitoring & Drift Detection

Continuous monitoring is the primary detective control for AI systems — catching silent degradation, drift, bias creep, and anomalous behavior in production.

    Why AI Monitoring is Different: Unlike traditional software, an AI model can degrade substantially with zero code changes. The real-world environment shifts; the model stays the same. Monitoring must be continuous, statistical, and multi-dimensional — performance metrics alone are insufficient.
  

Three Types of AI Drift

📊

Data Drift (Covariate Drift)

The statistical distribution of input features changes over time, but the underlying relationship between inputs and outputs remains the same. The model hasn't changed; the incoming data looks different from the training data.

Detection: Statistical tests (PSI, KS test) on input feature distributions. Monitor: feature mean/variance shifts over rolling windows.

🔄

Concept Drift (Label Drift)

The relationship between inputs and the correct output changes over time — the world has changed, so the model's learned mapping is no longer valid. Most dangerous form of drift because it silently invalidates the model's logic.

Detection: Monitor model accuracy against ground truth labels when available. Track prediction error rates over time. Requires labeled production data or delayed feedback loops.

📉

Prediction Drift (Output Drift)

The distribution of model outputs (predictions, scores, classifications) changes over time even if inputs look similar. Can be an early indicator of data or concept drift before performance metrics degrade.

Detection: Monitor output score distributions, prediction class ratios, confidence score histograms. Fastest to detect — doesn't require ground truth labels.

Key Model Performance Metrics

Accuracy

(TP + TN) / Total

Overall proportion of correct predictions. Misleading when classes are imbalanced — a model predicting "no fraud" 99% of the time has high accuracy on a 1% fraud dataset.

Precision

TP / (TP + FP)

Of all positive predictions, how many were correct? High precision = few false alarms. Critical when false positives are costly (e.g., blocking legitimate transactions).

Recall (Sensitivity)

TP / (TP + FN)

Of all actual positives, how many did the model catch? High recall = few misses. Critical when false negatives are costly (e.g., missing fraud, missing disease).

F1 Score

2 × (P × R) / (P + R)

Harmonic mean of precision and recall. Balances both — useful when both false positives and false negatives matter and the dataset is imbalanced.

AUC-ROC

Area under ROC curve

Measures model's ability to discriminate between classes across all thresholds. 1.0 = perfect; 0.5 = random. Threshold-independent — useful for comparing models.

PSI

Population Stability Index

Measures the shift in score distributions between training and production. PSI < 0.1 = stable; 0.1–0.2 = slight shift; > 0.2 = significant drift — investigate and consider retraining.

Explainability Tools (XAI)

LIME

Local Interpretable Model-agnostic Explanations

Local Explanation

Generates a locally faithful linear approximation around a specific prediction. Perturbs the input, observes output changes, and fits a simple model to explain that one instance. Works with any model type.

Audit use: Verify that explanations for individual decisions are consistent, non-discriminatory, and understandable to affected parties

SHAP

SHapley Additive exPlanations

Global + Local

Based on game theory (Shapley values) — assigns each feature a contribution score to a prediction. Provides both local (per-prediction) and global (overall model) explanations. More consistent than LIME but computationally heavier.

Audit use: Feature importance analysis at the model level to identify whether protected attributes (or proxies) are driving decisions — key for bias audits

Counterfactual Explanations

Minimum-change explanations for different outcomes

Local Explanation

Answers: "What is the minimum change to the input that would flip the decision?" E.g., "Your loan was denied. If your income were $5,000 higher, it would have been approved." Highly interpretable to affected individuals.

Audit use: Assess whether the explanations provided to users satisfy regulatory "right to explanation" requirements (GDPR Recital 71)

Attention Visualization

Attention weight maps for deep learning models

Local (DL Models)

For transformer-based models (NLP, vision), visualizes which parts of the input the model "attended to" when making a prediction. Not equivalent to feature importance — attention ≠ causation, but provides interpretability signals.

Audit use: Validate that NLP models are using relevant text features, not spurious correlations (e.g., attending to demographic names rather than content)

Monitoring Program — Key Components for AAIA Auditors

Component	What It Tracks	Audit Questions
Performance monitoring	Accuracy, precision, recall, F1 against ground truth labels	Is there a feedback loop to capture ground truth in production? How quickly?
Drift monitoring	Input distribution (PSI), output distribution, concept drift	What drift thresholds trigger alerts? What happens when a threshold is breached?
Bias monitoring	Fairness metrics across demographic groups over time	Are fairness metrics tracked for all protected groups? Is there a remediation SLA?
Anomaly detection	Unusual patterns in predictions, inputs, or volumes	Are there automated anomaly alerts? Who receives them? What is the response process?
Human override tracking	Rate, reasons, and patterns of human overrides of AI decisions	Is the override rate tracked? High override rates may indicate model degradation or poor calibration.
Audit logging	All AI decisions, inputs, outputs, model version, timestamp	Are logs immutable? Retained for required period? Sufficient to reconstruct any decision?

AI Incidents & Adversarial Threats

Classifying, responding to, and learning from AI incidents — plus the emerging landscape of adversarial attacks specific to AI systems.

    AI Incidents vs. IT Incidents: AI incidents can be subtle — gradual bias creep, slow drift, or unexplained output patterns — rather than the clear system outages of traditional IT. AI incident response must include root cause analysis of model and data factors, not just infrastructure failures.
  

AI Incident Severity Classification

Severity 1 — Critical

Immediate Harm or Legal Violation

AI is causing active, measurable harm to individuals or violating applicable law. System must be suspended immediately. Regulatory notification may be required.

Response time: Immediate (minutes–hours); escalate to executive and legal

Severity 2 — High

Significant Performance Degradation

AI accuracy or fairness has degraded significantly; potential for harm is high. Compensating controls (e.g., 100% human review) deployed while root cause is investigated.

Response time: Within hours; senior ownership assigned

Severity 3 — Medium

Threshold Breach — Monitoring Alert

AI performance or drift metrics breach defined thresholds. No confirmed harm yet, but retraining or investigation is required within the defined SLA.

Response time: Within 24–72 hours; model owner investigates

Severity 4 — Low

Minor Anomaly — Tracked

Minor anomaly or trend detected that does not yet breach thresholds. Logged and monitored for escalation. May be addressed in next scheduled model review.

Response time: Next scheduled review cycle

AI Incident Response Lifecycle

Detection & Alerting

Monitoring system (or user report) identifies a potential incident. Alert generated and routed to the model owner and on-call team. Incident ticket opened with severity classification.

Containment

Immediate actions to limit harm. May include: suspending the AI system, routing all decisions to human review, rolling back to a prior model version, or suppressing outputs above/below confidence thresholds.

Investigation & Root Cause Analysis

Deep dive into model performance logs, input data, output distributions, and change history. Determine whether the cause is data drift, concept drift, adversarial attack, data quality issue, or model logic error. Involves both technical and business stakeholders.

Remediation

Address the root cause: retrain on updated data, fix data pipeline, patch model architecture, update fairness thresholds, or replace the model. Remediation goes through change management validation before production re-deployment.

Recovery & Validation

Validate the fixed model meets all performance, fairness, and robustness criteria before returning to full production. Monitoring thresholds may be temporarily tightened post-incident.

Post-Incident Review & Reporting

Conduct a post-mortem: What happened? What controls failed? What new controls are needed? Update the AI risk register and incident log. Regulatory reporting if required. Lessons learned shared across model teams.

Adversarial AI Attack Types

☠️

Data Poisoning

Attacker injects malicious data into the training dataset, causing the model to learn incorrect or backdoored behaviors. Particularly dangerous for models that continuously retrain on new data.

Controls: Training data validation, data source authentication, anomaly detection in data pipelines, human review of data samples

🎭

Evasion Attack

Attacker manipulates input data at inference time to cause the model to misclassify. Classic example: slightly altered images that fool image classifiers while appearing normal to humans (adversarial examples).

Controls: Adversarial training, input validation and sanitization, ensemble models, confidence thresholds to flag uncertain predictions

🔓

Model Inversion

Attacker uses the model's outputs to reverse-engineer sensitive information about training data. E.g., repeatedly querying a medical diagnosis model to reconstruct patient data used in training.

Controls: Rate limiting on API queries, output rounding/perturbation, differential privacy in training, minimum prediction confidence thresholds

💬

Prompt Injection

Attacker embeds malicious instructions in input text to hijack an LLM's behavior — overriding system instructions, extracting private data, or generating prohibited content. Specific to generative AI and LLM systems.

Controls: Input sanitization, output filtering, privilege separation between user and system prompts, red-teaming before deployment, content moderation layers

🕵️

Model Stealing

Attacker queries a model's API extensively to reconstruct a functionally equivalent model without access to the original. Undermines IP protection and allows adversary to perform offline attacks on the cloned model.

Controls: Rate limiting, query monitoring and anomaly detection, output watermarking, API authentication, limiting query detail in API responses

🔍

Membership Inference

Attacker determines whether a specific record was in the model's training data. Significant privacy risk — can reveal whether a person's sensitive data (e.g., medical records) was used to train the model.

Controls: Differential privacy, regularization to reduce overfitting, output perturbation, limiting confidence scores returned by API

Practice Quiz

10 AAIA-style questions on AI Controls, Monitoring & Incidents. Select an answer to see instant feedback.

Question 1 of 10

In the three lines of defense model applied to AI, which line is responsible for day-to-day implementation of AI controls and has direct ownership of the AI system?

A Internal audit

B AI risk management and governance function

C AI development teams and business process owners

D External auditors

The first line of defense consists of the AI development teams and business units that own and operate the AI system. They are responsible for implementing controls, identifying risks in day-to-day operations, and maintaining the AI system's performance. The second line provides oversight and policy; the third line provides independent assurance.

Question 2 of 10

An AI monitoring system automatically flags a model for investigation when its accuracy drops below a defined threshold and routes all decisions to human review. This is BEST classified as which type of control?

A Preventive

B Detective

C Corrective

D Compensating

The monitoring system that detects the threshold breach is a detective control — it identifies that something has gone wrong. Routing to human review while investigation occurs is a corrective/compensating control. The question asks about the monitoring/flagging function itself, which detects the problem after it has occurred.

Question 3 of 10

An AI model's accuracy remains high, but the statistical distribution of input features has shifted significantly since training. Which type of drift is occurring?

A Concept drift

B Data drift (covariate drift)

C Prediction drift

D Model decay

Data drift (covariate drift) is when the input feature distributions change while the underlying relationship between inputs and outputs remains valid. The model can still perform well initially, but the risk of performance degradation grows as the gap between training and production distributions widens. Concept drift changes the input-output relationship itself.

Question 4 of 10

An auditor wants to determine which features are driving an AI model's decisions across all predictions — not just one specific decision. Which explainability tool is MOST appropriate?

A LIME — local approximation around a single prediction

B SHAP — global feature importance with local explanation capability

C Counterfactual explanations — minimum change to flip a decision

D Attention visualization — for transformer-based models only

SHAP provides both global model-level feature importance (which features matter most overall) and local per-prediction explanations. For understanding model behavior across all predictions, SHAP's global summaries (e.g., SHAP beeswarm plots, mean absolute SHAP values) are the most appropriate tool. LIME is local-only; counterfactuals are local-only.

Question 5 of 10

A "Human-on-the-loop" oversight model for AI means that:

A A human reviews and approves every AI decision before it takes effect

B The AI system operates autonomously but humans monitor outputs and can intervene

C Humans are fully removed from the decision process

D Humans only review AI decisions at the annual audit

Human-on-the-loop means the AI acts autonomously while humans monitor its outputs in real time and can intervene to override decisions. This is distinct from Human-in-the-loop (every decision reviewed before effect) and Human-out-of-the-loop (fully automated, no active monitoring). For high-stakes decisions, GDPR Article 22 may require Human-in-the-loop, not just on-the-loop.

Question 6 of 10

An attacker repeatedly queries an organization's AI credit scoring API, varying inputs systematically, to reconstruct a functionally equivalent model. This is classified as which type of adversarial attack?

A Data poisoning

B Membership inference

C Model stealing

D Evasion attack

Model stealing (also called model extraction) involves systematically querying a model API to reconstruct a functionally equivalent copy. This threatens IP protection and allows the attacker to perform offline attacks on the cloned model. Controls include rate limiting, query anomaly detection, and API authentication. Membership inference targets training data privacy, not model replication.

Question 7 of 10

During an AI audit, you find that the organization has a well-documented model validation policy requiring bias testing before deployment, but 15 of the last 30 model deployments have no bias testing documentation. This finding BEST represents:

A A design effectiveness gap — the control doesn't address the right risk

B An operating effectiveness gap — the control is well-designed but not consistently followed

C A risk appetite violation — the residual risk exceeds the stated appetite

D A third-party risk finding — vendors are not following the policy

When a control is properly designed (the policy requires bias testing) but not consistently performed (50% of deployments lack documentation), this is an operating effectiveness gap. The control design is adequate; the execution is failing. This is a common AAIA exam scenario — always distinguish between "the policy exists but isn't followed" (operating) vs. "the policy doesn't address the risk" (design).

Question 8 of 10

A fraud detection AI model has high accuracy (97%) but a recall rate of only 42% for actual fraud cases. As an AI auditor, what is your MOST significant concern?

A The model is generating too many false positives, causing customer friction

B The model is missing 58% of actual fraud cases — the false negative rate is unacceptably high

C The 97% accuracy is too high and suggests overfitting

D The model does not have sufficient explainability for regulatory purposes

Recall (sensitivity) measures what proportion of actual positives the model catches. A 42% recall means the model misses 58% of actual fraud — false negatives. For fraud detection, false negatives (missed fraud) are far more costly than false positives (false alarms). High accuracy with low recall is a classic imbalanced dataset problem — the model may predict "no fraud" almost always and still be 97% accurate if fraud is rare.

Question 9 of 10

Prompt injection attacks are a security concern primarily for which type of AI system?

A Traditional machine learning classification models

B Computer vision models processing images

C Large language models (LLMs) and generative AI systems

D Reinforcement learning agents in robotics

Prompt injection is specific to LLMs and generative AI — attackers embed malicious instructions in text inputs to override system prompts, extract private data, or cause the model to produce prohibited content. This attack type does not apply to traditional ML classifiers or vision models, which don't process free-text instructions as executable commands.

Question 10 of 10

To prevent an attacker from using repeated API queries to extract private information about individuals in the training data (membership inference attack), which control is MOST effective?

A Rate limiting and output perturbation to reduce information leakage

B Encrypting the model weights at rest

C Requiring multi-factor authentication to access the API

D Adding a network firewall in front of the API

Membership inference exploits the model's output confidence values to infer whether a specific record was in training data. Rate limiting slows the attack; output perturbation (adding noise or rounding confidence scores) reduces the signal the attacker receives. Differential privacy in training is the most rigorous defense. Encryption, MFA, and firewalls address access control but don't address the information leakage through the model's outputs themselves.

Practice Score — Keep studying with FlashGenius!

Memory Hooks

High-yield mnemonics and patterns to lock in AI Controls, Monitoring & Incidents for the AAIA.

🏛️

Three Lines of Defense — AI Roles

Line 1 = Do (AI teams own and operate). Line 2 = Oversee (Risk/governance sets policy and monitors). Line 3 = Assure (Internal audit independently tests). Key: Line 2 does NOT own AI systems — it oversees the first line's management of them.

Mnemonic: "Do, Oversee, Assure" — 1st does it, 2nd watches it, 3rd audits it. Never confuse lines — the auditor is always Line 3.

🔍

Control Types — PDCC

Preventive (stop it), Detective (find it), Corrective (fix it), Compensating (alternative when primary fails). Design effectiveness = right risk covered. Operating effectiveness = actually working. Both must be tested.

Mnemonic: "Please Don't Create Chaos" — Preventive, Detective, Corrective, Compensating

📉

Three Drift Types

Data drift = inputs shift (PSI detects it). Concept drift = the world's logic changes (hardest to detect — needs ground truth labels). Prediction drift = outputs shift (easiest to detect — no ground truth needed). Concept drift is the most dangerous.

Mnemonic: "Data drifts first, Concepts corrupt, Predictions flag it" — D → C → P from hardest to detect to easiest

⚖️

Precision vs. Recall

Precision = when you say YES, how often are you right? (TP/TP+FP). Recall = of all the actual YES's, how many did you catch? (TP/TP+FN). High recall = few misses. For fraud/disease — recall matters more (missing = bad). For spam — precision matters more (false alarms = annoying).

Mnemonic: "Recall = you Remembered the positives; Precision = your predictions were precise"

🛡️

6 Adversarial Attack Types

Data Poisoning, Evasion, Model Inversion, Prompt Injection (LLM-specific), Stealing, Membership Inference. Prompt injection is the newest — specific to generative AI. Model inversion + membership inference = privacy attacks.

Mnemonic: "Devious Enemies Make People Suffer Much" — Data, Evasion, Model Inversion, Prompt, Steal, Membership

🚨

LIME vs. SHAP

LIME = Local only (one prediction at a time; faster, less consistent). SHAP = Global + Local (model-level AND per-prediction; Shapley values; more rigorous). For bias audits across the whole model → SHAP. For explaining a single decision to a customer → either, but counterfactuals are most interpretable.

Mnemonic: "SHAP Sees All, LIME Looks Local" — SHAP for global model analysis; LIME for quick local explanations

High-Yield AAIA Facts — Controls, Monitoring & Incidents

Fact	Answer
The line of defense that provides independent AI audit assurance	Third line (Internal Audit)
Control that stops the risk before it occurs	Preventive control
Hardest drift to detect — requires ground truth labels	Concept drift (label drift)
PSI threshold indicating significant drift requiring investigation	PSI > 0.2 (significant); 0.1–0.2 slight shift; <0.1 stable
SHAP is based on which mathematical concept?	Shapley values from cooperative game theory
Adversarial attack that targets LLMs specifically	Prompt injection
Primary defense against membership inference attacks	Differential privacy in training; rate limiting; output perturbation
Design effectiveness vs. operating effectiveness	Design = control addresses the right risk; Operating = control is actually performed as designed
High override rate on an AI system signals what?	Potential model degradation, poor calibration, or inadequate model performance
First step in AI incident response	Detection & Alerting — identify, classify severity, open ticket

Flashcards & Study Advisor

Click any card to flip it. Use the Study Advisor for targeted guidance by topic area.

AI Controls

In the three lines of defense, which line owns AI systems, and which provides independent audit assurance?

Click to flip

Answer

First line (AI development/business units) owns and operates AI systems. Third line (Internal Audit) provides independent assurance. Second line provides oversight and policy but does NOT own systems or run audits.

AI Controls

What is the difference between design effectiveness and operating effectiveness of an AI control?

Click to flip

Answer

Design effectiveness: Is the control well-designed to address the targeted risk? (Walkthrough, inspection). Operating effectiveness: Is the control actually being performed as designed, consistently over time? (Sampling, re-performance). Both must be tested in an AI audit.

AI Monitoring

What are the three types of AI drift, and which is hardest to detect?

Click to flip

Answer

Data drift: Input distributions shift (detected by PSI/KS tests). Concept drift: Input-output relationship changes — hardest, requires ground truth labels. Prediction drift: Output distributions shift — easiest, no labels needed. Concept drift is most dangerous.

AI Monitoring

What is PSI, and what value signals significant drift requiring investigation?

Click to flip

Answer

Population Stability Index (PSI) measures the shift in score distributions between training and production. PSI < 0.1 = stable; 0.1–0.2 = slight shift; > 0.2 = significant drift — investigate and consider retraining. Threshold-based monitoring trigger.

AI Explainability

What is the key difference between LIME and SHAP, and when should an auditor use each?

Click to flip

Answer

LIME = local only (explains one prediction; faster but less consistent). SHAP = global + local (model-wide feature importance + per-prediction explanations; based on Shapley values). Use SHAP for bias audits across the whole model; LIME for quick local explanations of individual decisions.

AI Incidents

What are the 6 steps of the AI incident response lifecycle?

Click to flip

Answer

① Detection & Alerting ② Containment (limit harm — rollback, suspend, route to humans) ③ Investigation & Root Cause Analysis ④ Remediation (retrain, fix data, patch) ⑤ Recovery & Validation ⑥ Post-Incident Review & Reporting.

AI Security

What is prompt injection, and which AI systems are vulnerable to it?

Click to flip

Answer

Prompt injection = attacker embeds malicious instructions in user input to hijack an LLM's behavior — overriding system prompts, extracting private data, or generating prohibited content. Specific to generative AI and LLM systems. Controls: input sanitization, output filtering, privilege separation between system and user prompts.

AI Monitoring

A fraud detection model has 97% accuracy but 42% recall. What is the core problem and why?

Click to flip

Answer

The model is missing 58% of actual fraud cases (false negatives). High accuracy on an imbalanced dataset is misleading — predicting "no fraud" almost always yields high accuracy when fraud is rare. Recall (sensitivity) = TP/(TP+FN) is the critical metric for fraud detection because false negatives (missed fraud) are far more costly than false positives.

Ready for the Full AAIA Practice Tests?

Access hundreds of AI controls, monitoring, and incident flashcards and practice questions on FlashGenius.

Unlock Full Practice Tests on FlashGenius →

Study Advisor

Three Lines of Defense

Control Design & Testing

Monitoring & Drift

Incidents & Attacks

Exam Strategy

Three Lines of Defense Tips

Line 2 ≠ Line 3: The risk management function (Line 2) sets AI policy and monitors the first line — it does NOT conduct independent audits. Internal audit (Line 3) is independent and reports to the board/audit committee.
AI governance committee = Line 2: Chief AI Officers, AI risk managers, and compliance functions all sit in Line 2. They own the framework but not the AI systems.
Separation is essential: If AI developers are also validating their own models with no independent review, the three-lines model has broken down. This is a governance finding.
Exam trap: Questions may describe a scenario where "the data science team validates its own models." This is a Line 1 control gap — validation should have Line 2 or independent oversight.
External audit = 4th line: Some organizations recognize external auditors as a 4th line. Know this if the exam asks about additional assurance beyond internal audit.

Control Design & Testing Tips

Design before operating: Always assess design effectiveness first. If a control is poorly designed, testing operating effectiveness is irrelevant — it won't address the risk regardless of how well it's followed.
Evidence is everything: A control that cannot be evidenced is treated as non-operating. If bias testing is required but there are no test reports, the control is not operating effectively regardless of what staff claim.
HITL patterns on the exam: "Human-in-the-loop" = reviews every decision. "Human-on-the-loop" = monitors but AI acts autonomously. "Human-out-of-the-loop" = fully automated. High-stakes decisions (credit, employment) likely require HITL per GDPR Art. 22.
Model change management: Automated retraining pipelines without a formal approval gate are a common gap. Even minor threshold changes should go through change management for high-risk AI systems.
Compensating controls: When a primary control fails (e.g., model is a black box with no explainability), compensating controls (100% human review of high-confidence decisions) reduce residual risk. Document why the compensating control is equivalent.

Monitoring & Drift Tips

Concept drift is the exam favorite: It's the trickiest — the model's learned mapping becomes invalid because the real-world relationship has changed. No code changes, no data pipeline failures — the world just changed. Requires labeled feedback to detect.
PSI thresholds to memorize: <0.1 = stable, 0.1–0.2 = slight shift (investigate), >0.2 = significant drift (retrain). These exact thresholds appear in exam questions.
High override rate is a red flag: If humans are frequently overriding AI decisions, this may indicate model degradation, poor calibration, or a mismatch between model scope and actual use case. Track and investigate override patterns.
Audit logs must be immutable: For AI decision audit trails, logs must be tamper-proof, retained for the required period, and contain enough information to reconstruct any decision (input, output, model version, timestamp).
SHAP for bias audits: When the question involves understanding which features drive outcomes across all predictions (not just one), SHAP's global summaries are the right tool. Look for protected-attribute proxies with high SHAP values.

Incidents & Attacks Tips

Containment before root cause: In AI incident response, containment (stop the bleeding) always comes before deep investigation. A common exam trap presents options where investigation is listed before containment.
Prompt injection = LLM-specific: This attack only applies to systems that take free-text instructions as inputs. Traditional classifiers cannot be prompt-injected. Know which attack types apply to which model architectures.
Privacy attacks: Membership inference and model inversion both target training data privacy. Differential privacy is the most rigorous technical defense for both.
Data poisoning in continuous training: Models that retrain automatically on production data (common in recommendation systems) are especially vulnerable to poisoning attacks — an adversary can inject data into the production stream to corrupt future model versions.
Post-incident review is required: A complete incident response includes a post-mortem. Questions asking "what is the LAST step" in incident response should be answered with post-incident review and lessons learned — not recovery.

AAIA Exam Strategy

Domain 2 is 46% of the exam: The combined controls/monitoring/operations domain is the single largest portion. Prioritize controls and monitoring material heavily in your prep.
Auditor's perspective always: AAIA questions ask what the auditor should do, assess, or find — not what the AI team should build. "The auditor should recommend retraining" is usually wrong; "the auditor should assess whether the retraining process has appropriate controls" is right.
Detective controls dominate AI: Because AI can degrade silently, detective controls (monitoring, drift detection, anomaly alerts) are the dominant control type for AI systems. Most AAIA control questions will relate to monitoring and detection.
Risk-based approach: Not all AI systems need the same level of controls. High-risk AI (EU AI Act) requires more rigorous controls than minimal-risk AI. Always factor in risk level when assessing whether controls are appropriate.
Recall for high-stakes classifiers: In any scenario about fraud detection, medical diagnosis, or safety-critical classification, recall (catching all positives) is the priority metric. False negatives are almost always the larger risk in these domains.