AAIA Practice Questions: AI Operations Domain

A is correct because the audit issue is whether the corrective control operated effectively during the incident. Comparing system-generated alert, escalation, and rollback timing to predefined severity criteria is the strongest way to test timely containment and authority execution. B is wrong because a post-incident review is retrospective and does not prove the response was timely. C is wrong because stakeholder communication may be relevant, but it does not address whether harmful scoring continued when it should have been stopped. D is wrong because the playbook supports design evaluation, while the scenario requires testing actual operation during the incident.

A is the best answer because rising override rates without a formal escalation trigger indicates a monitoring design weakness: the control may not prompt timely review before business impact accumulates. B is a secondary usability concern, not the main control issue. C points to one metric that may be less informative, but the primary deficiency is that materially concerning indicators do not trigger action. D describes use of a lagging indicator in discussion, which does not address whether risk-based thresholds and escalation requirements exist.

B is best because a material AI failure that allowed fraud to bypass review should be formally classified in a way that triggers appropriate AI governance escalation, investigation, and corrective action tracking. Quick restoration does not remove that governance need. A is not the primary issue because containment can reasonably occur before final loss measurement. C may be a follow-up validation concern, but it is secondary to the incident-governance gap. D describes a positive recovery step and does not address whether the event received appropriate AI-specific oversight.

A is correct because it combines current independent assurance over vendor controls with evidence that the enterprise evaluates the operational impact of vendor changes. That best supports retained accountability. B addresses availability and vendor communications, but not whether controls over model changes actually operate effectively. C reflects initial and periodic vendor governance, not ongoing operational oversight. D may show business effects, but it does not provide assurance over vendor control operation or change management.

A is best because the key operational risk is silent omission of source records after the upstream change, and source-to-score reconciliation is the control that would detect that completeness failure. B is wrong because acceptable conversion rates for scored customers can mask missing customers who were never scored. C is wrong because lineage documentation explains dependencies but does not detect excluded records in daily operation. D is wrong because support-team identification affects accountability, not whether the scored population is complete.

A is best because successful restoration of the prior approved model from retained artifacts is the strongest evidence that rollback can be executed in practice. B is wrong because documented rollback steps show intent, not tested capability. C is wrong because management belief and source-file availability do not prove recoverability within required timeframes. D is wrong because current monitoring status does not demonstrate that rollback would work if triggered.

A is the best answer because regular retraining can materially change model behavior, and treating it as routine operations without defined revalidation and approval requirements creates the main lifecycle governance risk. B is a secondary monitoring concern, not the core control weakness. C raises a related tooling issue, but the more significant risk is uncontrolled model change entering production. D is a resource consideration rather than the primary audit concern.

A is best because reconciled workflow logs tied to actual loan records provide direct, transaction-level evidence that the required review occurred before the adverse decision was finalized. B is weaker because attestations are self-reported and not independent evidence of execution for each required case. C supports readiness and awareness, not operating effectiveness. D is an outcome measure; overturn rates may be informative, but they do not prove the required pre-decision human review control actually operated.

A is best because the key symptom is changed alert behavior after migration. The auditor should first determine whether postmigration validation was designed to detect changes in model outputs and fraud detection impact, rather than relying on infrastructure control success alone. B and C are relevant migration controls, but neither addresses whether the migrated model is producing appropriate operational results. D may affect user interpretation of alerts, yet it is secondary to confirming whether the model's behavior changed.

A is best because the audit issue is whether the monitoring control operated when approved thresholds were breached. Repeated alerts with no incident tickets suggest the escalation process may not be functioning. B is weaker because stable business KPIs do not prove the monitoring control was effective. C may become relevant later, but the auditor should first seek documented escalation, disposition, or approved exception handling rather than rely on an informal explanation. D addresses dashboard timeliness, while the main concern is the lack of response to threshold breaches.