AAIA Practice Questions: AI Auditing Tools and Techniques Domain

A is the best recommendation because the issue is evidence reliability, not merely sample size or presentation format. Materially different outputs across repeated runs indicate the tool should not be relied on without defined usage boundaries and corroborating evidence. B is insufficient because a larger sample does not fix instability in the explanations themselves. C addresses presentation and business approval, but neither establishes that the explanations are reliable for audit or compliance use. D is too absolute because surrogate explanations are not automatically unusable; their limitations must be governed and supplemented with other evidence.

A is best because the auditor should first establish that the retrained tool is complete and reliable for its intended audit use before relying on its rankings for scoping. That requires evidence such as population reconciliation and validation after the material change. B focuses on outcomes like efficiency and yield, which do not prove the tool is fit for audit reliance. C relies on weaker evidence because informal approval and attestation do not replace validation. D may be useful as a secondary compensating review, but it does not address the foundational question of whether the scoring itself is trustworthy.

A is the best recommendation because the core audit issue is unsupported reliance on AI-generated output. Before the audit function uses those outputs as evidence or risk indicators, it should establish documented validation, permitted use, and reliance criteria. B may reduce exposure, but it does not address the absence of validation or define when outputs are trustworthy enough to support audit conclusions. C could provide supplemental information, yet vendor claims are not a substitute for validation in the audit function's own use context. D is also plausible because human review can mitigate risk, but spot-checking alone is insufficient when it is informal and inconsistently documented; it does not replace a governed validation framework.

The central issue is whether the AI tool is governed well enough for its outputs to be relied on as audit evidence. Without documented validation, controlled changes, and consistent quality review, audit reliance is not adequately supported. A and D are credible concerns, but they are narrower consequences of the broader reliance problem. B addresses performance management of the audit function, not the reliability of audit evidence.

A is best because explainability artifacts and stable aggregate accuracy are not sufficient evidence that the model is fair. The key audit issue is that management lacks approved fairness criteria and retained subgroup testing for the current version, so the fairness conclusion is not supported by appropriate evidence. B is a narrower symptom that would help analysis but does not resolve the missing criteria and retained evidence. C is a governance weakness, but it is not the primary evidentiary gap. D increases risk context, yet comparing versions does not substitute for current-version fairness thresholds and subgroup results.

A is best because the auditor cannot rely on AI-generated exceptions until the completeness and accuracy of the population processed by the tool are established. Without reconciliation from the ERP source population to the input file, the auditor cannot know whether relevant transactions were omitted or altered. B may matter later, but a tool can perform accurately on an incomplete population and still produce unreliable audit evidence. C addresses only visibility of outputs, not whether the tool analyzed the full population. D is premature because risk acceptance does not substitute for validating the evidence base used to support audit conclusions.

B is correct because the auditor must first determine whether the AI population is complete before relying on any downstream governance testing. Reconciling discovery output to independent sources such as procurement and cloud usage records is the strongest way to identify shadow or externally procured AI that may be outside registered tenants. A is weaker because attestations are self-reported and do not independently prove completeness. C is not sufficient because the absence of incidents does not show that unrecorded AI is not in use. D is relevant, but ownership accuracy is a secondary attribute once the completeness of the inventory has been established.

C is the best answer because the facts show the monitoring tool generated alerts, but required follow-up and escalation did not occur within target time frames. That indicates an operating-effectiveness failure in exception handling, even though thresholds exist and aggregate accuracy remains stable. A and D are plausible design questions, but the stronger issue supported by the evidence is failure to act on generated alerts. B introduces a subgroup-performance concern that is not indicated by the scenario.

A is best because it uses independent system evidence to test the completeness assertion directly. Reconciling actual cloud deployment records to the model inventory is the strongest way to identify omitted models and investigate exceptions. B and D rely on management or owner assertions, which are weaker when the risk is that deployed models were not reported. C may help assess definitional consistency, but it does not determine whether all deployed models are actually captured in the inventory.

B is the best answer because it provides traceable evidence that summarized dashboard metrics are complete and accurate relative to underlying system-generated events. That directly supports both evidence reliability and the auditor's conclusion on monitoring control operation. A is relevant but remains secondary, summarized evidence prepared for reporting and does not by itself prove traceability to source events. C is plausible supporting evidence, but an attestation is weaker than independently generated records and does not verify that alerts actually occurred and were handled as stated. D may indicate acceptable business outcomes, but outcome data does not demonstrate that the monitoring control itself operated effectively or that dashboard reporting was accurate.