Free AAIA AI Auditing Tools and Techniques Practice Questions

This 10-question domain test represents 21% of the ISACA AAIA exam and covers audit planning, evidence, testing approaches, AI-assisted audit tools, reporting, and follow-up. Work through each question, then review the explanation to identify the audit principle or control behind the answer.

Return to the AAIA practice-test hub or try the 10-question mixed test.

10 Sample Questions with Answers

Sample Question 1 — AI Auditing Tools and Techniques

An audit team uses an explainability tool to support testing of a customer eligibility model. The tool produces explanations based on a surrogate model, and the explanations vary materially across repeated runs. The business has used the explanations to justify model decisions to compliance reviewers. Which recommendation is MOST appropriate?

  1. A. Define acceptable use limits and require corroboration before using explanations as audit evidence. (Correct answer)
  2. B. Increase the number of sampled decisions so unstable explanations average out over the audit period.
  3. C. Replace the explainability tool with a visualization dashboard approved by the business owner.
  4. D. Report that model decisions are unsupported because surrogate explanations are inherently invalid.

Correct answer: A

Explanation: A is the best recommendation because the issue is evidence reliability, not merely sample size or presentation format. Materially different outputs across repeated runs indicate the tool should not be relied on without defined usage boundaries and corroborating evidence. B is insufficient because a larger sample does not fix instability in the explanations themselves. C addresses presentation and business approval, but neither establishes that the explanations are reliable for audit or compliance use. D is too absolute because surrogate explanations are not automatically unusable; their limitations must be governed and supplemented with other evidence.

Sample Question 2 — AI Auditing Tools and Techniques

An organization maintains an AI model inventory used to scope AI audits. Business owners annually certify that their inventory entries are complete. The auditor is concerned that models deployed through cloud machine learning services may be omitted. Which audit procedure is MOST appropriate?

  1. A. Reconcile cloud deployment records to the model inventory and investigate unmatched services. (Correct answer)
  2. B. Review annual business owner certifications for completeness statements and sign-off dates.
  3. C. Compare the inventory taxonomy to AI definitions used in the enterprise risk policy.
  4. D. Interview model owners about whether they know of unregistered cloud-based models.

Correct answer: A

Explanation: A is best because it uses independent system evidence to test the completeness assertion directly. Reconciling actual cloud deployment records to the model inventory is the strongest way to identify omitted models and investigate exceptions. B and D rely on management or owner assertions, which are weaker when the risk is that deployed models were not reported. C may help assess definitional consistency, but it does not determine whether all deployed models are actually captured in the inventory.

Sample Question 3 — AI Auditing Tools and Techniques

An audit analytics script using machine learning is reused across multiple regulatory audits to identify unusual transactions. The script was modified during the current audit, and the results were consistent with prior-period findings, but there is no evidence of independent review of the modified script. Which recommendation is MOST appropriate?

  1. A. Require version control and independent review before relying on modified audit analytics scripts. (Correct answer)
  2. B. Accept the current results because they are consistent with prior-period audit findings.
  3. C. Limit future use of machine learning scripts to nonregulatory audit engagements only.
  4. D. Request management to confirm that the unusual transaction population is complete.

Correct answer: A

Explanation: A is the most appropriate recommendation because once an audit analytics script is modified, the reliability of the audit evidence depends on controlled change management, including versioning and independent review, before the results are relied upon. B is incorrect because similar results do not validate that the modified logic operated as intended. C is disproportionate and does not address the actual deficiency, which is lack of governance over script changes. D may relate to population completeness, but it does not mitigate the risk that the modified audit tool itself produced unreliable results.

Sample Question 4 — AI Auditing Tools and Techniques

A global manufacturing company is using a vendor-supplied AI anomaly detection module to identify unusual invoices for an accounts payable audit. The vendor reports high detection accuracy, and the audit team has screenshots of flagged exceptions. However, there is no documented reconciliation between the ERP transaction population and the file processed by the tool. Management wants to reduce manual sample sizes based on the AI results. What should the auditor evaluate FIRST?

  1. A. Whether the ERP population was completely and accurately processed by the AI tool (Correct answer)
  2. B. Whether the vendor accuracy claims are consistent with the exceptions identified
  3. C. Whether the audit team reviewed enough screenshots of the flagged invoices
  4. D. Whether management has accepted the residual risk from using the tool

Correct answer: A

Explanation: A is best because the auditor cannot rely on AI-generated exceptions until the completeness and accuracy of the population processed by the tool are established. Without reconciliation from the ERP source population to the input file, the auditor cannot know whether relevant transactions were omitted or altered. B may matter later, but a tool can perform accurately on an incomplete population and still produce unreliable audit evidence. C addresses only visibility of outputs, not whether the tool analyzed the full population. D is premature because risk acceptance does not substitute for validating the evidence base used to support audit conclusions.

Sample Question 5 — AI Auditing Tools and Techniques

A retail bank uses a monitoring dashboard for a production card fraud model. During the quarter, the dashboard showed several amber and red drift alerts. Management states the alerts were reviewed informally, monthly performance reports were produced, and false positive rates improved overall. Ticketing evidence is incomplete for some alerts. Which evidence BEST supports the conclusion that the monitoring control operated effectively?

  1. A. Dashboard screenshots showing alert status and quarterly model performance trends
  2. B. System alert logs linked to review, escalation, and resolution records (Correct answer)
  3. C. Management meeting notes describing informal review of drift patterns
  4. D. Aggregate fraud loss metrics showing improvement during the quarter

Correct answer: B

Explanation: B is best because operating effectiveness requires traceable evidence that alerts were generated, reviewed, escalated, and resolved in accordance with the control process. Linked alert logs and resolution records show end-to-end control execution. A shows that monitoring information existed, but not that anyone acted on it. C is weaker because informal review notes do not reliably demonstrate timely escalation or closure of specific alerts. D is outcome evidence only; improved results may occur for many reasons and do not prove the monitoring control operated as designed.

Sample Question 6 — AI Auditing Tools and Techniques

An insurance company relies on a quarterly external AI risk scan for a vendor-hosted generative AI customer service platform. The report rates the vendor environment as low risk, but it excludes customer-specific integrations. The company uses proprietary claims data with the platform, and the vendor initially provides only the summary report. Which risk is MOST relevant?

  1. A. Management may rely on assurance that does not cover the deployed use case (Correct answer)
  2. B. The external assessor may not have used the latest prompt testing methods
  3. C. The vendor may not provide raw findings within the requested review period
  4. D. Quarterly scanning may not align with internal audit reporting timelines

Correct answer: A

Explanation: A is best because the core audit risk is scope misalignment: the favorable external scan excludes customer-specific integrations even though the organization uses proprietary claims data in its actual deployment. That means the report may not provide assurance over the enterprise's real risk exposure. B could matter, but methodology currency is secondary to whether the assessment covered the relevant environment at all. C is a valid transparency concern, yet lack of raw findings is less fundamental than the report's failure to assess the deployed use case. D affects scheduling, not the sufficiency or relevance of the assurance being relied upon.

Sample Question 7 — AI Auditing Tools and Techniques

A global enterprise is beginning an audit of AI governance. Management provides an official AI inventory supported by business-unit attestations, but a discovery tool identifies additional AI-enabled SaaS subscriptions in cloud billing and expense records. Some business units state these tools are only pilots and should not be included. What should the auditor evaluate FIRST?

  1. A. Whether discovery results are reconciled to procurement, CMDB, and model registry records. (Correct answer)
  2. B. Whether business-unit attestations classify pilot AI tools consistently with policy.
  3. C. Whether model owners completed risk ratings for systems already in inventory.
  4. D. Whether the inventory dashboard summarizes AI use by application and business unit.

Correct answer: A

Explanation: A is best because the auditor must first establish whether the AI population is complete before relying on downstream governance or control testing. Reconciling discovery output to authoritative sources is stronger evidence of completeness than business-unit attestation and directly addresses shadow AI risk. B is relevant but depends on management classification after the population has been identified. C is a downstream step that applies only to systems already known to be in scope. D may support reporting, but a dashboard summary does not demonstrate that omitted systems were identified and resolved.

Sample Question 8 — AI Auditing Tools and Techniques

A financial services company uses a vendor platform to monitor fairness and explainability metrics for a credit decisioning model. The vendor dashboard shows no fairness alerts during the quarter. However, a production hotfix occurred after the last approved model review, and the model owner changed alert thresholds without documented approval. What is the PRIMARY audit concern?

  1. A. Dashboard metrics may not correspond to the deployed model and approved thresholds. (Correct answer)
  2. B. Vendor-produced reports may not provide enough detail for management review.
  3. C. Model-owner review of exceptions may reduce independence in issue closure.
  4. D. Stable fairness trends may not reflect future performance under changed conditions.

Correct answer: A

Explanation: A is best because the hotfix and unapproved threshold changes undermine the reliability of the monitoring evidence itself. If the dashboard is not traceable to the actual production version and governed alert criteria, favorable metrics cannot support an audit conclusion. B is a secondary evidence-quality issue, but lack of detail is less fundamental than uncertainty about what was actually monitored. C is a plausible governance concern, yet independence in issue closure matters only after the auditor can rely on the monitoring output. D addresses possible future drift, not the immediate problem that current dashboard results may not be tied to the approved model and thresholds.

Sample Question 9 — AI Auditing Tools and Techniques

An internal audit team used an AI-assisted document review tool to classify thousands of vendor contracts for third-party AI risk. The tool reported a high classification accuracy during a pilot, but some contracts were excluded because of unreadable scans and unsupported file types. Management wants the audit team to use the tool output to define the high-risk vendor population. What should the auditor evaluate FIRST?

  1. A. Whether processed contracts reconcile to the repository and exclusions were resolved (Correct answer)
  2. B. Whether the pilot accuracy rate was approved by the audit methodology owner
  3. C. Whether reviewers documented their conclusions for a sample of high-risk contracts
  4. D. Whether the tool configuration changes were reviewed before fieldwork began

Correct answer: A

Explanation: A is best because the auditor must first determine whether the AI-assisted output represents a complete and reliable population before using it for scoping or sample selection. Reconciling processed contracts to the source repository and resolving exclusions addresses the primary risk that high-risk vendors were omitted. B is relevant to tool governance, C to review documentation, and D to change control, but none of those directly establish population completeness for audit reliance.

Sample Question 10 — AI Auditing Tools and Techniques

A bank uses a third-party explainability dashboard for an AI credit decision model. Management asserts the dashboard supports transparency controls for adverse action reviews. The dashboard provides monthly global feature importance, while case-level explanations are retained only for selected decisions. The vendor documentation warns that global importance should not be used to explain individual outcomes. What is the PRIMARY audit concern?

  1. A. Vendor limitations were not incorporated into the transparency control assessment
  2. B. Global outputs are relied on for decision-level transparency evidence (Correct answer)
  3. C. Case-level explanation records are retained for only selected decisions
  4. D. Manual adverse action narratives are used when explanations are unavailable

Correct answer: B

Explanation: B is best because the core design issue is that the evidence being relied on does not fit the control objective. A control intended to support transparency for individual credit decisions cannot be supported primarily by global feature importance, especially when vendor documentation states that such output should not be used to explain single outcomes. A is a related governance weakness, C is an important evidence-retention gap, and D may indicate compensating manual effort, but each is secondary to the fundamental mismatch between global explainability output and decision-level transparency requirements.

AAIA Practice Test FAQs

What is the ISACA Advanced in AI Audit (AAIA) exam?

AAIA is an ISACA certification exam focused on auditing artificial intelligence systems. It covers AI governance and risk, AI operations, and AI auditing tools and techniques.

How many questions and how much time does the AAIA exam have?

The AAIA exam configuration is 90 questions in 150 minutes across three domains.

What score is required for AAIA?

ISACA uses a scaled score, with 450 on an 800-point scale listed as the passing score in the exam configuration. Practice-test percentages are not equivalent to an official ISACA scaled score.

How should I use AAIA practice questions?

Use mixed questions to find broad knowledge gaps, then use domain practice to review the concepts and explanations behind incorrect answers. FlashGenius readiness thresholds are study guidance, not official ISACA passing scores.

Which AAIA domain has the greatest weight?

AI Operations has the largest listed weight at 46%, followed by AI Governance and Risk at 33% and AI Auditing Tools and Techniques at 21%.

Explore AAIA Tests