ISC2 Building AI Strategy Certificate - AI Security: Managing Overconfidence (10 Practice Questions)
The ISC2 Building AI Strategy Certificate program is a certificate earned by completing six on-demand courses (16 hours). Below are some sample questions created based on the AI Security: Managing Overconfidence module objectives.
AI Security: Managing Overconfidence — 10 Practice Questions
Goal: ~20 minutes. Read the scenario, pick an answer, then expand to check the key and rationale.
Q1. Hot but Wrong: Calibrating a Triage Model
Your LLM-based triage tool assigns 0.98 probability to “benign” on several alerts that analysts later flag as true positives. You need a quick production fix without a full retrain. What’s the best next step?
- Lower the decoding temperature to 0
- Retrain from scratch with a larger dataset
- Apply post-hoc temperature scaling on a held-out set and enable ECE monitoring
- Add more stop-words to the prompt
Show answer & rationale
Answer: C — Post-hoc calibration (e.g., temperature scaling) adjusts confidence without changing classifications; monitor with Expected Calibration Error (ECE).
Q2. Safety Gate for High-Stakes Actions
An “IR copilot” can auto-close low-severity tickets. To reduce overconfident mistakes, what control should you implement first?
- Keep the current 0.90 threshold—most predictions are confident
- Route predictions below a calibrated threshold to a human (abstain/deferral policy)
- Add extra adjectives to the system prompt
- Increase top-p for more diverse outputs
Show answer & rationale
Answer: B — Selective prediction/deferral gates risky, low-confidence cases to humans and directly counters harmful overconfidence.
Q3. Overconfidence on Unknowns (OOD)
Your classifier is highly confident on log lines from a brand-new product the model has never seen. What reduces overconfident errors on out-of-distribution data?
- Penalize low-probability tokens during decoding
- Add an embedding-based OOD detector with reject/route behavior
- Increase model size to memorize more patterns
- Drop all unknown fields from inputs
Show answer & rationale
Answer: B — Use OOD detection (e.g., distance/likelihood tests) to flag “unknowns” and abstain or route to human review.
Q4. Which Metric Captures Miscalibration?
You need a metric that quantifies the gap between predicted probabilities and observed outcomes. Which do you choose?
- Accuracy
- ROC-AUC
- Expected Calibration Error (ECE)
- BLEU
Show answer & rationale
Answer: C — ECE (and Brier score) are purpose-built for calibration; accuracy/AUC don’t measure confidence alignment.
Q5. “No Source, No Answer”
Your summarization assistant occasionally fabricates citations with high confidence. What’s the most effective policy update?
- Tell the model “don’t hallucinate”
- Ground answers via retrieval and require verifiable citations (refuse if none)
- Set temperature to 0.0
- Switch to top-k = 1
Show answer & rationale
Answer: B — Grounding + enforceable citation rules reduce confident fabrication and create auditability.
Q6. Picking a Threshold the Right Way
You must cap the automated error rate at 2% while keeping reasonable coverage. How should you set the confidence threshold?
- Default to 0.50 because it’s “standard”
- Use a validation risk-coverage curve to pick a threshold that meets the error target
- Choose the F1-maximizing threshold
- Pick whatever top analyst prefers
Show answer & rationale
Answer: B — Risk-coverage analysis tunes abstention thresholds to satisfy target risk under real data distribution.
Q7. Estimating Epistemic Uncertainty
On sparse training data, your model appears very sure but often wrong. Which technique helps quantify epistemic uncertainty?
- Monte Carlo dropout / deep ensembles
- L2 regularization only
- Label smoothing only
- Random seed changes
Show answer & rationale
Answer: A — MC dropout and ensembles approximate model uncertainty, curbing overconfidence in low-data regimes.
Q8. Verifier-Backed Responses
Your QA bot confidently states wrong CVE details. What’s the most effective change to reduce high-confidence errors?
- Increase max tokens
- Introduce a verifier/fact-checker that scores answers; answer only if the verifier exceeds a calibrated threshold
- Add a warning banner to the UI
- Replace the model with rules entirely
Show answer & rationale
Answer: B — Verifier-gated generation (self-consistency, retrieval checks, learned PoC models) reduces confident falsehoods.
Q9. UI to Counter Automation Bias
Analysts tend to accept model outputs when they “sound certain.” Which UI change best mitigates automation bias?
- Hide all probabilities
- Show raw, uncalibrated probability to be transparent
- Display calibrated confidence bands (High/Med/Low) with “verify” prompts and rationale highlights
- Add more emojis to warnings
Show answer & rationale
Answer: C — Calibrated bands + nudges (“verify when Low”) reduce over-reliance on confidently worded but uncertain outputs.
Q10. When the World Shifts
A major product release changes log schemas and behaviors. How should your overconfidence controls respond?
- Monitor ECE once a month; otherwise ignore
- Detect data/feature drift; tighten thresholds (lower coverage), raise deferrals, and trigger recalibration/retraining
- Add synonyms to prompts
- Do nothing—confidence is fine if accuracy is stable
Show answer & rationale
Answer: B — Shift-aware playbooks should automatically reduce automation, increase human review, and kick off calibration/refresh.
ISC2 Building AI Strategy Certificate: Everything You Need to Know
Understand how the six-course, ~16-hour certificate works, who it’s for, what each module covers, plus a study plan and FAQs.
- Module breakdown + sample questions
- Study plan, resources, and tips
- How it differs from proctored exams
Not affiliated with (ISC)².