FlashGenius Logo FlashGenius
SecAI+ CY0-001 ยท Domain 2 of 4

Securing AI Systems
The biggest domain โ€” 40%

Threat-modeling frameworks, model & gateway controls, access controls, data security, monitoring & auditing, and the full catalog of AI-specific attacks with their compensating controls.

40%
Exam Weight
6
Key Concept Areas
15
Flashcards
10
Quiz Questions
๐Ÿ  Hub 1 ยท Basic Concepts 2 ยท Securing AI Systems 3 ยท AI-Assisted Security 4 ยท AI GRC

What Domain 2 Covers

Domain 2 is the heart of SecAI+ at 40% of the exam. It covers how to design and enforce controls that protect AI models, data, agents, and integrations โ€” and the full range of AI-specific attacks you're expected to recognize and counter.

Objective A

Implement Security Controls

Protect AI systems, data, and models using technical safeguards โ€” model controls, gateway controls, and access controls.

Objective B

Secure Deployment Environments

Apply best practices across on-premises, cloud, and hybrid AI infrastructure, with monitoring and auditing built in.

Objective C

Mitigate Adversarial Risks

Defend against attacks targeting AI models, data pipelines, and inference layers using compensating controls.

MITRE ATLAS OWASP LLM Top 10 Guardrails & Prompt Firewalls Least Privilege Encryption in Use Monitoring & Auditing Prompt Injection Model Inversion Excessive Agency
๐Ÿ’ก
How to use this page: Because this domain is worth 40%, give it the most study time. Work through all six Key Concepts cards โ€” especially the attacks & compensating controls table โ€” drill the flashcards, then take the 10-question Knowledge Check.
๐Ÿ”—
Connect the dots: Every attack in this domain has at least one compensating control. The exam loves "given this attack, which control stops it?" questions โ€” study them as pairs, not separate lists.

Key Concept Areas

Click each card to expand the explanation and study tip.

1. Threat-Modeling Resources & Frameworks โ–พ
  • OWASP Top 10 for LLM Applications โ€” catalogs the most critical risks specific to LLM-based apps (prompt injection, training data poisoning, supply chain, etc.).
  • OWASP Machine Learning Security Top 10 โ€” a broader ML risk catalog covering the entire pipeline (data, model, infrastructure).
  • MIT AI Risk Repository โ€” a comprehensive, categorized database of AI risks drawn from academic and industry sources.
  • MITRE ATLAS โ€” an adversarial threat-modeling framework (modeled on MITRE ATT&CK) cataloging real-world tactics and techniques used against AI systems.
  • CVE AI Working Group โ€” extends standard vulnerability tracking and disclosure practices to AI-specific vulnerabilities.
๐Ÿ’ก
Study tip: Know which framework to reach for: ATLAS = adversary tactics/techniques, OWASP LLM/ML Top 10 = app/pipeline risk catalogs, MIT AI Risk Repository = broad risk taxonomy, CVE AI WG = vulnerability disclosure.
2. Model & Gateway Security Controls โ–พ
Model Controls
  • Evaluation โ€” testing model behavior/outputs before and after deployment.
  • Guardrails โ€” rules that constrain model inputs/outputs.
  • Prompt templates โ€” constrain and standardize input structure.
Gateway Controls
  • Prompt firewalls โ€” inspect and filter prompts before they reach the model.
  • Rate & token limits โ€” cap usage to prevent abuse and cost overruns.
  • Input quotas by size/quantity and modality limits (restrict input/output types โ€” text/image/audio).
  • Endpoint access controls โ€” restrict which systems/users can reach the model API.
Validation
  • Guardrail testing & validation โ€” continuously test guardrails against known jailbreak/bypass techniques.
๐Ÿ’ก
Study tip: Think "defense in depth" โ€” model-level controls + gateway-level controls + ongoing validation, layered together.
3. Access Controls for AI Systems โ–พ

Apply least privilege across four surfaces:

  • Models โ€” who can query, fine-tune, or modify a model.
  • Data โ€” who can access training and inference data.
  • Agents โ€” what actions an AI agent is authorized to take.
  • APIs & networks โ€” network segmentation and endpoint authentication.

Agents need particularly tight, scoped permissions โ€” an over-permissioned agent is the root cause of "excessive agency" (see card 6).

๐Ÿ’ก
Study tip: For any access-control question, ask: "is this the minimum access needed for this specific function?"
4. Data Security Controls โ–พ
  • Encryption in transit โ€” protects data moving between systems.
  • Encryption at rest โ€” protects stored data.
  • Encryption in use โ€” protects data while it's being actively processed; the hardest to achieve (relates to confidential computing).
  • Anonymization โ€” removing identifying information so individuals can't be re-identified.
  • Labels / classification โ€” tagging data by sensitivity (public, internal, confidential) to drive handling rules.
  • Redaction / masking โ€” hiding or replacing sensitive values (e.g., showing only the last 4 digits of an account number).
  • Minimization โ€” collecting and retaining only the data actually needed.
๐Ÿ’ก
Study tip: Map each control to a data lifecycle stage โ€” collection (minimization), storage (encryption at rest, classification), processing (encryption in use), output (redaction/masking).
5. Monitoring & Auditing โ–พ
  • Prompt/query/response monitoring โ€” logging what users ask and what the model returns.
  • Log monitoring, sanitization & protection โ€” ensure logs don't leak sensitive data and are protected from tampering.
  • Response confidence monitoring โ€” tracking how "certain" a model is, to flag low-confidence (possibly hallucinated) outputs.
  • Rate & cost monitoring โ€” tracking prompt, storage, response, and processing costs to detect abuse or runaway spend.
  • Auditing for hallucinations, accuracy, bias/fairness, and access โ€” periodic structured reviews.
๐Ÿ’ก
Study tip: Monitoring = real-time/ongoing observation. Auditing = periodic structured review. The exam tests whether you can tell these apart.
6. AI-Specific Attacks & Compensating Controls โ–พ

This is the longest list on the exam โ€” study attacks and their compensating controls as pairs.

AttackWhat It DoesCompensating Control(s)
Prompt injectionMalicious instructions embedded in input override system intentPrompt firewalls, templates, guardrails
Model / data poisoningCorrupts training data or model weightsData integrity checks, provenance tracking, access controls
JailbreakingBypasses a model's safety guardrailsGuardrail testing, layered controls
Input manipulationCrafted inputs cause unintended behaviorInput validation, rate limiting
Bias introductionDeliberately skews training data or outputsData auditing, fairness testing
Guardrail circumventionFinds gaps in safety rulesContinuous guardrail validation
Integration abuseExploits how AI connects to other systems/plug-insLeast privilege, scoped agent permissions
Model inversion / theftExtracts training data or parameters via queriesRate limiting, output filtering, access controls
Supply chain / transfer learning attackCompromised pre-trained models or dependenciesProvenance verification, vetted sources
Model skewingGradually shifts model behavior via crafted inputs over timeMonitoring/auditing for drift
Output integrity attackTampers with model outputs in transitEncryption, integrity checks
Membership inferenceDetermines whether specific data was in the training setDifferential privacy, access controls
Insecure output handlingBlindly trusting/executing model output (e.g., as code)Output validation, sandboxing
Model denial of service (DoS)Overwhelms a model with costly queriesRate/token limits, quotas
Sensitive data disclosureModel reveals confidential training data or PIIData minimization, redaction, guardrails
Insecure plug-insVulnerable third-party extensionsLeast privilege, vetting, sandboxing
Excessive agency / overrelianceAI agent given too much autonomy or trustLeast privilege, human-in-the-loop oversight
๐Ÿ’ก
Study tip: Notice how often least privilege, guardrails, and rate limiting reappear โ€” these three controls cover a huge share of this table.

Flashcards

Click a card to flip it. There are 15 terms covering Domain 2's controls and attacks.

Knowledge Check

Question 1 of 10 ยท Score: 0

๐ŸŽ‰

Exam Ready

0 / 10

SecAI+ CY0-001 ยท V1

Next up โ€” Domain 3: AI-Assisted Security (24%)

Now flip perspectives: instead of defending AI systems, learn how AI accelerates security operations and how attackers use AI tools too.