CCDV-F Practice Questions: Model Selection and Optimization Domain

Correct answer (C): Haiku is the best fit when a task is simple, high-volume, latency-sensitive, and cost-sensitive, provided evaluation shows the quality is sufficient. The scenario states that simple instructions already meet the labeling requirement, so paying for a higher-capability model or additional reasoning is not justified.

Why the other options are wrong:
- Option A: Opus may be attractive because it is the highest-capability tier, but the scenario does not require complex reasoning and explicitly prioritizes cost and latency.
- Option B: Sonnet is a strong balanced default for many production workloads, but this specific task is simple and already validated with straightforward instructions, making Haiku the better starting point.
- Option D: Extended thinking can help reasoning-heavy tasks, but this is a simple classification workload where added latency and token usage would not be justified.

Correct answer (A): Prompt caching is most useful when many requests share a large, stable prefix or reusable context. Here, almost all input tokens come from the same manual and instructions repeated across many requests, while only a small user question changes. Caching the stable prefix can reduce repeated processing cost when the prompt is structured consistently around stable boundaries.

Why the other options are wrong:
- Option B: A larger model might improve quality in some cases, but the problem described is repeated input-token cost from stable context. Switching to Opus does not address that cost driver.
- Option C: Increasing output limits could raise cost and latency. The artifact shows the main issue is large repeated input, not insufficient answer length.
- Option D: Frequently changing content should not be placed in the cacheable prefix because it prevents cache reuse. The user question should remain outside the stable cached portion.

Correct answer (A): Examples can improve quality but increase token usage. The single-shot prompt meets the stated 95% quality requirement at much lower token cost than the 10-example prompt. Zero-shot is cheaper but fails the accuracy requirement, so the single-shot prompt is the best tradeoff.

Why the other options are wrong:
- Option B: Zero-shot has the lowest token cost, but it does not meet the required accuracy threshold.
- Option C: The 10-example prompt has slightly higher accuracy, but the requirement is to meet at least 95% with the lowest practical token cost.
- Option D: A stronger model may help in some cases, but the provided evaluation already identifies a prompt format that satisfies quality and cost constraints.

Correct answer (A): Claude outputs can vary because LLM generation is based on probabilistic next-token prediction and sampling behavior. For acceptable summaries with different wording, tests should evaluate required facts, structure, and quality criteria rather than requiring exact string equality.

Why the other options are wrong:
- Option B: The artifact shows key facts are present, so there is no evidence that the SDK is dropping tokens. Retrying for an exact match treats normal variation as an integration failure.
- Option C: The summaries include the required facts, so the issue is not insufficient context. Adding a repository would increase cost and distraction without addressing non-determinism.
- Option D: Prompt caching can reduce repeated processing of static prompt content, but it does not guarantee identical generated outputs.

Correct answer (A): The checkpoint should capture the stable reusable prefix: the behavior guide, return policy, and catalog excerpt. Placing it after section 3 maximizes reuse while excluding customer account history and the current question, which change per request and would reduce cache effectiveness.

Why the other options are wrong:
- Option B: Placing the checkpoint after the full prompt includes unique customer content, which makes reuse much less likely.
- Option C: Putting the checkpoint before the stable sections does not capture the reusable long prefix that drives the optimization.
- Option D: Account history may be important for answer quality, but it is unique per request, so including it in the checkpoint hurts reuse.

Correct answer (A): LLM outputs can vary across runs because generation is probabilistic and based on next-token prediction and sampling. For this type of CI check, exact string comparison is brittle when semantically correct answers can be phrased differently. The better test verifies required meaning, structured fields, or acceptance criteria rather than identical prose.

Why the other options are wrong:
- Option B: Exact wording comparison may be appropriate for deterministic code paths, but it is brittle for probabilistic model output when the meaning is correct.
- Option C: WebSockets or streaming-oriented transports affect response delivery and perceived responsiveness. They do not make model generation identical across runs.
- Option D: A larger context window does not guarantee identical wording. It can also increase cost and latency without fixing the test-design issue.

Correct answer (A): Prompt caching is most useful when many requests share a large static prefix. The stable system instructions and policy manual are good cache candidates because they are reused across requests. Frequently changing claim details and user questions should come after the cached prefix rather than serving as cache anchors.

Why the other options are wrong:
- Option B: The customer claim and user question change on each request, so placing the checkpoint after them makes the reusable cached portion less effective or not reusable across requests.
- Option C: The generated answer changes, but caching applies to repeated static prompt content. Variable outputs do not prevent caching the shared policy manual prefix.
- Option D: Caching generated answers is not the prompt caching optimization described here, and it would not help with new claims that require fresh reasoning over different inputs.

Correct answer (A): Fast mode fits the low-reasoning, tight-latency autocomplete feature, while higher effort or extended thinking fits the reasoning-heavy deployment recovery task where more latency is acceptable. Thinking settings should be matched to task complexity and latency budget because higher effort can increase token use and latency.

Why the other options are wrong:
- Option B: Extended thinking for both features wastes latency and tokens on the simple autocomplete task.
- Option C: Fast mode for both features may harm the reasoning-heavy recovery analysis, where quality depends on deeper deliberation.
- Option D: Higher effort is misplaced on the simple tagging task, and zero-shot alone does not address the need for deeper reasoning in recovery planning.

Correct answer (A): Sonnet is commonly the balanced default when an application needs strong quality while preserving better latency and cost than the most capable model. The task requires meaningful reasoning over legal text, so the smallest model for all cases is risky, but the scenario does not justify using the most expensive model for every request because lawyers review outputs and budget is moderate.

Why the other options are wrong:
- Option B: Haiku may reduce cost, but the scenario calls for strong reasoning over legal language. Choosing it for all reviews without validation would underweight quality requirements.
- Option C: Opus may help the hardest cases, but the scenario states a moderate budget and interactive latency needs. Using it for every clause is not the best default.
- Option D: Fast mode may improve latency, but skipping quality evaluation is inappropriate for a legal-review workflow with stated reasoning requirements.

Correct answer (A): Fast response modes are appropriate for simple, latency-sensitive interactions, while extended or adaptive thinking is most appropriate when complex reasoning quality is worth additional latency and token usage. The two paths have different complexity and latency requirements, so they should not be tuned identically.

Why the other options are wrong:
- Option B: Deeper thinking for the simple near-realtime path would add unnecessary latency and token usage without a matching quality need.
- Option C: Fast responses for the complex incident analysis path may under-serve the reasoning requirement where the team can tolerate additional latency.
- Option D: Model family and thinking mode should reflect task complexity; the artifact states Path B needs complex reasoning, so this option ignores the key requirement.