Free CCDV-F Eval, Testing, and Debugging Practice Test — Claude Certified Developer Questions

This free CCDV-F Eval, Testing, and Debugging practice test covers evaluating and debugging Claude applications — eval design, graders and rubrics, regression testing, prompt iteration, and diagnosing model behavior. Each question includes a detailed explanation aligned to Anthropic's official Claude Certified Developer – Foundations exam outline.

Key Topics in CCDV-F Eval, Testing, and Debugging

6 Free CCDV-F Eval, Testing, and Debugging Practice Questions with Answers

Each question below includes 4 answer options, the correct answer, and a detailed explanation. These are real questions from the FlashGenius CCDV-F question bank for the Eval, Testing, and Debugging domain (2.6% of the exam).

Sample Question 1 — Eval, Testing, and Debugging

A staging chatbot suddenly fails before returning any assistant message. The trace shows: ``` request_id: req_4821 POST /v1/messages status: 401 response: {"error":"invalid API key"} model_response_received: false retry_count: 3 ``` What is the BEST classification of this failure?

  1. A. A model-output failure caused by an unhelpful response
  2. B. An integration-layer failure caused before model generation (Correct answer)
  3. C. A schema validation failure in the post-processing layer
  4. D. A prompt-quality failure caused by unclear instructions

Correct answer: B

Explanation: Correct answer (B): The log shows an HTTP 401 and explicitly says no model response was received. That places the first failure boundary before Claude generated output, so the correct diagnosis is an integration-layer issue involving authentication or request setup. Retrying without fixing the API key will not address the root cause. Why the other options are wrong: - Option A: A model-output failure may seem plausible because the chatbot produced no useful answer, but the log shows no model response was generated. - Option C: Schema validation is a post-processing concern, but the request failed at authentication before any response content could be validated. - Option D: Prompt quality can affect generated content, but the model was never reached, so prompt changes are not the right diagnosis.

Sample Question 2 — Eval, Testing, and Debugging

A team tests a Claude-powered support reply generator. The failing test log shows: ``` expected: "Your refund will be processed in 5-7 business days." actual: "We’ll process your refund within five to seven business days." policy_check: passed required_fields: passed customer_tone: passed ``` The task does not require exact wording. What is the BEST test change?

  1. A. Replace the exact-string assertion with semantic acceptance checks (Correct answer)
  2. B. Increase retry count until the exact sentence is produced
  3. C. Fail the response because it differs from the golden text
  4. D. Disable automated testing for this generated reply

Correct answer: A

Explanation: Correct answer (A): Because the task does not require exact wording and all task-specific checks passed, the test is brittle. LLM output can vary while still satisfying the requirement. A better test validates semantics, required constraints, structured fields, and acceptance criteria instead of a single exact sentence. Why the other options are wrong: - Option B: Retries may occasionally reproduce the target wording, but they increase cost and hide a brittle test design rather than fixing it. - Option C: Golden text can be useful when exact wording matters, but the scenario says equivalent wording is acceptable. - Option D: Disabling testing avoids false failures but removes regression coverage instead of improving the assertion.

Sample Question 3 — Eval, Testing, and Debugging

A batch summarization worker logs the following for one document: ``` attempt 1: status 429, response "rate limit exceeded" attempt 2: status 429, response "rate limit exceeded" attempt 3: scheduled after 200 ms, no jitter operation: generate summary only, no external side effect ``` What is the BEST recovery strategy?

  1. A. Stop all retries because every 429 is a permanent request error
  2. B. Use bounded exponential backoff with jitter for this request (Correct answer)
  3. C. Rewrite the prompt because rate limits indicate poor instructions
  4. D. Ignore the failed document and mark the summary as complete

Correct answer: B

Explanation: Correct answer (B): A rate-limit response is a transient operational failure, and the operation is a generation-only task with no external side effect. The best recovery is a bounded retry policy with exponential backoff and jitter. This avoids hammering the service while still allowing recovery from temporary capacity limits. Why the other options are wrong: - Option A: Some errors are permanent, but the scenario identifies a rate-limit response, which is commonly handled as transient with bounded backoff. - Option C: Prompt wording does not address a rate-limit status because the failure is operational rather than model-output quality. - Option D: Marking the document complete loses work and hides the failure instead of applying an appropriate transient-error policy.

Sample Question 4 — Eval, Testing, and Debugging

A Claude-powered order assistant uses a shipping lookup tool. The trace for a failed request shows: ``` 1. API status: 200 2. assistant tool_use: get_shipping_quote {"zip":"94107","weight_kg":2} 3. app dispatch: get_shipping_quote 4. tool result: HTTP 500 from carrier API 5. assistant continuation: not requested ``` Where did the failure first occur?

  1. A. During Claude API request construction
  2. B. During model generation before tool selection
  3. C. During external tool execution after dispatch (Correct answer)
  4. D. During final response schema validation

Correct answer: C

Explanation: Correct answer (C): The trace shows that the Claude API request succeeded and the model produced a tool request. The application then dispatched the tool, and the first failure signal is the carrier API HTTP 500. Therefore the failing boundary is external tool execution, not model generation or validation. Why the other options are wrong: - Option A: Request construction succeeded because the API returned 200 and a valid tool_use event was produced. - Option B: The model successfully selected a tool and supplied arguments, so the first failure is not before tool selection. - Option D: No final assistant continuation occurred, so final response validation is downstream of the observed failure.

Sample Question 5 — Eval, Testing, and Debugging

A product classifier started failing after a backend refactor. The raw Claude output and validator log show: ``` raw_model_content: {"category":"hardware","confidence":0.82} old_contract: {category:string, confidence:number} new_validator_error: missing required field "label" new_contract: {label:string, confidence:number} prompt_version: unchanged for 3 weeks ``` What is the BEST next diagnostic conclusion?

  1. A. The model has become less capable and needs a larger model
  2. B. The prompt is newly ambiguous and should be rewritten first
  3. C. The validator contract changed without aligning the expected output (Correct answer)
  4. D. The API transport failed and returned malformed response bytes

Correct answer: C

Explanation: Correct answer (C): The raw model output still matches the old contract, and the prompt has not changed. The failure appeared after a backend refactor that introduced a validator expecting `label` instead of `category`. The best conclusion is an application contract mismatch, not a sudden model-quality or transport problem. Why the other options are wrong: - Option A: A larger model might be considered for quality issues, but the evidence points to a schema contract mismatch introduced by the refactor. - Option B: The prompt is unchanged, and the model output matches the old expected shape, so prompt ambiguity is not the first conclusion. - Option D: The raw content is well-formed and available to the validator, so malformed transport bytes are not indicated.

Sample Question 6 — Eval, Testing, and Debugging

A Claude workflow drafts an invoice email, calls a payment API tool, then asks Claude to summarize the result. A timeout occurs after the tool call: ``` tool_call: charge_customer {"invoice_id":"INV-77","amount":200} payment_api: request sent client_error: timeout waiting for payment_api response retry_policy: immediate retry of whole workflow idempotency_key: none ``` What is the BEST debugging-driven mitigation before enabling retries?

  1. A. Retry the full workflow immediately to maximize completion rate
  2. B. Change the summary prompt so Claude reports fewer timeouts
  3. C. Use a cheaper model so retries cost less when failures occur
  4. D. Add idempotency or status checks around the payment operation (Correct answer)

Correct answer: D

Explanation: Correct answer (D): The timeout happened after a side-effecting payment request may have been sent. Retrying the whole workflow without an idempotency key or status check can duplicate the charge. The correct mitigation is to make recovery idempotency-aware before retrying an ambiguous external action. Why the other options are wrong: - Option A: Immediate full-workflow retry may look like a simple recovery, but it risks duplicate external side effects. - Option B: The prompt cannot determine whether the payment API already processed the request after the timeout. - Option C: Reducing model cost does not address the correctness risk of duplicating a payment action.

How to Study CCDV-F Eval, Testing, and Debugging

Combine these CCDV-F Eval, Testing, and Debugging practice questions with the official Anthropic documentation and hands-on projects. Build a small Claude-powered application with tool use, try Claude Code on a real repository, and iterate on prompts with evals — practical experience is the fastest way to master this domain.

About the CCDV-F Exam

Other CCDV-F Domains

Start the free CCDV-F Eval, Testing, and Debugging practice test now | 10-question quick start | All CCDV-F domains