NVIDIA NCA-AMAD exam · Official objectives · Original practice content · Last reviewed October 16, 2026

Free NCA-AMAD Inference Concepts Practice Questions

The Inference Concepts domain is 15% of the NVIDIA NCA-AMAD exam and covers prefill and decode, batching, KV and prefix caching, quantization, TensorRT-LLM, NVIDIA Dynamo, latency, throughput and cost. Review each answer and explanation as you practice.

The NCA-AMAD exam has 50-60 multiple-choice questions plus a hands-on lab in 1 hour. It costs $125, results are pass or fail, and the certification is valid for 2 years.

Take the 10-question NCA-AMAD quick-start test · NCA-AMAD cheat sheet · Create a free account

What Inference Concepts covers on the NCA-AMAD exam

This domain is 15% of the NCA-AMAD exam (about 9 questions in a 60-question mock). NVIDIA's objectives include:

How to study Inference Concepts

NCA-AMAD Inference Concepts sample questions with answers

Inference Concepts

Sample question 1

What does NVIDIA Dynamo add on top of inference engines such as TensorRT-LLM, vLLM and SGLang?

  1. A quantization toolkit that converts checkpoints to FP8 or NVFP4 before deployment
  2. A training framework that shards model weights across nodes during pretraining
  3. A distributed layer for disaggregated serving, KV-aware routing and autoscaling
  4. A replacement engine with its own kernels that runs models without those libraries
Show correct answer and explanation

Correct answer: C. A distributed layer for disaggregated serving, KV-aware routing and autoscaling

C is correct because Dynamo leaves the single-GPU forward pass to the engine and adds the system layer around it: disaggregated prefill and decode, KV cache-aware routing, KV cache offloading across memory tiers and SLA-based autoscaling. A is incorrect because post-training quantization is done with TensorRT Model Optimizer; Dynamo serves models but does not quantize them. B is incorrect because Dynamo is an inference serving framework; distributed training is the job of tools such as Megatron-LM and the NeMo framework. D is incorrect because Dynamo is engine-agnostic and runs on top of vLLM, SGLang and TensorRT-LLM rather than replacing them.

Inference Concepts

Sample question 2

A team set up its NVIDIA Dynamo deployment from offline estimates. During daily peaks every worker is saturated and TTFT misses its target, while overnight many workers sit idle. Which Dynamo capability addresses this?

  1. KV cache-aware routing that sends requests to workers holding their prefixes
  2. KV cache offloading that moves cache blocks from GPU memory to host memory
  3. SLA-based planning that scales prefill and decode workers from live traffic
  4. Offline configuration search that estimates the best layout before launch
Show correct answer and explanation

Correct answer: C. SLA-based planning that scales prefill and decode workers from live traffic

C is correct because Dynamo's planner takes TTFT and time-per-output-token targets, watches live traffic and adds or removes prefill and decode workers, so capacity follows peaks that a static offline configuration cannot handle. A is incorrect because routing improves cache hits among the existing workers; it cannot add capacity when every worker is saturated. B is incorrect because offloading enlarges the cache to avoid recomputing prompts; it does not change how many workers serve the load. D is incorrect because an offline estimate is what produced the static setup; it does not react to traffic once the model is in production.

Inference Concepts

Sample question 3

When NVIDIA Dynamo uses KV cache-aware routing, which information does the router use to choose a worker for a new request?

  1. Each worker's current load and how much of the request's KV cache it already holds
  2. The model size on each worker, so larger requests go to workers with more GPUs
  3. The client's network location, so each request goes to the nearest data center
  4. The order in which workers registered, sending requests to each in strict rotation
Show correct answer and explanation

Correct answer: A. Each worker's current load and how much of the request's KV cache it already holds

A is correct because the router balances worker load against existing cache hits, because a worker that already holds the request's precomputed KV pairs can skip that prefill work. B is incorrect because workers in a pool serve the same model; the router decides by load and cache hits, not by model size. C is incorrect because geographic routing reduces network latency but does not consider which worker already holds the request's KV cache. D is incorrect because strict rotation is round-robin load balancing, which ignores both load and cache contents.

Inference Concepts

Sample question 4

A manufacturing team serves a stateful anomaly-detection model on Triton Inference Server. Each machine sends a stream of sensor windows, and every window from one machine must reach the same model instance so the model can keep its state. Which scheduler should the team configure for this model?

  1. The sequence batcher, which routes each sequence's requests to one instance
  2. A custom batching library that drops requests arriving out of order
  3. The dynamic batcher, which combines requests into batches across all instances
  4. The dynamic batcher with preferred batch sizes set to the number of machines
Show correct answer and explanation

Correct answer: A. The sequence batcher, which routes each sequence's requests to one instance

A is correct because the sequence batcher is meant for stateful models: it batches requests dynamically while keeping all requests of a sequence, identified by a correlation ID, on the same model instance. B is incorrect because custom batching rules add conditions to the dynamic batcher; they do not provide sequence routing or state handling. C is incorrect because the dynamic batcher is meant for stateless models and spreads batches over every instance, so one machine's windows could land on different instances. D is incorrect because preferred batch sizes only shape batch size; they do not pin a sequence to one instance, and they are not recommended for most models.

Inference Concepts

Sample question 5

A retail company runs an LLM NIM on Kubernetes. During holiday peaks, each new pod added by the autoscaler takes many minutes to become ready because it downloads the model weights from NGC before serving. Which change MOST directly shortens this cold start?

  1. Lower the autoscaler's scale-down delay so idle pods are removed sooner
  2. Pre-cache the model on shared network storage that every new pod mounts
  3. Switch the readiness probe to report ready before the model finishes loading
  4. Raise the maximum batch size so each existing pod accepts more requests
Show correct answer and explanation

Correct answer: B. Pre-cache the model on shared network storage that every new pod mounts

B is correct because caching the model in the cluster removes the per-pod download: a single cached copy on a network-backed persistent volume serves every new pod, so startup time drops. A is incorrect because removing pods sooner affects how long capacity lingers after a peak, not how long a new pod takes to start. C is incorrect because marking a pod ready early sends it traffic it cannot serve yet, which causes failed requests instead of a faster start. D is incorrect because a larger batch can raise the throughput of running pods but does nothing about the download that delays new ones.

Inference Concepts

Sample question 6

A public-sector agency calls LLMs from two cloud providers and one self-hosted endpoint. The developers want to move a workload to a different provider or endpoint without editing application code each time. Which design BEST supports this?

  1. A gateway layer that maps semantic model names to provider endpoints and IDs
  2. A larger context window on every model, so that prompts never need changes
  3. A single provider for every workload, so that there is nothing to switch
  4. Hard-coded client code for each provider, chosen by an if-statement per call
Show correct answer and explanation

Correct answer: A. A gateway layer that maps semantic model names to provider endpoints and IDs

A is correct because a provider abstraction gives each model target a semantic name and keeps the mapping to endpoint and model ID in the gateway, so moving a model is a configuration change, not an application change. B is incorrect because context length has nothing to do with where requests are sent or how endpoints are configured. C is incorrect because removing choice avoids the problem only by giving up the flexibility the agency wants, and it creates lock-in. D is incorrect because embedding each provider's endpoint in the application is exactly what forces code edits whenever a model moves.

Inference Concepts

Sample question 7

An insurance claims API on Triton Inference Server must answer within 2 seconds or the caller retries. During traffic spikes, requests sit in the dynamic batcher queue for longer than that and are processed anyway, wasting GPU time on answers nobody receives. Which dynamic batcher setting BEST addresses this?

  1. Preferred batch sizes so the batcher sends only batches of 4 or 8 requests
  2. A queue policy with a maximum queue size and a timeout for waiting requests
  3. A larger max_queue_delay_microseconds value so that fuller batches can form
  4. The preserve_ordering option so responses return in the order received
Show correct answer and explanation

Correct answer: B. A queue policy with a maximum queue size and a timeout for waiting requests

B is correct because the dynamic batcher's queue policy (max_queue_size with a timeout and timeout action) rejects or defers requests that wait longer than allowed, so the GPU does not spend time on answers the caller has abandoned. A is incorrect because preferred batch sizes shape how batches are formed; they do not limit how long a request may wait. C is incorrect because a longer batching delay holds requests in the queue even longer, which adds latency rather than removing stale work. D is incorrect because preserving order only controls the response sequence; it neither limits the queue nor drops expired requests.

Inference Concepts

Sample question 8

A coding-assistant service runs 3 replicas of an LLM, each configured with a maximum batch size of 64. Monitoring shows the values below:

Concurrent requests   TTFT p50 (ms)   GPU utilization
120                   310             71%
180                   340             88%
240                   2,950           93%

What is the MOST likely reason TTFT jumps at 240 concurrent requests?

  1. The decode phase slowed down, which delays when the first token appears
  2. The GPUs reached full utilization, which stops them from running prefill
  3. The prompts became longer at higher load, which lengthens the prefill phase
  4. Requests beyond the 192 batch slots wait in a queue before prefill starts
Show correct answer and explanation

Correct answer: D. Requests beyond the 192 batch slots wait in a queue before prefill starts

D is correct because the replicas can process at most 3 x 64 = 192 requests at once, so at 240 concurrent requests about 48 must queue for a free slot, and that queueing time is part of TTFT. A is incorrect because the first token is produced at the end of prefill; decode speed affects the time between later tokens, not TTFT. B is incorrect because the GPUs report 93%, and high utilization slows work rather than stopping prefill; the sharp step comes from queueing. C is incorrect because nothing indicates that prompt length changed; the jump appears exactly when concurrency passes the available batch slots.

Inference Concepts

Sample question 9

A legal-drafting assistant served with NVIDIA Dynamo streams answers that take several minutes to generate. When a worker fails mid-answer, users lose the partial text and must resubmit. The team cannot afford to run every request twice. Which approach BEST protects these long requests from worker failures?

  1. Migrate in-flight requests to healthy workers, keeping the partial output
  2. Send every request to two workers and keep whichever finishes first
  3. Have the client resubmit the whole request to another worker on timeout
  4. Add more replicas so that each worker carries a smaller share of load
Show correct answer and explanation

Correct answer: A. Migrate in-flight requests to healthy workers, keeping the partial output

A is correct because request migration moves in-progress requests from a failed worker to a healthy one while preserving partial generation state, so the token stream continues without a restart or a duplicate run. B is incorrect because duplicating each request doubles GPU use, which the team cannot afford. C is incorrect because a resubmission starts generation over, so users still lose minutes of progress on every failure. D is incorrect because more replicas reduce load per worker but do not save the requests running on a worker that fails.

Inference Concepts

Sample question 10

During a product launch, traffic to a retail assistant served with NVIDIA Dynamo exceeds capacity before new workers can start. Queues grow and TTFT climbs for every new request. Which behavior BEST keeps latency acceptable for the requests the system accepts?

  1. Raise the batch size on each worker beyond its tested KV cache capacity
  2. Accept every request and let the queue grow until new workers come online
  3. Reject new requests with a retryable error once workers pass an overload limit
  4. Route all new requests to a single worker so the others can finish their work
Show correct answer and explanation

Correct answer: C. Reject new requests with a retryable error once workers pass an overload limit

C is correct because load shedding turns away new work when workers are overloaded (Dynamo rejects requests based on configurable overload thresholds), so admitted requests keep their latency while clients retry and new capacity starts. A is incorrect because exceeding KV cache capacity risks out-of-memory failures that would hurt every in-flight request. B is incorrect because an unbounded queue raises TTFT for everyone and can push all requests past their timeouts. D is incorrect because concentrating new load on one worker overloads it without reducing total demand.

Keep practicing

NCA-AMAD practice test hub · Mixed quick start · NCA-GENL practice test

FlashGenius is an independent study resource and is not affiliated with or endorsed by NVIDIA. NVIDIA and NCA-AMAD are trademarks of NVIDIA Corporation. 600 original practice questions.