NCP-GENL Practice Questions: LLM Architecture Domain

Correct Answer: B

Explanation: Option B is correct because differential privacy techniques help ensure that the model does not memorize specific data points, thereby protecting sensitive information. Option A, while useful for generalization, does not address privacy concerns. Option C is aimed at reducing overfitting but does not specifically prevent memorization of sensitive data. Option D might limit capacity but does not provide any guarantees regarding data privacy.

Correct Answer: B

Explanation: B is correct because quantization reduces model size and accelerates inference by using lower precision (e.g., INT8) arithmetic, typically with minimal accuracy loss. A is incorrect as lower precision can slightly reduce accuracy, not improve it. C is incorrect because quantization focuses on performance, not interpretability. D is incorrect as quantization is used in inference optimization, not training. Best practice: Use quantization to achieve a balance between performance improvement and accuracy retention.

Correct Answer: A

Explanation: Option A is correct because checking GPU utilization and memory usage can provide insights into whether the hardware resources are being fully utilized or if there are bottlenecks. This step is crucial for diagnosing throughput issues. Option B might not be effective if the GPU is already at capacity. Option C is not a diagnostic step and could introduce other issues. Option D, while potentially beneficial, should not be the first troubleshooting step without understanding the current resource usage.

Correct Answer: A

Explanation: The correct answer is A. Incorporating bidirectional attention mechanisms enhances the model's ability to understand context by allowing it to consider information from both directions in a sequence, which is crucial for understanding user intent. Option B may not directly improve context understanding. Option C, while reducing overfitting, may limit model capacity. Option D is less effective for sequence-based tasks compared to attention mechanisms. Best practice: Use bidirectional attention in NeMo for context-aware LLMs.

Correct Answer: C

Explanation: The correct answer is C. Utilizing mixed precision inference can significantly accelerate computation by using lower precision arithmetic without sacrificing model accuracy, thus meeting strict response time requirements. Option A might reduce efficiency by not leveraging batching. Option B, while useful for some applications, may not directly address response time for a single model. Option D could lead to increased latency. Best practice: Use mixed precision inference in Triton for faster LLM deployment.

Correct Answer: B

Explanation: Option B is correct because implementing gradient clipping helps prevent exploding gradients, which can cause instability and performance degradation during training. Option A might worsen the problem by causing larger updates. Option C is not relevant to the issue of performance degradation over time. Option D would increase computational load without addressing the underlying issue.

Correct Answer: C

Explanation: Enabling dynamic batching (C) allows the server to combine multiple requests into a single batch, improving GPU utilization and throughput. Increasing the number of model instances (A) could help, but dynamic batching is more directly related to optimizing resource usage. Reducing concurrent requests (B) would likely decrease utilization. Switching to a CPU-based deployment (D) is counterproductive in this context as it would underutilize the GPU. Best practice involves using dynamic batching for efficient resource utilization.

Correct Answer: C

Explanation: C is correct because using too many GPUs can introduce significant communication overhead, especially if the model's architecture and data parallelism are not optimized for such scale. A is incorrect because mixed-precision training affects memory usage and speed, not just throughput. B is incorrect as data preprocessing would affect loading times, not throughput in distributed settings. D is incorrect because a large batch size would typically cause memory issues, not throughput bottlenecks directly. Best practice: Balance the number of GPUs with the model's architecture to minimize communication overhead.

Correct Answer: A

Explanation: Implementing RLHF (A) helps align model outputs with human values and ethical guidelines by incorporating human feedback into the training process. Data augmentation (B) can improve model robustness but does not directly address ethical compliance. Adversarial testing (C) is useful for identifying biases but is not a comprehensive solution for ensuring compliance. Logging inferences (D) is important for auditing but does not actively guide the model towards ethical outcomes. Best practice is to use RLHF for ethical alignment.

Correct Answer: A

Explanation: Enabling TensorRT optimization is a key strategy for reducing latency in model inference, as it optimizes the model execution on NVIDIA GPUs. Increasing the batch size (B) might increase throughput but can also increase latency for individual requests. Switching to FP32 precision (C) would typically increase latency due to higher computational demands. Adding more GPUs (D) can help with throughput but may not directly address latency issues. Best practice is to use TensorRT for optimized inference performance on NVIDIA hardware.