NCP-GENL Practice Questions: Data Preparation Domain

Correct Answer: D

Explanation: Text normalization is crucial in data preparation for LLMs as it standardizes the dataset, making it easier for the model to learn patterns. This includes converting text to lowercase, expanding contractions, and normalizing slang terms. While spell checking and custom tokenization can be useful, they are secondary steps. Data augmentation is more applicable to expanding dataset diversity rather than correcting data quality issues.

Correct Answer: B

Explanation: The correct answer is B. Creating a custom tokenizer that can recognize and preserve domain-specific terms ensures that the model can learn from and understand the context of such terms, which is critical for fine-tuning in specialized domains. Option A would result in loss of important information. Option C might alter the meaning of the text. Option D could lead to suboptimal model performance as the standard tokenizer might misinterpret the jargon.

Correct Answer: A

Explanation: Option A is correct because oversampling can help address class imbalance by increasing the representation of underrepresented languages, which is crucial for training a balanced multilingual model. Option B can lead to loss of linguistic diversity. Option C is plausible but may introduce synthetic biases. Option D ignores the importance of multilingual representation.

Correct Answer: C

Explanation: The correct answer is C. Using language-specific tokenizers ensures that the unique grammatical and syntactical nuances of each language are preserved, which is crucial for effective learning from multilingual data. Option A would eliminate the benefits of multilingual training. Option B may not be practical or necessary, depending on the use case. Option D is a good practice for training but does not directly address language-specific processing needs.

Correct Answer: B

Explanation: The correct answer is B. Implementing a data cleaning pipeline is crucial in removing noise and irrelevant data, which enhances the quality of the dataset and the resulting model. Option A is incorrect because increasing batch size does not address data quality issues. Option C is incorrect because reducing data size does not solve the underlying noise problem. Option D is incorrect because data augmentation does not inherently clean or improve data quality.

Correct Answer: A

Explanation: Using oversampling techniques to increase the representation of the underrepresented class is a proactive way to address class imbalance during data preparation. This approach ensures that the model has sufficient examples to learn from all classes. Option B is incorrect because ignoring the imbalance can lead to biased models. Option C might result in loss of valuable data from the majority class, and Option D, while useful, is a training-time adjustment rather than a data preparation strategy.

Correct Answer: B

Explanation: Option B is correct because data cleaning is a crucial step in preparing data for training generative AI models. It involves normalizing text formats, removing duplicates, and filtering noise, which helps in improving the quality of the input data. Option A is incorrect as relying solely on the model's preprocessing can lead to suboptimal results. Option C is too simplistic and doesn't address deeper issues like duplicates or noise. Option D might reduce complexity but risks losing valuable information and doesn't address data quality issues.

Correct Answer: B

Explanation: The correct answer is B. Tokenizing and cleaning text data is crucial for preparing a dataset for training a large language model, as it ensures the model learns from relevant and high-quality information. Option A is incorrect because converting all data into a structured format may not be feasible or necessary for unstructured text data. Option C is more relevant for numerical datasets rather than text data. Option D is incorrect as duplicating entries can lead to overfitting and does not genuinely increase dataset diversity.

Correct Answer: C

Explanation: Option C is correct because curating additional data that represents underrepresented topics can help mitigate bias in the dataset. Option A would exacerbate the bias. Option B might not address the underlying bias issue. Option D is not relevant to addressing data bias.

Correct Answer: B

Explanation: Option B is correct because NeMo provides specialized data processing pipelines that can handle both structured and unstructured data, ensuring they are normalized and tokenized appropriately for LLM training. Option A is incorrect because simply converting structured data to JSON doesn't ensure it's in a suitable format for the model. Option C is incorrect because structured data also needs to be normalized and tokenized. Option D is incorrect because generic tools may not handle the nuances of LLM data preparation. Best practice is to use NeMo's pipelines for comprehensive data preparation.