NCA-ADS: Foundations & Environment
Setup

1. GPU vs CPU — Why Acceleration Matters

CPUs have a few powerful cores (typically 4–64) optimized for sequential tasks. They feature large caches, complex branch prediction, and high single-thread performance. Excellent for tasks that require logic, branching, and complex decision-making.

GPUs have thousands of smaller cores designed for parallel tasks. A modern NVIDIA A100 has 6,912 CUDA cores and 2 TB/s HBM bandwidth. GPUs excel when the same operation must be applied to millions of data points simultaneously.

• Large datasets (>100K rows) with parallel mathematical operations
• Matrix operations (linear algebra, neural networks)
• Data stays in VRAM — no constant CPU↔GPU transfers
• GroupBy, sort, join, merge on millions of rows

• Small datasets where PCIe transfer overhead outweighs GPU speed
• Complex sequential logic with many conditional branches
• Tasks that cannot be meaningfully parallelized

CUDA is NVIDIA's parallel computing platform that enables software to directly program GPU cores. RAPIDS is built on CUDA. Memory transfer is the critical bottleneck: moving data from CPU RAM to GPU VRAM over PCIe (~32 GB/s) is orders of magnitude slower than GPU internal memory bandwidth (~2 TB/s for A100). The golden rule: load data once into GPU memory and keep it there.

2. The RAPIDS Ecosystem Overview

RAPIDS is a suite of open-source GPU-accelerated data science libraries from NVIDIA. It brings GPU speed to the familiar Python data science API — same method names, GPU execution engine.

All RAPIDS libraries share GPU memory via zero-copy — a cuDF DataFrame can be passed directly to cuML without any data movement. This is a major performance advantage over CPU-only workflows that serialize data between libraries.

3. Python Data Science Stack for NCA-ADS

CPU Library	Purpose	GPU Equivalent
NumPy	Numerical arrays, math ops	CuPy
pandas	DataFrames, tabular data	cuDF
scikit-learn	ML algorithms	cuML
NetworkX	Graph analytics	cuGraph
Matplotlib / Seaborn	Visualization	CPU only — call .to_pandas() first
Jupyter Notebook/Lab	Interactive dev environment	RAPIDS works natively in Jupyter

import cudf          # instead of import pandas as pd
import cuml           # instead of from sklearn import ...

df = cudf.read_csv("data.csv")      # loads directly into GPU memory
model = cuml.linear_model.LinearRegression()
model.fit(X_gpu, y_gpu)             # trains on GPU
preds = model.predict(X_gpu)        # predicts on GPU

The API is intentionally identical — same .fit()/.predict()/.transform() calls as scikit-learn. Most migration requires only changing import statements.

4. GPU Environment Verification (nvidia-smi)

nvidia-smi (NVIDIA System Management Interface) is the primary CLI tool for verifying GPU health and compatibility. Run it before any RAPIDS work to confirm your environment.

nvidia-smi          # full status table
nvidia-smi -L       # list all GPUs (multi-GPU systems)
nvidia-smi --query-gpu=name,memory.total --format=csv

Always verify bottom-up: GPU Compute Capability → Driver Version → CUDA Version → RAPIDS Version. A mismatch at any level (e.g., CUDA version too old for the installed RAPIDS) will cause import cudf to fail at runtime.

5. Software and Environment Management

Conda manages Python + CUDA + native library dependencies together in isolated environments. This is the recommended approach for RAPIDS because RAPIDS has complex CUDA-linked native library dependencies that pip cannot reliably resolve.

conda create -n rapids-env python=3.10
conda activate rapids-env
# Then install RAPIDS via the rapids.ai release selector command

Export for reproducibility: conda env export > environment.yml
Recreate: conda env create -f environment.yml

pip is Python's package installer. Works well for pure-Python packages but is less reliable for CUDA-linked libraries like RAPIDS components. Use pip inside a conda environment for pure-Python additions; rely on conda for RAPIDS core.

Docker containerizes the entire environment — OS layer, CUDA compatibility, RAPIDS libraries, Python environment. NVIDIA provides official RAPIDS Docker images:

docker pull nvcr.io/nvidia/rapidsai/base:24.10-cuda12.6-py3.12
docker run --gpus all -it nvcr.io/nvidia/rapidsai/base:24.10-cuda12.6-py3.12

6. Version Control with Git

git init                             # initialize repo
git add notebook.ipynb environment.yml
git commit -m "feat: add data prep pipeline"
git push origin main                 # push to remote
git checkout -b experiment/v2-features  # branch for experiment

• Exclude: large data files (*.csv, *.parquet >10MB), model artifacts (*.pkl, *.pt)
• Exclude: .ipynb_checkpoints/ (auto-generated Jupyter metadata)
• Exclude: __pycache__/, .env files with secrets
• Include: environment.yml, requirements.txt, notebooks, source code

Always commit environment.yml alongside your notebooks in the same commit. This ensures any team member can recreate the exact RAPIDS environment that produced the results. For large datasets, reference cloud storage paths rather than committing data directly (or use DVC — Data Version Control).

Tag model releases: git tag -a v1.0-model -m "baseline model". Use branches per experiment so results can be compared and reverted. This is the foundation of reproducible ML workflows covered in NCA-ADS MLOps topic.

7. End-to-End GPU Data Science Workflow