Module 1: GPU Fundamentals and Deep Learning Acceleration

Topics

• CPU vs. GPU vs. TPU architectural differences and parallelism inference
• Checking GPU availability and utilization
• Getting started with acceleration

Objectives

• Understand architectural differences between CPU, GPU, and TPU and their role in parallelism and inference
• Learn how to check GPU availability and utilization
• Explore initial steps to enable acceleration in AI/ML workflows

Module 2: Understanding GPU-Optimized Deep Learning Workflow

Topics

• GPU acceleration libraries for deep learning
• Data flow between CPU, GPU, model, and output
• Best practices for batching, shuffling, and memory management

Objectives

• Understand how GPU acceleration libraries (CUDA, cuDNN, tensor cores) optimize computations
• Learn the flow of data between CPU, GPU, model, and output
• Apply best practices for batching, shuffling, and memory management in GPU-enabled training

Module 3: TensorFlow and Keras on GPUs

Topics

• Moving tensors and models to GPU
• Training networks on datasets with and without GPU acceleration
• Monitoring GPU memory usage and training speed

Objectives

• Learn how to move tensors and models to GPUs
• Compare training performance with and without GPU acceleration
• Monitor GPU memory and training speed in real time

Module 4: Performance Optimization for Single-GPU and Multi-GPU Training

Topics

• Identifying and reducing data pipeline bottlenecks
• Mixed precision training
• Gradient accumulation and efficient batch sizing strategies

Objectives

• Identify and reduce data pipeline bottlenecks
• Apply mixed precision training for faster computation and reduced memory usage
• Use gradient accumulation and batch sizing strategies for optimal training

Module 5: Advanced Controls

Topics

• Custom GPU kernels
• Avoiding race conditions
• Multidimensional grids
• Sorting and memory optimization techniques

Objectives

• Learn to design and implement custom GPU kernels
• Understand how to avoid race conditions in GPU programming
• Explore multidimensional grid strategies and advanced GPU operations

Module 6: Debugging, Troubleshooting, and Best Practices

Topics

• Debugging out-of-memory (OOM) and bottlenecks
• Framework-specific GPU error handling (TensorFlow, PyTorch)
• Profiling GPU performance

Objectives

• Debug out-of-memory (OOM) and data bottleneck issues in GPU workflows
• Understand common framework-specific GPU errors in TensorFlow and PyTorch
• Profile and optimize GPU performance for production readiness