Tutorial

In this tutorial, we’ll be using the Gadi HPC machine at NCI. A Python virtual environment will be provided for you during the session.

Topics	Material	Exercises	Duration
Requesting a Job	10 minutes	10 minutes	20 minutes
High-level HPC Architecture	20 minutes	15 minutes	35 minutes
Basic of Parallelism	20 minutes	15 minutes	35 minutes
Vector Parallelism	20 minutes	10 minutes	30 minutes
Multi-core Parallelism	20 minutes	10 minutes	30 minutes
GPU Parallelism	30 minutes	15 minutes	45 minutes
Multi-node Parallelism	20 minutes	15 minutes	35 minutes

• Getting Started
• Requesting a Job
• High-level HPC Architecture
  • HPC Compute Node
  • How do you figure out the node architecture?
  • How does cache influence performance?
• Basics of Parallelism
  • Process
  • Threads
  • Challenges with Threads
  • The Room and the Key: An analogy
  • Exercise
  • Single Instruction Multiple Data (SIMD)
• Vector Parallelism
  • What is vectorization?
  • How does vectorization influence peformance?
  • Exercise
• Multi-core Parallelism
  • Non-Uniform Memory Access (NUMA)
  • Exercise
• GPU Parallelism
  • GPU Execution Model
  • Thread Indexing
  • How do we assign a unique thread id to each thread?
  • Warps and Warp Schedulers
  • Data Movement in GPUs
  • Streams
  • Exercise
• Multi-node Parallelism
  • Broadcast Operation
  • GPU-aware MPI and All-Gather Operation
  • Exercise