Seven lessons, one site. Each lesson is a Carpentries Workbench lesson at https://supersurf.rgoswami.me/<slug>/. The workshop is three sessions of four hours. A lesson keeps every episode. The schedule below is what a session teaches, and the rows marked "cut first" are the ones to drop when the clock runs out.
Commands that differ by machine are in tabs: Snellius, the course cluster, and a generic Magic Castle cluster with EESSI. The course cluster is a SURF Research Cloud workspace, and its tab requests --partition=rome. See the setup page.
Elapsed times include the break. Start with this route, rather than the complete episode estimates below. The instructor guide names slide pages, commands, answers and stopping points.
| Elapsed | Minutes | Block |
|---|---|---|
| 00:00-00:10 | 10 | Purpose and destination |
| 00:10-00:35 | 25 | Access and shell |
| 00:35-00:55 | 20 | Nodes, processes and cores |
| 00:55-01:25 | 30 | EESSI and modules |
| 01:25-01:55 | 30 | First batch job |
| 01:55-02:10 | 15 | Break |
| 02:10-02:40 | 30 | Resource requests and accounting |
| 02:40-03:20 | 40 | Python and OpenMP |
| 03:20-03:40 | 20 | Independent work and arrays |
| 03:40-03:50 | 10 | Keep the result |
| 03:50-04:00 | 10 | Exit check and recovery margin |
Teacher workbook (PDF) | Matching slide pack
| Session | Hours | Focus | Lessons |
|---|---|---|---|
| Part I | 4 | Fundamentals and the EESSI stack | Software from EESSI, Fundamentals of HPC |
| Part II | 4 | File systems, moving data, and many tasks | File systems, Moving data, Many tasks |
| Part III | 4 | Hybrid jobs and performance | Hybrid MPI and OpenMP, Performance analysis |
These estimates cover each entire lesson. Part I uses the selected route above.
| Episode | Teaching (min) | Exercises (min) |
|---|---|---|
| A shared software stack | 15 | 10 |
| Finding and loading a module | 20 | 20 |
| The EESSI client container | 15 | 15 |
| Episode | Teaching (min) | Exercises (min) |
|---|---|---|
| Why I/O matters | 15 | 0 |
| The file systems | 20 | 10 |
| Files and directories | 10 | 25 |
| Permissions and quota | 10 | 20 |
| Space for a job | 15 | 25 |
| Episode | Teaching (min) | Exercises (min) |
|---|---|---|
| Why copying hurts | 10 | 0 |
| Pack, then checksum | 10 | 15 |
| rsync | 10 | 15 |
| Several streams cut first | 15 | 10 |
| From outside the cluster | 10 | 5 |
| Stage in, stage out | 10 | 20 |
| Episode | Teaching (min) | Exercises (min) |
|---|---|---|
| Embarrassingly parallel | 15 | 0 |
| Slurm arrays | 15 | 25 |
| QCG-PilotJob | 25 | 20 |
| Averages with a dependency | 10 | 25 |
| Idle time | 10 | 20 |
| Other tools cut first | 20 | 25 |
| Episode | Teaching (min) | Exercises (min) |
|---|---|---|
| Nodes, tasks, and threads | 20 | 20 |
| AXPY | 15 | 25 |
| Dot product | 15 | 25 |
| Matrix multiplication | 10 | 30 |
| Episode | Teaching (min) | Exercises (min) |
|---|---|---|
| How to evaluate a run | 15 | 5 |
| What limits a run | 20 | 10 |
| Strong and weak scaling | 15 | 20 |
| The roofline | 25 | 20 |
| Energy cut first | 10 | 10 |
| Slurm resources | 10 | 15 |
Three people this day is for. The exercises assume a terminal and a text editor. They do not assume a previous batch job.
Noor writes Python on a laptop and has just been told the data set is on Snellius. Noor can use a terminal and has not seen a module or a queue. Noor needs the fundamentals lesson and the EESSI lesson before any of the later material makes sense. The win for Noor is one sbatch that runs, and a clear rule about which directory holds the result.
Idris has logged in before and has a directory of scripts in /home. Jobs sit in the queue, or they die at the start because the module was loaded in a different order last time. Idris needs the scheduler episode, module purge before module load, and the file system table. The array is the part Idris came to see.
Sam already submits jobs and is now generating thousands of one-second tasks. The queue time is longer than the work. Sam can skim fundamentals, should still do the file system lesson because the outputs are many small files, and should spend the afternoon on QCG-PilotJob and the comparison with HyperQueue, Snakemake, and FireWorks. The performance lesson is there so Sam can say whether more cores helped, with a number.
The course text is sara-nl/introduction-to-supercomputing and Rohit's fork HaoZeke/introduction-to-supercomputing. Deck branches used here: fix/slides-fundamentals-source, fix/slides-fs-source, fix/slides-qcg-source, fix/slides-performance-source, feat/workflow-alternatives, feat/data-movement-deck, and src/eessi-notes.
Fundamentals vendors carpentries-incubator/hpc-intro at 7081080674aceeb0a4bc57b9348da22d876af802. That lesson is CC-BY 4.0, Copyright (c) The Carpentries. Authors are in lessons/fundamentals/AUTHORS.
Lesson text is CC-BY 4.0. The hybrid MPI and OpenMP lesson is adapted from Introducción a la Supercomputación by José Luis Bosque, Borja Pérez, Ramón Beivide, and Carmen Martínez, Universidad de Cantabria, by the EuroCC NL team at SURF.