HPC for Humanists

If someone has told you to use HPC but references to SLURM or nodes or parallelization leave you feeling baffled and overwhelmed, you’re in the right place! HPC for Humanists (and Social Scientists) is a practical guide to getting started with high-performance computing for people with some basic familiarity with the command line and working with code, but not much more than that.

If you’re looking for more background first, check out the Programming Historian tutorial on the command line. Examples here will mostly involve working with text using Python, so you might want to review Python text manipulation basics too.

It is possible to help people get started with HPC by giving them a series of functionally-magical incantations they can copy and paste into the command line. That approach might be faster at first, but as soon as something doesn’t work, the user doesn’t have the background or vocabulary to try to get themselves unstuck. This guide aims to cover just enough about HPC – using human-oriented language, not computer-oriented commands– to give you a set of practical tools, without overwhelming you with details that are unlikely to matter for most humanities users.

Most of this guide has been written as a big-picture explainer for HPC that isn’t tied to the peculiarities of any one institution’s cluster, because the details vary. There are also more detailed, step-by-step instructions specific to Stanford. The titles of those sections start off with an evergreen tree emoji 🌲.