Three ways to use slurm on a high performance computer (HPC) (CC130)

July 21, 2021 • PD Schloss • 1 min read • •

Slurm scripts

`interactive.slurm`

#!/bin/bash

srun --account=pschloss1 --partition=standard --time=02:00:00 --ntasks=1 --cpus-per-task=1 --nodes=1 --mem=6GB --pty /bin/bash

`single.slurm`

#!/bin/bash
#SBATCH --mail-user=you@umich.edu
#SBATCH --mail-type=BEGIN,END
#SBATCH --cpus-per-task=16
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
#SBATCH --mem-per-cpu=4g
#SBATCH --time=2:00:00
#SBATCH --account=pschloss1
#SBATCH --partition=standard
#SBATCH --output=%x.o%A_%a

make -j 16 processed_data/l2_genus_pooled_performance.tsv

`array.slurm`

#!/bin/sh
#SBATCH --mail-user=you@umich.edu
#SBATCH --mail-type=BEGIN,END
#SBATCH --cpus-per-task=1
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
#SBATCH --mem-per-cpu=4g
#SBATCH --time=24:00:00
#SBATCH --account=pschloss1
#SBATCH --partition=standard
#SBATCH --output=%x.o%A_%a
#SBATCH --array 1-100

SEED=$((SLURM_ARRAY_TASK_ID))
echo $SEED

make processed_data/l2_genus_$SEED.Rds

Code

You can browse the state of the repository at the

Installations

If you haven’t been following along, you can get caught up by doing the following:

(windows) Install the Ubuntu Linux BASH shell for Windows 10

(mac) Install homebrew and git

  /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
  brew install git

To get to where we are at the beginning of this episode (you won’t have the same issue numbers at Pat)…
- Set up a GitHub account
- Create a new GitHub repository
  - Call it “mikropml_demo”
  - Make it Public
  - Don’t check the box next to “Initialize this repository with a README”
  - Click the green “Create repository” button
- Go to your command line and enter the following replacing <your_github_id> with your GitHub user id
```
git clone git@github.com:SchlossLab/mikropml_demo.git
cd mikropml_demo
git reset --hard 01898b3e8b3edbb4d7099c4c523a2e2bde643993
git remote set-url origin git@github.com:<your_github_id>/mikropml_demo.git
git push -u origin master
```
- Return to GitHub and refresh your browser.
- Go to the mikropml_demo directory on your computer and double click on the mikropml_demo.Rproj icon. This will launch RStudio and you’ll be good to go.