Reusable Job Scripts
45 Minutes
Objectives
- Use Slurm environment variables inside a job, such as the number of CPUs it was given.
- Pass arguments to a batch script, so one script can process any sample.
- Override the options in a script's header when you submit it.
Prerequisite
- You can use variables and loops in a script. See Loops and Variables.
Skipped ahead? Set up here
Slurm environment variables¶
#
Where Slurm variables exist
You run echo ${SLURM_CPUS_PER_TASK} on the login node. What do you see?
Match threads to CPUs¶
One script for any sample¶
Process a second sample
Solution
Override options when you submit¶
#
Command line or header
sample.sl has #SBATCH --mem 2G in its header. You submit it with sbatch --mem 4G sample.sl SRR2584863.
How much memory does the job get?
Variables in the header¶
Keypoints
- Slurm sets environment variables inside a job, such as
SLURM_CPUS_PER_TASK. - Script arguments let one script process different inputs.
- Options on the
sbatchcommand line override the script's header. - Variables do not work in
#SBATCHlines. Use tokens like%xand%j.
What next
- Next lesson: Debugging Failed Jobs.
- For reference: the Batch Computing Guide and the Slurm Reference Sheet.