Tutorial

Measuring Job Efficiency

45 Minutes

Objectives

• Compare the resources a job requested with what it used, using sacct and seff.
• Profile a job to see how its CPU, memory and I/O use change over time.
• Run commands as job steps, to measure each part of a job separately.

Prerequisite

• You can write and submit a batch script, and check on it with squeue and sacct. These are covered in the Introduction to HPC tutorials.

Set up

Check a finished job

What would you request next time?

Solution

#

Reading seff

seff reports Avg CPU Utilisation: 26% for a job that requested 4 CPUs. What does that mean?

On average, the job kept about one of its four CPUs busy.

Each CPU was busy for 26% of the job, so the job needed all four.

Not necessarily: an average can hide a short burst that used all four CPUs, or long stretches using one.

The job ran for 26% of its time limit.

That is the Job Wall-time line.

See resource use over time

What stands out?

Solution

Run commands as job steps

#

Job steps

Inside a batch script, what does putting srun in front of a command do?

Runs the command as a job step, which sacct and profile_plot report separately.

Submits the command as a new job, which waits in the queue.

That is what sbatch does. Inside a job, srun uses the job's own resources.

Runs the command on the login node.

What the profile tells us

Keypoints

• sacct and seff compare what a job requested with what it used, but only as totals and averages.
• Profiling shows how use changes over time, and job steps split it by command.
• Different parts of a job can need different resources.

What next

• Next lesson: Loops and Variables.
• For reference: checking job efficiency and Slurm Native Profiling.

Quiz Progress

• Answered: 0 / 0 (0%)
• Correct: 0 / 0 (0%) Reset