Slurm Environmental Variables
When a job scheduled
by Slurm starts, it needs to know certain
things about how it was scheduled, etc. E.g., what is it's working
directory, or what nodes were allocated for it. Slurm passes this
information to the job via environmental variables. In addition to
being available to your job, these are also used by programs like
mpirun to default values. This way, something like
mpirun already knows how many tasks to start and on which
nodes, without you needing to pass this information explicitly.
The following is a list of commonly used variables that are set by Slurm for each job, along with a brief description, sample value, and the nearest analog for PBS/Torque based schedulers. A full list of the variables set by Slurm for each job is available in the sbatch man page.
| Env Variable Name | Description | Example values | PBS/Torque analog |
|---|---|---|---|
| SLURM_CPUS_ON_NODE | Number of cores per node | 8,3 | PBS_NUM_PPN |
| SLURM_CPUS_PER_TASK | Number of cores per task; e.g. the value given to the --cpus-per-task or -c flag |
8,3 | PBS_NUM_PPN |
| SLURM_JOB_ID | Job ID | 5741192 | PBS_JOBID |
| SLURM_JOBID | Deprecated. See SLURM_JOB_ID | ||
| SLURM_JOB_NAME | Job name | myjob | PBS_JOBNAME |
| SLURM_JOB_NODELIST | Nodes assigned to job | compute-b24-[1-3,5-9],compute-b25-[1,4,8] | PBS_NODEFILE |
| SLURM_JOB_NUM_NODES | Number of nodes allotted to job | 2 | PBS_NUM_NODES |
| SLURM_LOCALID | Index to core running on (within node) | 4 | |
| SLURM_NODEID | Index to node running on (relative to those assigned to job) | 0 | PBS_NODENUM |
| SLURM_NNODE | Deprecated, see SLURM_JOB_NUM_NODES | ||
| SLURM_NODELIST | Deprecated, see SLURM_JOB_NODELIST | ||
| SLURM_NTASKS | Number of (MPI) tasks for job | 12 | PBS_NP ??? |
| SLURM_PROCID | Index to task (relative to job) | 0 | PBS_O_TASKNUM |
| SLURM_SUBMIT_DIR | Submission directory | /home/voldemort/scratch | PBS_O_WORKDIR |
| SLURM_SUBMIT_HOST | Host submitted from | login-1.zaratan.umd.edu | PBS_O_HOST |
| SLURM_TASKS_PER_NODE | Comma-delimited list of integers representing the tasks per node. Nodes use the same ordering as SLURM_JOB_NODELIST. If consecutive nodes have the same task count, the integer may be followed by (xN) |
2(x3),1 | |
| SLURM_ARRAY_JOB_ID | The base job number shared by all elements of a job array | 11959351 | |
| SLURM_ARRAY_TASK_ID | Our job's index in the job array, unique for each element of the array | 19 | |
| SLURM_ARRAY_TASK_COUNT | The number tasks in a job array | 100 | |
| SLURM_ARRAY_TASK_MIN | The minimal index in the job array | 1 | |
| SLURM_ARRAY_TASK_MAX | The maximal index in a job array | 100 |
The list of nodes in the SLURM_JOB_NODELIST environmental
variable uses a compact notation in which square brackets
(i.e. [ and ]) are used to delimit lists and/or ranges
of numeric values. You can use the
scontrol show hostnames
command to expand that to a list of hostnames.
The script slurm_hostnames_by_tasks can be used to expand SLURM_JOB_NODELIST and SLURM_TASKS_PER_NODE into some formats which you might find more useful.