This is an old revision of the document!
SLURM
Conceptos básicos
Jobs
Particiones
Task
Comandos básicos
Consultar cola
Enviar un programa
Cancelar un trabajo
Trabajando con SLURM
Simple usage for soroban 0. Note. '-p intel' (–partition=intel) is required for soroban.
1. Save below as text file (e.g. my_first_slurm.sh).
#!/bin/bash #SBATCH --job-name=example # Nombre para el trabajo a ejecutar en el cluster #SBATCH --partition=intel #SBATCH -n 32 # Debe de ser un número múltiplo de 16 #SBATCH --ntasks-per-node=16 # máximo por blade #SBATCH --output=example_%j.out #SBATCH --error=example_%j.err ls -lh pwd
2. Submit it as SLURM job. sbatch <filename> (e.g. sbatch my_first_slurm.sh)
3. Check progress. squeue
Ejecutando un programa con openMPI , usando un script base para SLURM:
#!/bin/bash #SBATCH --job-name=example # Nombre para el trabajo a ejecutar en el cluster #SBATCH --partition=troquil #SBATCH -n 32 # Debe de ser un número múltiplo de 16 #SBATCH --ntasks-per-node=16 # máximo por blade #SBATCH --output=example_%j.out #SBATCH --error=example_%j.err #SBATCH --mail-user=username@ufrontera.cl #SBATCH --mail-type=ALL srun ./mpi_programa
List of available clusters and partitions
List of available clusters
List of available partitions
FAQ (Frequently Asked Questions)
Q. What is the difference of cluster and particion?
Q. I always use only one cluster. Is there any way to omit –clusters=[cluster_name] when I check/delete jobs by scontrol/scancel?
Further information
Use 'man' command after login to servers
man sbatch
SLURM commands
- sacct
- salloc
- sbatch
- scancel
- scontrol
- squeue
- sreport
Useful reference pages
About job array
https://slurm.schedmd.com/job_array.html
https://rcc.uchicago.edu/docs/running-jobs/array/index.html
https://www.accre.vanderbilt.edu/wp-content/uploads/2016/04/UsingArrayJobs.pdf
How to rewrite PBSPro/SGE script to SLURM script
Common command
| PBS command | SGE command | SLURM command | |
| Job submission | qsub [scriptfile] | qsub [scriptfile] | sbatch [scriptfile] |
| Job deletion | qdel [job_id] | qdel [job_id] | scancel –clusters=[cluster_name] [job_id] |
| Job status (for user) | qstat -u [username] | qstat -u [username] | squeue -u [username] |
| Extended job status | qstat -f [job_id] | qstat -f -j [job_id] | scontrol –clusters=[cluster_name] show jobid=[job_id] |
| Hold a job temporarily | qhold [job_id] | qhold [job_id] | scontrol hold [job_id] |
| Release job hold | qrls [job_id] | qrls [job_id] | scontrol release [job_id] |
| List of usable queues | qstat -Q | qconf -sql | sinfo, squeue |
Resource speciation
| PBS command | SGE command | SLURM command | |
| Queue | #PBS -q [queue] | #$ -q [queue] | #SBATCH -M=[queue] / #SBATCH –clusters=[queue] |
| Processors (Single host) | #PBS -l select=1:ncpus=[#] | #$ -pe smp [#] | #SBATCH -c=[#]? |
| Wall clock limit | #PBS -l walltime=[hh:mm:ss] | #$ -l time=[hh:mm:ss] | #SBATCH -t=[#] (“minutes”, “minutes:seconds”, “hours:minutes:seconds”, “days-hours”, “days-hours:minutes” and “days-hours:minutes:seconds”) |
| Memory requirement | #PBS -l mem=XXXXmb | #$ -mem [#]G | #SBATCH –mem=[#][unit;K/M/G/T]? |
| Standard output file | #PBS -o [file] | #$ -o [path] | #SBATCH -o [path] |
| Standard error | #PBS -e [file] | #$ -e [path] | #SBATCH -e [path] |
| Array job | #PBS -J [#-#] | #$ -t [#-#] | #SBATCH -a=[#-#] |
| Array number Variable name | ${PBS_ARRAY_INDEX} | ${SGE_TASK_ID} | ${SLURM_ARRAY_TASK_ID} |
| Max simulaneously running tasks for an array job | n/a? | #$ -tc [#] | #SBATCH -a=[#-#]%[#] (e.g. -a=0-15%4) |
| Copy environment | #PBS -V | #$ -V | #SBATCH –get-user-env |
| Notification event | #PBS -m abe | #$ -m abe | ? |
| Email address | #PBS -M [email] | #$ -M [email] | #SBATCH -M [email] |
| Job name | #PBS -N [name] | #$ -N [name] | #SBATCH -J [name] |
| Job restart | #PBS -r [y/n] | #$ -r [yes/no] | #SBATCH –requeue / #SBATCH –no-requeue? |
| Move current directory | n/a | #$ -cwd | ? |
| Move working directory | n/a (in the main part of script, add cd ${PBS_O_WORKDIR}) | #$ -wd | #SBATCH -D [working_dirpath] |
| Use BASH | #PBS -S /bin/bash | ? | shebang line (At the first line of the script, add #!/usr/bin/bash |