r: R statistical computing scripting language

Overview of package

Package:	r
Description:	R statistical computing scripting language
For more information:	https://www.r-project.org
Categories:	SoftwareDevelopment
License:	OpenSource (GPL)

General Usage

R is a language and environment for statistical computing and graphics. It is similar to the S language and environment, and can be considered as an open-source implementation of S. There are some important differences, but much code written for S runs unaltered under R.

This module will add the R and Rscript commands to your path.

In case you need to link against this library in your code, the following environmental variables have been defined:

$R_ROOT has been set to the root of the r installation
$R_LIBDIR points to the directory containing the libraries
$R_INCDIR points to the directory containing the header files

You will probably wish to use these by adding the following flags to your compilation command (e.g. to CFLAGS in your Makefile):

-I$R_INCDIR

and the following flags to your link command (e.g. LDFLAGS in your Makefile):

-L$R_LIBDIR -Wl,-rpath,$R_LIBDIR

Available versions by cluster

This section lists the available versions of the package ron the UMD HPC clusters.

Versions available on the Zaratan cluster

Version	Module tags	CPU(s) optimized for	GPU ready?
4.3.2	4.3.2 r/4.3.2/gcc/11.3.0/nocuda/linux-rhel8-icelake	icelake	Yes
4.3.2	4.3.2 r/4.3.2/gcc/11.3.0/nocuda/linux-rhel8-zen2	zen2	Yes
4.3.2	4.3.2 r/4.3.2/gcc/11.3.0/nocuda/linux-rhel8-x86_64	x86_64	Yes
4.1.1	4.1.1 r/4.1.1/gcc/9.4.0/nocuda/linux-rhel8-zen2	zen2	Yes

System installed R packages/modules

The system installations of R include a large number of R modules to enhance the capability of R. The table below list those modules and version numbers for the various installations of R which are made available with the "module load r" or similar commands.

Packages/modules for R

The following table lists the various packages/modules/extensions for R along with their version which are installed on the UMD HPC clusters, by cluster, Rversion, and platform. These are the packages/modules/extensions which are enabled by default when you do the "module load r".

R Version
Compiler
CPU optimized for
CUDA

Installing Modules

R's capabilities can be significantly enhanced through the addition of modules. Code can then enable the library with the library command.

The R builds provided as part of the Research Software Library (RSWL) have a large selection of installed R modules. f a module you are interested in is not in that list, you can either install a personal copy of the module for yourself, or request that it be installed system wide. Installation of software packages by HPC staff is done on a "as time allows" basis --- HPC staff have many other duties with higher priorities, but will make reasonable efforts to accomodate such requests as staffing resources allow. We will also consider how widely used the package is expected to be, and whether it is designed for system or individual installation, etc.

Installing modules yourself

Using R CMD INSTALL

The recommended method for installing your own R packages is with the R CMD INSTALL command or similar. This method for installing R packages is usually fairly straightforward, and generally allows for you to install your own packages while still availing yourself of system installed packages. Obviously not all packages will install in the same manner. But most will follow the procedure below:

module load R/X.Y.Z to select the version of R you wish to use.
Create the directory to hold your R modules, if you have not already done so. The default is in the directory R underneath your home directory, but you might wish to put it elsewhere (e.g. in your scratch directory); ; this will have subdirectories for R version and platform added.
Unless you opted for the default directory ~/R, you need to tell R what directory you are using. To do this, you must set the environmental variable R_LIBS_USER. Multiple directories can be listed; separate the paths with the colon (:) character. This needs to be set whenever you wish to use the modules in R, so you will generally want to set it in your .bashrc or .Renviron or similar files.
There are two standard methods for installing a package, one from the command line, and one from inside R itself. Assuming you are putting stuff in ~/myRpkgs and installing the package foo the commands would be: - From the command line, you will first need to download a tarball with the source code for the package. Many packages can be found at the Comprehensive R Archive Network (CRAN). Assuming you downloaded foo.tar.gz to the current directory, you could then install it with:
```
R CMD INSTALL -l ~/myRpkgs foo.tar.gz
```
- From within R, the install.packages function will connect to CRAN and download and install the package all in one step, with:
```
install.packages("foo", lib="~/myRpkgs", repos="http://cran.r-project.org")
```

If all goes well, the package is now installed in the directory you specified and should be available for use by your R scripts.

Of course, not all packages install quite that easily. If you are comfortable building modules, hopefully the error messages will provide reasonable guidance as to how to proceed. Otherwise, you can just request for Division of Information Technology staff to install it, but that might take time depending on the availability of our time.

Using conda

Note

We generally recommend that you try installing your own R packages using the previously mentioned R CMD INSTALL method rather than using conda as that integrates better with the system installed R packages.

Conda is another option for installing python and R packages, but it does even more separation and isolation than the venv method. This can either be an advantage or a liability depending on the use case. This isolation means that you are not constrained but what system staff have installed on the cluster, but it also means it is very difficult if not impossible to take advantage of any packages system staff have installed on the cluster. Also, since you are relying on an entire environment which you installed, the ability of HPC staff to provide assistance should there be issues is limited. Therefore, we encourage users to try one of the other methods if such is feasible.

Warning

To avoid risking violating the Anaconda terms of service, we strongly recommend users at UMD use the Miniforge/Mambaforge installers for Conda.

For more information, see: - The UMD HPC Conda page - Conda Installation instructions - Installing packages using Conda

Running R in batch mode

Although R's interactive mode is nice for certain things, when you are doing production runs with tried and true scripts, it is usually easier to use R's batch interface. This is especially useful when submitting jobs to an HPC cluster.

If you have some R code in a file test.R and you wish to run it from the command line (or equivalently, from a shell script), you can simply use the Rscript command. E.g.

Rscript --no-save --no-restore test.R

The --no-save and --no-restore prevent the saving of the workspace at the end of the session and the restoring of saved objects at startup. These are typically what you want when running in batch mode. Older versions of R used the R CMD BATCH instead of the Rscript command; the main difference with the former is that it optionally takes the name of an output file. Both should work with currently installed versions of R.

For use on one of the HPC clusters, you will generally need to include the above in a job script, like:

#!/bin/bash
#Request 5 hours
#SBATCH -t 5:00
#Request 4 GiB per CPU-core
#SBATCH --mem-per-cpu=4096
#Request 1 core
#SBATCH -c 1

#Get our profile (and define module command)
source /etc/profile

#Load R
module load R

cd *MY_WORK_DIRECTORY*

#Make sure OpenMP is effectively disabled
OMP_NUM_THREADS=1
export OMP_NUM_THREADS

Rscript --no-save --no-restore test.R

Using R and MPI

User of one of the high-performance computing (HPC) clusters will likely be interested in running R codes that span multiple processors often over multiple nodes. This generally is done using MPI There are a number of R packages that deal with MPI, including

Rmpi
snow
doSNOW: provides a dopar functionality via snow

Most users seem to prefer the snow package, which is presumably higher level and therefore easier to use than Rmpi. There are assorted guides to using R with the snow package on the web, including:

Below are just a few tips gleaned from these pages, etc. that users at UMD might find helpful.

For best results, use the same version of compiler and MPI as used for building R and its MPI packages. The MPI libraries and compiler used for the different versions of R are listed in the package version table. It is best to module load the compiler first and then the OpenMPI library.
We have also had reports of wierd errors occurring when using Rmpi (and the packages depending on it) with Infiniband; segfaults and other seemingly random errors when setting up connections. This appears to be related to complications with the used of pinned memory and forking within the R interpretter (see e.g. CRMDA blog and OpenMPI developers mailing list archives regarding this issue). As such, we strongly recommend R users who wish to use MPI disable Infiniband in their mpirun command by adding the arguments --mca btl tcp,self as shown in the example below.
When using snow or one of its derivatives (e.g. doSNOW), you should launch your code with something like #!/bin/bash #Request 5 hours #SBATCH -t 5:00 #Request 4 GiB per CPU-core #SBATCH --mem-per-cpu=4096 #Request 40 cores #SBATCH -n 40
```
#Get our profile (and define module command)
. ~/.profile

#Load required modules
module load gcc/4.9.3
module load openmpi/1.8.6
module load R/3.3.2

cd <var>MY_WORK_DIRECTORY</var>

#Make sure OpenMP is not "on"
OMP_NUM_THREADS=1
export OMP_NUM_THREADS

#NOTE THE -np 1 below!!!!
#The --mca btl tcp,self arguments restricts communications to
#tcp instead of infiniband.  We have seen issues with Rmpi and infiniband
mpirun -np 1 --mca btl tcp,self R CMD BATCH --no-save --no-restore <var>my_R_code.R</var>
```
NOTE: the use of -np 1 in the above. Although that looks suspicious (telling mpirun to only start one MPI tasks when we asked for 40 cores), it is actually correct for most uses of the snow (and derivative) libraries. This is because when using snow, typically snow will spawn its own workers. If you request something more than 1 MPI task to be launched via the openmpi, or omit the -np 1 altogether (which effectively is asking for mpirun to launch the number of tasks given in the #SBATCH -n line, 40 in this case), you will end up running e.g. 40 copies of the same code, each of which will try to spawn about 40 workers via snow, resulting in a mess (at best very sluggish performance, and more likely wierd errors).
Most snow based R code will at some point invoke the makeCluster function. This takes a parameter specifying the size of the "cluster" to create. Typically, one wants this size to be one less than the number of cores requested from Slurm. This is because the process running the R code which spawns the workers is already consuming one CPU core, so if you try to spawn a number of workers equal to the number of cores requested of Slurm, there will be one core oversubscribed, which causes issues. I typically see an error about there being an insufficient number of "slots" available, and typically the R script just hangs (doing nothing, but not dying until the job is killed for exceeding its walltime, and thereby wasting a lot of SUs). Typically, it is better to do something like: cl<-makeCluster(mpi.universe.size()-1, type="MPI")