Available Software

Contents

1. The module command
   1. The module command
   2. Library packages
   3. Common errors
2. Research Software Library Versions
3. Software packages available
   1. Unclassified by field
   2. Classified by field, etc
      1. Packages related to Aeronautics
      2. Packages related to Astronomy
      3. Packages related to Biology and biological systems
      4. Packages related to Computional Fluid Dynamics
      5. Packages related to Chemistry
      6. Packages related to Computer Vision
      7. Packages related to Data Compression
      8. Packages related to specific Data Formats
      9. Packages related to various Earth sciences
      10. Packages related to various Engineering fields
      11. Packages related to Genomics and Genetics
      12. Packages related to Image and/or Video processing
      13. Packages related to Machine Learning
      14. Packages related to Material Science
      15. Packages related to Meteorology
      16. Miscellaneous Packages
      17. Packages related to Molecular Dynamics
      18. Packages related to Multiphysics
      19. Packages related to Neuroscience
      20. Packages related to Numerical Methods
      21. Packages defining a Python Module
      22. Packages for Software Development (compilers, scripting languages, etc.)
      23. Basic Utility Packages
      24. Packages related to Visualization and Graphics
   3. Available by cluster
      1. All packages available on the Zaratan cluster
4. Packages for scripting languages
   1. Packages/modules for Python
   2. Packages/modules for R
   3. Packages/modules for Perl

The module command

A large number of software packages are available on each of the clusters, and for some of them, multiple versions of the same package. Not all of them like to co-exist with each other, so many of the packages are not available by default. You need to tell the system you want to use them with the module command.

For example, matlab is available on the HPC clusters, but only after you module load it. E.g.

login-1: matlab
matlab: Command not found.
login-1: module load matlab
login-1: module help matlab

----------- Module Specific Help for 'matlab/2014b' ---------------


    This is Matlab Release 2014b

    Releases after 2009 require Red Hat 5 or later.
    Releases after 2013 require Red Hat 6 or later.
    Matlab 2009b is the last release supported under Solaris.

    Run command 'matlab' to start up the program,
    or 'matlab -h' to see various command-line options.

    ***  To see all releases, issue the command
	module avail matlab

login-1: matlab
                              < M A T L A B >
                  Copyright 1984-2014 The MathWorks, Inc.
                   R2014b (8.4.0.150421) 64-bit (glnxa64)
                             September 15, 2014


  To get started, type one of these: helpwin, helpdesk, or demo.
  For product information, visit www.mathworks.com.

>> 

You can use the module help and module whatis commands before (or without) loading the package.

The module command

The module command associates a tag with the various packages. It has a hierarchical nature, so we have tags like matlab/2013b and matlab/2011a in addition to plain old matlab. The former specify a specific version of matlab, whereas the latter will enable the default version of matlab.

With the roleout of a new software library with the RHEL8 upgrade of the Deepthought2 HPC cluster, we are no longer keeping the "default" versions of software packages fixed over long periods of time. Instead, if you do not provide a version in your "module load" command, you will almost always get the latest installed version compatible with other modules you have loaded. (There will be rare and brief exceptions if we find problems in the latest version of a package). This is a change from the behavior of many packages on the RHEL6 version of Deepthought2. For reproducibility over time, it is strongly recommended that in job submission batch scripts you provide the full version specification for any packages for which you care about the version number.

For many packages, the tag is basically just the base name of the package, followed by a slash (/) and a version number. E.g., the examples matlab/2013b and matlab/2011a refer to versions 2013b and 2011a, respectively, of matlab. You can also just use matlab as the tag when e.g. loading a module, in which case you will get one of the versions (usually the latest, sometimes we pin to an earlier version if the newest version is still being tested).

Many packages compiled from source will have different builds depending on what CPU microarchitecture they were compiled for. Running a code compiled for a more modern CPU than you are currently using will likely result in the code crashing; running a code compiled for a less modern CPU will generally work, but you might lose some of the optimizations available on the newer chip. Generally, these packages will have two equivalent module tags, one of the form "foo/VERSION/ARCHITECTURE" and one of the form "foo/ARCHITECTURE/VERSION"; both forms load foo version VERSION built for architecture ARCHITECTURE. The module load command will attempt to load the build for the most advanced architecture compatible with the CPU you are running on. If you omit the version, it will look for the latest version built for the most advanced CPU architecture compatible with the CPU you are running on.

Some packages are fussy about the compiler and/or MPI libraries they are used with. This is particularly true of libraries which have fortran90 and/or C++ components; some examples are openmpi, hdf5, and netcdf. In these cases, the package will again have multiple module tags referring to the same build of the software. The first starts like the simple case, with the base name of the package followed by the version of the package, but to this is appended the other dependencies (like compiler family and version, and perhaps MPI library family and version, etc.). For example, one tag for version 1.8.13 of hdf5, built with version 4.8.1 of the GNU compiler suite and version 1.8.1 of , with shared linkage would be:

hdf5/1.8.13/gnu/4.8.1/openmpi/1.8.1/shared  

There is also an alternative form giving the compiler, MPI library, and linkage portions of the tag immediately after the base package name, and ending with the version of the package, e.g.

hdf5/gnu/4.8.1/openmpi/1.8.1/shared/1.8.13

Both of these tags do exactly the same thing, and set up exactly the same build of hdf5, when loaded. You can use either.

The advantage of the module command is that you generally do NOT need to give these long tag names, but can leave off components on the right and it will try to do the right thing. In particular, if you try loading just the base package name, e.g. module load hdf5, the system will determine if you have previously loaded a compiler and/or MPI library. If so, then the system will see if there are any builds of any version of the package matching the compiler and MPI library versions you previously loaded, and if one is found load the build for the most recent version that matches. If no matching build is found, and error will be displayed and no module loaded.

The following except gives an example of this:

login-2:> module purge 
login-2:> module list 
No Modulefiles Currently Loaded.
login-2:> module load gcc/4.8.1
login-2:> module load openmpi/1.8.1
login-2:> module load hdf5
login-2:> module list
Currently Loaded Modulefiles:
  1) gcc/4.8.1(default)
  2) openmpi/1.8.1/gnu/4.8.1
  3) hdf5/gnu/4.8.1/openmpi/1.8.1/shared/1.8.13
login-2:> 
login-2:> module purge
login-2:> module list
No Modulefiles Currently Loaded.
login-2:> module load intel/2016
login-2:> module load openmpi/1.10.2
login-2:> module load hdf5
login-2:> module list
Currently Loaded Modulefiles:
  1) ofed/3.12(default)
  2) intel/2016.3.210(default)
  3) openmpi/1.10.2/intel/2016.3.210
  4) hdf5/intel/2016.3.210/openmpi/1.10.2/shared/1.10.0
login-2:> 
login-2:> module purge
login-2:> module list
No Modulefiles Currently Loaded.
login-2:> module load hdf5/1.8.15p1/sunstudio/12.4/openmpi/1.8.6/shared
login-2:> module list
Currently Loaded Modulefiles:
  1) sunstudio/12.4(default)
  2) openmpi/1.8.6/sunstudio/12.4
  3) hdf5/1.8.15p1/sunstudio/12.4/openmpi/1.8.6/shared

We start by issuing a module purge command to delete any previously loaded modules (in general, you should use this with caution. If you do use it, you should immediately load the modules hpcc/deepthought or hpcc/deepthought2, as appropriate for the cluster you are on, and slurm. These are the default modules on the Deepthought HPC clusters and things might not work properly without them). We then do a module list to confirm that there are no modules loaded. We then load a specific version of the gcc compiler (4.8.1) and the openmpi libraries (1.8.1) and then ask the system to load the best hdf5 module. We then list what was loaded, and note that the system loaded an openmpi build (version 1.8.1) for the compiler we loaded, and a hdf5 version (1.8.13) for the previously loaded compiler and MPI library.

In the second example above, we load the 2016 version of the intel compiler suite, and the 1.10.2 version of openmpi, and when we issue the simple module load hdf5, this time it loads hdf5 version 1.10.0 built for that compiler and openmpi library. (Note also that the openmpi library loaded was also built for the correct compiler.)

In the final example above, we do not explicitly load any compiler or MPI library, but instead give a full path the the hdf5 module, e.g. hdf5/1.8.15p1/sunstudio/12.4/openmpi/1.8.6/shared. In this case, we note that not only was the requested hdf5 module loaded, but we also loaded the corresponding compiler (sunstudio/12.4) and openmpi (openmpi/1.8.6) modules as well.

In summary, in almost all cases, you need only specify a portion of the tag, and the system will try to default it properly. Generally, you should load the compiler you wish to use first (or do not load anything if you want to use the default gcc 4.6.1 compiler) followed by the MPI library if needed. This will allow the system to pick the correct build of libraries which depend on such. Alternatively, if you specify the full path to a package which depends on a specific compiler and MPI library, these will be loaded for you if not already loaded. In general, if the package has dependencies, they will be loaded automatically for you if needed.

However, if you want reproducable results, it is generally recommended that you fully specify version numbers when loading modules in your job scripts. I.e., the latest version of the foo package might be 1.5 when you run your job today, but if you run it again a month from today, the latest version might by 1.7 at that time, and that could change the results unexpectedly if your script merely did a module load foo. If you wish to ensure the same version is always used, specifying the version number in the module load command is advised, e.g. module load foo/1.5; this way you will get the same version of the foo package both today and a month from now. But if you always prefer to use the latest package available, then in general you should leave the version numbers off.

NOTE: If you are submitting a job using a bourne style shell (e.g. sh or bash), your dot files will not be read automatically if your default shell is something else (e.g. csh or tcsh). So you will need to include a . ~/.bashrc line near the top of your script, after the #SBATCH lines but before you try to use tap or module commands.

The module command allows you to issue commands of the form module SUBCOMMAND OPTIONAL_ARGUMENT. Recognized subcommands include:

• list: This will list the packages you currently have loaded. Note that if a tap entry was migrated to modules, it will show up here if you loaded it using tap.
• avail OPTIONAL_TAG: this will list all the packages available. If a tag is given, it will list all packages which match the tag. Note the matching is purely lexical, so module avail netcdf will pick up all netcdf modules, as well as netcdf-fortran and netcdf-cxx modules. To restrict to just netcdf modules, use something like module avail netcdf/ (note the final /)
• load TAG: this is the most common usage, and it sets up the environment for the application associated with TAG. Again, if you do not specify the full tag, the system will attempt to default it, and usually it can do that with some success. Most of our application definitions will pull in any dependencies as well. Note that unlike tap, this is a silent operation when it succeeds.
• help TAG: This displays a brief intro message about the package. Generally the same message you see when you tap a package. It is advised that you read this once for each package you use.
• unload TAG: This unloads the package; i.e. all the environment that was set up for the package is returned to what it was before you loaded it. This is not 100% successful all the time, but for the most part works. You generally do not need this unless you which to change the version of an application you are using.
• purge: This unloads ALL currently loaded modules. This should be used sparingly. If you do use it, please follow it with the loading of either hpcc/deepthought or hpcc/deepthought2 (as appropriate for the cluster you are on) and slurm. These modules are loaded for you automatically when you log into the clusters, and if you remove them (e.g. with the purge command), things might not work as expected.

Please note that the HPC compute nodes have abridged copies of the software directories available on other Glue/TerpConnect systems. The HPC login nodes have the full software directories. Please look at software lists below, and in particular click on the software package name for the full breakdown by version number to see which, if any, HPC clusters a specific version of the package is on.

Library packages

Some packages are not so much applications as libraries; i.e. they provide libraries which your code can use, and might or might not have any binaries that you can run directly. To make use of these libraries in your code takes a little bit of work. Please see the section on compiling codes for more information on how to do this.

Common errors from module/tap commands

As stated early, not all packages co-exist nicely. This is particularly true of different versions of the same package, or related libraries (e.g. the various MPI libraries). The module command will generate an error if you try to load a package which does not play well with packages you have previously loaded; if this occurs you will need to unload the previous package and reload the new package. E.g.

login-1: module load matlab/2012b
login-1:~: module load matlab/2013b
matlab/2013b(8):ERROR:150: Module 'matlab/2013b' conflicts with the currently loaded module(s) 'matlab/2012b'
matlab/2013b(8):ERROR:102: Tcl command execution failed: source $moduledir/common
login-1: module unload matlab
login-1:~: module load matlab/2013b

Although not as user friendly as one would like, you can see that the first time I tried to load matlab/2013b it complained because matlab/2012b was already loaded. Also note that I did not need to specify the full tag to unload matlab/2012b, as there was only one matlab package loaded, so it could default it.

You can also run into similar incompatibillity problems if you mix packages (especially library packages) built for one compiler/MPI library/etc with a different compiler/MPI library/etc. For example

login-1: module avail fftw/3.3.3

------------------- /cell_root/system/common/modulefiles/sys -------------------
fftw/3.3.3/gnu/4.6.1/openmpi/1.6/shared
fftw/3.3.3/gnu/4.6.1/openmpi/1.6/static
fftw/3.3.3/gnu/4.6.1/openmpi/1.6.5/shared
fftw/3.3.3/gnu/4.6.1/openmpi/1.6.5/static
fftw/3.3.3/intel/2013.1.039/openmpi/1.6.5/shared
fftw/3.3.3/intel/2013.1.039/openmpi/1.6.5/static

login-1: module load intel/2013.1.039

login-1: module load fftw/3.3.3/gnu

***** Compiler mismatch

You have loaded compiler intel/2013.1.039
This package wants compiler gnu/4.6.1
login-1: 

In the above, we show that there are various builds of fftw/3.3.3 available. We then load the intel compiler module, and try to load a GNU compiler build of fftw/3.3.3, and get an error. That is because using a GNU build of fftw/3.3.3 with the intel compiler suite is likely not to work and cause you headaches. In general, you should not be specifying more than fftw/3.3.3 anyway, and the module command will figure out which build to load based on which compilers you already have loaded. Alternatively, you can give the full path (e.g. module load fftw/3.3.3/intel/2013.1.039/openmpi/1.6.5) and the module command will auto-load the appropriate intel compiler and openmpi module for if, if it can. If you have a conflicting module already loaded, you will still get the error.

For the most part, tap is just a wrapper around the module command for backwards compatibility, and so you will get the same errors with tap if you try to load conflicting modules. The tap command is limited, so you generally do NOT get a whole lot of choice as to which builds of packages get loaded, so it is less likely to be a problem, but it can still arise. Some packages have not been migrated from tap to module command, and these will NOT generate such errors when you tap incompatible modules. But things will probably NOT work as you expected; the incompatibility is REAL, not an artifact of the module command.

Research Software Library Versions

The Ressarch Software Library tries to present a set of packages which are generally compatible with each other (within a given tool chain). Periodically, we will do a major upgrade of the library, deprecating some older software versions and introducing some new packages and newer versions of some packages. To minimize any adverse affect from these upgrades, we can maintain several versions of the software library:

1. production: This is the default version of the library which you see if you do not explicitly select another version. It is the production version, and is the only version which is fully supported.
2. devel: This is the version of the library being built to replace production. As the name implies, it is still being actively developed and so is likely not as stable as the production version. Once development is finished, is will get promoted to the new version.
3. new: This refers to the new version of the software library we are proposing to replace the current production version. We will make the new version available for some time under this tag before promoting it to production.
4. old: This refers to the version of the software library which previously was the production version. When we promote a new version to production, we will make this tag point to the version that was replaced.

Note: although the four tags will exist, they usually will not refer to four distinct versions of the library. The command module help umd-software-library will display which tags point to which versions (which are typically given as the calendar date when we started to build out that version of the library).

You can use the command module switch umd-software-library/TAG to switch between versions of the software library, where TAG is one of production, devel, new, or old. (The "switch" subcommand of module essentially unloads any previously loaded version of the package and then loads the specified version).

Note that while we try to keep all of the packages within a given library version compatible with each other, that is not the case between library versions. It is strongly advised that you do not mix packages from different library versions --- doing such is not supported and very likely will cause problems.

The existance of and ability to select different versions of the software library is intended to facilitate the upgrade of the software library with minimal adverse impact to users. When we have a new version of the software library ready for testing (and/or ready for use by users waiting for an upgraded package), it will be made available to users under the devel tag. Once development has stabilized, it will be made available to users under the new tag as well. This will allow users to test things out and/or access the new packages they desire, without affecting any users still using the production version.

Once we believe the new version is working properly, we will announce a switch over date. On the switch over date, the production tag will point to the new software library, and old will point to the old production version. Hopefully, most users will just pick up the new builds and work without issue. Some users may need to modify their workflows due to the change in library version --- these changes should hopefully be small and minor, but if not they can revert to the old software library version until they have time to work out the issues. Note that the old version will persist only until the development cycle begins again.

List of available software packages

In this section we list the various software packages available on the UMD HPC clusters. We start with a list of all packages, unclassified by field of research, and follow with some shorter lists for various fields of research. If there is a mistake in the classification of software (e.g. a package is listed in the wrong category, or is missing from a category where it belongs), please contact systems staff so that we may correct the situation. Similarly, contact us if there is a category for software which is missing and you believe would be useful to have.

For each list, you can click on the name of the package to bring up a page with more detailed information on what versions are available on which clusters, and some basic (and sometimes not so basic) usage and help. Note that all packages, much less all versions of all packages, are not necessarily available on all clusters, so please verify that te package and/or version of the package you need is available on the cluster you intend to use.

1. All packages (unsorted by field)
2. Software lists classified by field of research, etc
   1. Packages related to Aeronautics
   2. Packages related to Astronomy
   3. Packages related to Biology and biological systems
   4. Packages related to Computional Fluid Dynamics
   5. Packages related to Chemistry
   6. Packages related to Computer Vision
   7. Packages related to Data Compression
   8. Packages related to specific Data Formats
   9. Packages related to various Earth sciences
   10. Packages related to various Engineering fields
   11. Packages related to Genomics and Genetics
   12. Packages related to Image and/or Video processing
   13. Packages related to Machine Learning
   14. Packages related to Material Science
   15. Packages related to Meteorology
   16. Miscellaneous Packages
   17. Packages related to Molecular Dynamics
   18. Packages related to Multiphysics
   19. Packages related to Neuroscience
   20. Packages related to Numerical Methods
   21. Packages defining a Python Module
   22. Packages for Software Development (compilers, scripting languages, etc.)
   23. Basic Utility Packages
   24. Packages related to Visualization and Graphics
3. Software lists by cluster
   1. All packages available on the Zaratan cluster

All packages (unsorted by field)

This section list all installed research software, unsorted by field of research/category/etc.

Packages related to Aeronautics

This section lists various software packages related to Aeronautics, Aviation, Aerospace Sciences and related fields (Astronautics, Hypersonics, Aerothermodynamics, etc.)

Packages related to Astronomy

This section lists various software packages related to Astronomy and related fields.

Packages related to Biology and biological systems

This section lists various software packages related to Biology, Bioinformatics and related fields. See also Genomics.

Packages related to Computional Fluid Dynamics

This section lists various software packages related to Computational Fluid Dynamics (CFD).

Packages related to Chemistry

This section lists various software packages related to Chemistry, Computational Chemistry, Quantum Chemistry, etc. See also MolecularDynamics and MaterialScience.

Packages related to Computer Vision

This section lists various software packages related to Computer Vision and related fields. You might also want to look at the section on packages related to Machine Learning.

Packages related to Data Compression

This section lists various software packages related to Data Compression. Basically libraries and binaries for compressing and uncompressing data to/from various compressed formats and/or compression algorithms.

Packages related to specific Data Formats

This section lists various software packages related to various scientific or other data formats, including libraries and binaries for reading, writing, and manipulating data in specific formats.

Packages related to various Earth sciences

This section lists various software packages related to various Earth sciences. See also Meteorology.

Packages related to various Engineering fields

This section lists various software packages related to various Engineering fields.

Packages related to Genomics and Genetics

This section lists various software packages related to Genomics, Genetics, and related fields.

Packages related to Image and/or Video processing

This section lists various software packages related to the processing/editting/manipulation/etc of image or video files.

Packages related to Machine Learning

This section lists various software packages related to Machine Learning, Deep Learning, Neural Nets and related fields. You might also wish to look at the section on Computer Vision

Packages related to Material Science

This section lists various software packages related to Material Science and related fields. See also Chemistry and SolidStatePhysics.

Packages related to Meteorology

This section lists various software packages related to Meteorology, Weather, Atmospheric and Oceanographic Sciences and related fields. See also Earth Sciences

Miscellaneous Packages

Miscellaneous packages. Libraries and other stuff which we need a modulefile for but do not belong anywhere else and do not merit there own category.

Packages related to Molecular Dynamics

This section lists various software packages related to Molecular Dynamics and related fields. See also Chemistry and Material Science.

Packages related to Multiphysics

This section lists various software packages related to Multiphysics.

Packages related to Neuroscience

This section lists various software packages related to Neuroscience and related fields liek brain imaging, etc.

Packages related to Numerical Methods

This section lists various software packages related to Numerical Methods and Computational Mathematics. It includes both underlying numerical libraries as well as advanced computational (and often GUI) environments. These packages are generally useful across a wide range of disciplines.

Packages defining a Python Module

This section lists various software packages containing python modules which can be imported in your python code, after the corresponding environmental module is loaded.

Packages for Software Development (compilers, scripting languages, etc.)

This section lists various software packages related to the development and optimization, etc. of software and codes. It includes compilers, debuggers, profiling tools, as well as relatively low level libraries. It also includes various scripting languages.

Basic Utility Packages

This section lists various utility software packages, etc. This is sort of a catch-all category for packages which are useful across a wide range of disciplines but do not really fall into any discipline. These are often just newer versions of utilities supplied by the OS.

Packages related to Visualization and Graphics

This section lists various software packages related to Visualization; basically packages to assist in visualizing data generated from other packages/codes. It includes both basic graphics libraries and high end (GUI) visualization platforms.

All packages available on the Zaratan cluster

This section lists all software packages available on the Zaratan cluster

Packages installed for scripting languages

The Division of Information Technology maintains several versions of several scripting languages on the clusters. These scripting languages can be enhanced by the addition of packages/modules/etc, and the Division of IT maintains a collection of packages for these. In most cases, the packages/modules are fairly lightweight, and we simply install them into the standard system directory location for the particular scripting language, upgrading them as needed. This means that they will just be found when you import/use/etc them; no additional work is needed on your part.

For a few cases, they are major packages in their own right, and we give them their own module. Those packages are included in the lists of software earlier in this document, and you will need to "module load" the appropriate package in order to use them.

This section is for the packages of the first type, and for the convenience of our users we provide a list of the packages/modules available for each install of a scripting language, by cluster and version. Note: that these lists do not include the standard library for the scripting language, i.e. those packages/modules that get included automatically in the installation of the scripting language, only those that need to be installed separately.

These sections are still under development. We hope to have it more accurate in the near future.

1. Modules/packages for Python
2. Modules/packages for R
3. Modules/packages for Perl

Packages/modules for Python