Quick Start (using the OnDemand web portal)

This is a "quick start" introduction into using the HPC clusters at the University of Maryland using the OnDemand Web Portal. Although the OnDemand portal is generally not as powerful as using the Linux command line, those not familiar with the Linux command line will usually find that the OnDemand portal has a much gentler learning curve.

Note: The OnDemand portal is only supported on the Deepthought2 and Juggernaut clusters at UMD. It is not supported on the MARCC/Bluecrab cluster at this time.

  1. Prequisites
  2. Connecting to the OnDemand Portal
  3. Viewing/Editing/Managing Files
  4. Interactive Jobs and GUIs
  5. Batch Jobs
  6. Monitoring job status, etc
  7. Monitoring your allocation

Prequisites for the OnDemand Portal Quick Start

This quick start assumes that you already

  1. have a TerpConnect/Glue account (REQUIRED for the Deepthought clusters, advisable for the others)
  2. have an account on/access to the cluster (i.e., have an allocation or have been granted access to someones allocation). Note that you must have an allocation on either the Deepthought2 or Juggernaut cluster; OnDemand is not supported on the MARCC/Bluecrab cluster at this time.

If not, follow the above links before proceeding with this quick start.

Connecting to the OnDemand Portal

Connecting to the OnDemand Portal is easy. Just open a web browser on your desktop/workstation/etc, and go to the URL for the HPC cluster you are using:

HPC Cluster URL for OnDemand Portal
Deepthought2 https://portal.deepthought2.umd.edu
Juggernaut https://portal.juggernaut.umd.edu
MARCC/Bluecrab No OnDemand Portal at this time
WARNING
You must completely exit your web browser in order to fully log out from the OnDemand portal. If you ever use OnDemand from a public/shared system, ALWAYS BE SURE to COMPLETELY EXIT THE BROWSER when you are finished. Otherwise your login credentials could be stored in session cookies and the next person to use the machine might be able to access your accounts on the HPC clusters.

You will be prompted for your username and password, use the same credentials you use to access the login node for the cluster. Once logged in, you will see the Dashboard home page, the main feature of which is the red navigation menu bar across the top.

For more details, please see the main OnDemand page.

Viewing/Editing/Managing Files

The OnDemand portal includes a built-in File Manager which you can use to navigate the filesystem. From the home page, select the leftmost Files drop down on the red navigation bar, and select the path from which you wish to start exploring. You should see your home directory and your lustre directory in the dropdown.

This will bring up a fairly standard File Manager interface. You can navigate directories, upload and download files, etc. It even includes a WYSIWYG editor application, allowing you to edit files easily.

More information on the use of the File Manager can be found on the main OnDemand page.

Interactive Jobs and GUIs

The OnDemand portal has support for submitting interactive jobs with full GUI support. This means that you no longer need to setup an X server on your workstation in order to view graphics and/or use GUI based applications. These applications can be accessed from the Interactive Apps drop down on the navigation bar of the home page. Currently, the following interactive applications are supported:

  • Interactive Desktop Session: this will open up a standard Linux GUI desktop environment, from which you can launch various applications.
  • Interactive Matlab Session: this will launch a Matlab GUI
  • Interactive Jupyter Session: this will launch a Jupyter notebook GUI session

Note: These interactive sessions are launched as jobs through the scheduler. As such, they will wait in the queue before starting, and depending on the current demand for resources on the cluster, this could be several hours. For quick sessions, it is recommended that you use the debug partition --- this is limited to 15 minute sessions, but typically will have queue times of no more than 5-10 minutes.

More information on Interactive Applications can be found on the main OnDemand page.

Batch Jobs

Although the interactive app features of the OnDemand are useful at times, particularly when one needs graphics, most production work is still going to rely on batch jobs. Because they are supporting complex research, the batch jobs themselves have a certain degree of complexity. However, the OnDemand portal has features which try to reduce the complexity.

The main such feature is the OnDemand Job Composer, accessible from the navigation bar on the home page under the Jobs dropdown. The Job Composer enables one to submit jobs based on either standard templates or other jobs that you submitted. Since often production work on an HPC cluster involves submitting many similar jobs, differing only a few parameters or in the data sets being acted upon, this structure can often reduce the learning curve in using the cluster.

Typically you start by finding a standard job template that most closely matches what you want to do, and creating a Job Composer job from it. You can then edit the job script, add files, edit files, etc. as needed to get the job to do the calculation you desire. When you are satisfied, you can submit the Job Composer job to the scheduler from within the Job Composer (so it now becomes a Slurm job), and monitor it also. This may take several iterations to get everything working properly, but usually this can all be done from within the OnDemand portal. Once it is working, you can enter a "production" mode of iteratively copying the Job Composer job, tweaking the parameters, etc., and submitting the new job.

Currently, the available job templates fairly generic and simple, but we are working on expanding that collection. If you have a OnDemand job that you think would be helpful to others in the UMD HPC community and/or wish to work with systems staff in developing an useful job template, please contact systems staff.

More information on the Job Composer, Job Templates and batch jobs can be found on the main OnDemand page.

Monitoring job status, etc

The OnDemand portal also includes various features for monitoring your jobs on the cluster.

Interactive jobs can be monitored from the My Interactive Sessions buttong on the navigation bar on the home page. This will open a page listing any active Interactive jobs you have running, and allow you to connect to a disconnected session.

On the navigation bar on the main OnDemand home page, under the Jobs dropdown there is an Active Jobs option. Selecting that will show a list of all jobs you have on the system (either running, queued, or recently completed). This includes both batch and interactive jobs. You can click on specific jobs to see more detail and to manage jobs (e.g. delete queued or running jobs).

More information on monitoring jobs from the OnDemand portal can be found on the main OnDemand page.