Skip to content

VisIt: interactive, scalable, visualization, animation and analysis tool

Overview of package

Package: VisIt
Description: interactive, scalable, visualization, animation and analysis tool
For more information: http://visit.llnl.gov/
Categories: Visualization
License: OpenSource (LLNL License)

General Usage

VisIt is an open-source interactive parallel visualization and graphical analysis tool for viewing scientific data. It can be used to visualize scalar and vector fields defined on 2D and 3D structured and unstructured meshes. VisIt was designed to handle very large data set sizes in the terascale range and yet can also handle small data sets in the kilobyte range.

This module will add the visit and related commands to your PATH.

Available versions by cluster

This section lists the available versions of the package VisIton the UMD HPC clusters.

Versions available on the Zaratan cluster

Version Module tags CPU(s) optimized for GPU ready?
3.3.3 3.3.3 zen2 Yes

Remote Visualization

This section discusses some of the strategies available for doing remote visualization using VisIt, with special attention to doing so using the UMD HPC systems. HPC clusters can generate large amounts of data, and visualization tools are useful in enabling researchers to understand the data that what produced. But generally the researchers are not sitting anywhere near the HPC clusters, HPC clusters generally do not have displays attached, and users usually wish to view the data on their desktop workstations. While users can copy the data files from the HPC clusters to their workstations, this can be time consuming as sometimes the data files are quite large. And that assumes there is room on the workstation disk.

The issues listed above can generally be addressed by remote visualization, in which parts or all of the VisIt code run on the HPC nodes, accessing data on the HPC system, with the display going back to your local workstation (e.g. your desktop or laptop). The basic strategies for doing this include: 1. Remote visualization using the OnDemand Portal: This is probably the easiest solution, and is recommended for most users.

  1. Remove visualization using VisIt Client/Server mode: This makes use of the client/server mode built into VisIt. It requires some initial setup, but can be very convenient.

Using the OnDemand Portal

The OnDemand Portal has a number of features for making use of HPC resources more user friendly. One of the most convenient features is the interactive applications which allows one to run GUI applications on one of the compute nodes of the HPC cluster with the display being presented in a web browser on your workstation.

This provides for the easiest way to achieve remote visualization using VisIt and other applications on the HPC cluster. Currently, we do not have an interactive application specifically for VisIt, but you can use the interactive desktop instead.

  1. Launch an interactive desktop job using the OnDemand Portal, as explained here. When resources are allocated and the job is running, connect to the desktop.
  2. In the desktop presented in your web browser, open a command line/ terminal window. In that window, module load visit and then invoke visit.
  3. You will get the full VisIt GUI appearing in your browser window. This VisIt process will be running on one of the compute nodes of the cluster, so you can issue any commands (including computationally expensive commands) and get the results displayed back on your web browser.

Using VisIt Client/Server mode

VisIt supports a client/server mode wherein you launch the VisIt GUI on your workstation/desktop but the data processing is handled on one or more remote systems. Graphical processing is split between the workstation and the remote systems.

This is particularly advantageous when working on High Performance Computing (HPC) clusters, as this mode of operation can:

  1. enable you to work on large data sets on the HPC cluster, without needing to transfer many GBs (or TBs) of data back to your workstation.
  2. allow you to leverage the CPU power of the HPC cluster to speed up the processing for visualization.

Within VisIt, this client/server mode is controlled by "Host Profiles". The following subsection deals with setting up these profiles (and includes some standard profiles for the UMD HPC clusters). After that, we discuss using the profiles for visualization tasks.

Standard Host Profiles

Before you can do client/server visualization with VisIt, you need to set up Host Profiles. You can probably do this fairly easily by just copying one or both of our standard profiles for the appropriate UMD HPC clusters to the appropriate hosts directory on your workstation.
The currently available standard profiles are:

Bug

Unfortunately we do not have any standard profiles available at this time

These files should go into the appropriate "hosts" directory on your workstation. For Unix-like systems, this is usually ~/.visit/hosts. On Windows systems I believe it is something like My Documents\VisIt VISIT_VERSION\hosts. After copying the files there, you will need to start VisIt again for them to be detected.

If you use one of these files, you can probably skip over the manual configuration described below, and proceed on to the section on using the profiles subsection is still useful if you wish to customize the standard profiles.

Manually Defining Host Profiles

Note

The following instructions are based on VisIt 2.10, but things should be similar for later versions.

From inside the VisIt GUI, do the following:

  1. Start by opening the Options | Host Profiles page from the menu bar.
  2. If you copied one of the standard host profiles, the should be visible in the Hosts area to the left, and you can select one of them to edit it. Or can click the New Host button to create a new host entry. Either way, it will open the entry with fields on the right side. There are two tabs on the right, Host Settings and Launch Profiles. We deal with Host Settings first.
  3. Host Nickname is the name that will be shown to you for the host profile. I suggest something like UMD Zaratan Cluster.

  4. Remote hostname is the hostname that VisIt will ssh to to open the remote VisIt process. Here you should give the appropriate hostname for the cluster, e.g login.zaratan.umd.edu

  5. In the Hostname aliases field, you should include the pattern that will match the hostnames for specific login nodes for the cluster. E.g.: login-*.zaratan.umd.edu

  6. Leave both Maximum nodes and Maximum processors unchecked

  7. For Path to VisIt Installation, enter the value /software/visit. This will cause VisIt to find custom wrapper scripts for these clusters which will ensure the correct environmental variables are set to run the compute engines, etc. on the UMD HPC clusters.
  8. For Username, enter your username on the cluster.
  9. You will probably need to click the box for Tunnel data connections through SSH. This is required if your workstation has any sort of firewall on it, which is typically the case.
  10. The other fields can be left to the defaults.
  11. Now select the Launch Profiles tab. The previous tab gave basic information about connecting to the cluster, we now provide information about how to run on the cluster. You can select an existing launch profile and edit below, or use "New Profile" button to create a new profile. We are going to define three profiles:
    1. serial: this runs VisIt in one process on the login node.
    2. parallel (debug partition): this will run VisIt in a job submitted to the debug partition. I.e., a short job, but run at somewhat higher priority for better interactive use.
    3. parallel: this will run VisIt in a more generic job. You can specify the number of cores/nodes/etc.
  12. The serial launch profile is easiest. Just click the "New Profile" button, and enter its name, e.g. serial. That's it.
  13. The two parallel profiles are defined similarly. Click the "New Profile" button and enter its name. Then select the Parallel tab, and:
    1. Click the Launch parallel engine checkbox.
    2. Click the Parallel launch method checkbox and select sbatch/mpirun in the drop down (probably the last entry).
    3. For the parallel (debug partition) profile, also click the Parition/PoolQueue checkbox and enter debug in the text box. For the generic parallel profile, you are probably best just leaving this unchecked/blank.
    4. You can adjust the Number of processors value to the desired default value. You will be able to adjust this each time you use the profile, but this will be the default value. The standard Zaratan compute nodes have 128 cores per node, but that is probably more than you wish to use for a typical VisIt job. Select a reasonable default value.
    5. For the next 4 items (Number of nodes, Bank/Account, Time limit, and Machine file), if you check the checkbox you can set a default which can be modified each time you use the profile. If left unchecked, you will not be able to modify when using the profile, and it will default to whatever sbatch decides. I would recommend checking the boxes for Number of nodes, Bank/Account and Time Limit, but typically Machine File can be left unchecked.

    6. There is also an Advanced subtab just under the Launch parallel engine checkbox. Most of these can be left empty/defaults.

      1. The Use VisIt script to set up parallel envionrment checkbox should be checked (this should be by default).
      2. The remaining arguments can typically be left unchecked except in certain cases. I would recommend that if you need to make modifications here, you create (or copy) a new profile for that specific need. E.g., if the default memory requested for the Slurm job is insufficient, you can create a "parallel-large memory" profile with everything copied from the "parallel" profile and then make the needed changes.
      3. Launcher arguments: Here you can provide additional arguments to be passed to the sbatch command. E.g., to request 9 GB of RAM per CPU core (instead of the default 4 GB) you could add here something like --mem-per-cpu=9000.
      4. Sublauncher arguments: Here you can provide additional arguments to be passed to the mpirun command.
      5. Sublauncher pre-mpi command: Here you can provide a command to be run before the mpirun command in the batch script. In order to ensure the proper environmental variables are set in the sbatch script, We recommend setting this to: source /etc/profile; module load visit/3.3.3
      6. Sublauncher post-mpi command: Here you can provide a command to be run after the mpirun command in the batch script.
      7. Once you have things as you like them, click the Apply button to make it effective. If you editted anything (i.e. created new profiles or changed a profile), you should select the new/modified host profiles and Export host them to ensure they are saved and available in your next VisIt session.
      8. Click the Dismiss button to close the Host Profiles window.
      9. Make sure you click on Options|Save settings to save your new profile.

Using Profiles

In this section we will briefly describe how to use the profiles. It assumes you already have a Host Profile of one of the UMD HPC clusters, either a standard profile or a manually created/edited profile. It is also assumed that this profile has the three launch profiles (serial, debug, and parallel) as defined above, and that you have access to use to use the HPC cluster the profile is for.

Note

The version of VisIt that you are running on your workstation must match what is available on the cluster, at least down to the minor (second/middle) version number. If you do not have that on your workstation, you can try running VisIt using the OnDemand portal.

In general, using VisIt in client/server starts with opening the data file. Just start to open a file as usual, but in the Host dropdown at the top of the file open dialog, there should be an option for the various host profiles you have defined. Select the appropriate host profile. It will likely prompt you for a password (make sure the username given, if any is correct, and correct it if not) (If no username is given, it assumes your username on the workstation is the same as on the remote cluster). Within a few seconds you should see a file list corresponding to your home directory on the cluster. You can then select a file as usual.

If multiple launch profiles exist for that host, you will be given an option of choosing which profile you wish to use, and what options you wish to use with that launch profile if it supports any. If there is only a single launch profile, you obviously cannot choose a different launch profile, but a pop up will appear if there are any options for that launch profile. Otherwise, VisIt will just launch the profile with the defaults.

If you just wish to use VisIt on a file that resides on the HPC cluster (without copying the file to your local workstation) but do not need (or cannot use) the parallel capabilities of VisIt and you will not be doing heavy computation, the serial option is the easiest, and does not take additional options. Just select it and hit OK. It may take a couple of seconds to start the remote engine, but then should return and you can visualize your data as if it were local.

Note

As the serial launch profile runs on the login node, you can only use this for light visualization. For sessions which will be involving heavy computation, please use one of the other launch modes, even if not availing yourself of VisIt's parallel capabilities.

The parallel launch options offer more power, but are a bit more complicated to use even though VisIt does a good job of hiding most of the complexity from the user. VisIt generally uses data based parallelization, which means generally one will need a parallel mesh data set to effectively use its parallelization.

To use one of the parallel profiles, just select it after selecting the file. The parallel (debug partition) is good for a short interactive visualization, but is limited in number of processes/nodes and is also limited to a run time of 15 minutes. However, since it uses the debug partition, it generally will spend less time waiting in the queue. The generic parallel profile is less restrictive, but depends on jobs submitted via sbatch and can have significant wait times before the job starts running.

When you select the profile, you typically will have the opportunity to change the defaults for wall time, number of nodes, and allocation account to which the job is to be charged.

Warning

VisIt seems to assume 8 processes per node by default, so e.g. if you request 20 processes, it will try to spread them over 3 nodes, even though recent UMD clusters can easily support such on a single node.
The HPC team strongly advises manually setting the number of nodes appropriately. Note also that the memory of the node is split evenly over all the VisIt processes on the node, so you might need to adjust the node count to use more than the minimal number of nodes in cases where memory requirements are higher.

When you finish updating options and hit "OK", your VisIt GUI will ssh to the login node for the cluster and submit a batch job requesting the desired number of nodes/cores. Typically you will see a pop up showing that VisIt is awaiting a connection from the compute engines --- this will not occur until after the batch job starts. For batch jobs submitted to the debug partition, this should typically be within a minute or two, but it is likely significantly longer for the general parallel profile.

When the job starts, after 20 seconds or so the connection should be made and the pop up will go away. At this point you can use VisIt as normal.

At some point, the scheduler may terminate your compute engines (e.g. due to exceeding walltime). You should be able to continue using the GUI, and when you try to do something that requires the compute engine, a pop up will appear allowing you to start up a new launch profile.

Client/server mode example

Here we provide a quick example of using VisIt with parallel processing in client/server mode. This example assumes that you have already set up an appropriate host profile and placed it in the proper VisIt configuration directory for your workstation.

Tip

When running in client-server mode, VisIt will use ssh to establish a connection to the login node of the cluster in order to submit jobs, etc. While VisIt can parse certain responses from ssh, there are many responses which will confuse it. Therefore, we recommend:

  1. You use the campus VPN before starting VisIt in client-server mode.
  2. You verify that you can ssh from your workstation (where you intend to run VisIt from) and the HPC login node without being prompted for anything but your password. In particular, if you get any warnings/etc. about ssh fingerprints, resolve these before starting to use VisIt client-server mode.
  1. Startup visit on your workstation.

  2. From the main control window, under the File tab, select Open File.

    1. This will open the File Open dialog.
    2. On the top line (Host), click on the drop down to the right.
      You should see an option for the profile you configured. If so, select it. If not, you either did not create or download a profile, or you did not put the profile file in the correct location. Exit VisIt, fix the issue , and restart VisIt.
    3. The dialog will gray out for a couple of seconds as VisIt connects to the login node and looks at your home directory.
      In general, it will ask for a password (and if it does, verify that the username is correct). After connecting, VisIt should display a file browser for your home directory.
    4. We are going to load one of the VisIt example files, so in the second line, labelled Path, replace the existing text with /software/visit. After the file browser window updates, descend down the directory named after the VisIt and related subdirs (e.g. /software/visit/3.3.3/linux-rhel8-x86_64/data for VisIt version 3.3.3).
    5. Double click on the multi_ucd3d.silo file. You can try other files if you want, but only the multi_* files are multidomain datasets --- if you open a file not starting with multi_*, it will be a single domain dataset and you will not be able to effectively use the parallelism in VisIt.
    6. You will now get the dialog for selecting options on the cluster. Basically, a list wherein you can select one of the launch profiles for the cluster. Click either parallel (debug partition) or parallel (generally parallel (debug partition) will give you quicker turn around for this simple example). Set the time limit to something reasonable (like 15 minutes), and select the number of cores and nodes you want. The multi_ucd3d.silo file I believe has 36 data domains, so it is probably best to choose a number which divides 36 evenly. Requesting more than 36 cores would be wasteful. You should probably select a single node.
    7. Click OK when satisfied. A window should come up showing that it is waiting for the compute engine to start up. This will cause a batch job to be submitted to the scheduler to run your compute engine. It should take only a few minutes if you are using the debug queue. If it is going on for a long time, you might wish to check if there is a job submitted on your behalf on the cluster (the job name should be something like visit.*USERNAME*, and the number should be in the terminal from which you launched visit.
    8. Shortly after the job for the compute engine starts, you should be back to the normal VisIt GUI.
    9. To finish our example, define a procid scalar.
    10. From Controls menu, select Expressions
    11. Click New, and enter procid in the Name field.
    12. Click Insert function, go to the Miscellaneous submenu item, and click on procid.
    13. Click Insert variable, go to the Meshes submenu item, and click on mesh1.
    14. The definition box should now display procid(mesh1). If not, fix it so it does. When done, click Apply and Dismiss.
    15. Now in the Plots section, click on the Add button, and in the Pseudocolor submenu select procid. If no procid appears, you did not define the procid scalar properly above.
    16. Now in the Plots section, click Draw.
    17. You should see a multicolored 3D half cylinder (if you chose the multi_ucd3d.silo file; other multi_* files will be different shapes but still multicolored. Single domain files will be a solid color). The different colors represent the processor id of the processor responsible for rendering that section of the data.
    18. Congratulations. You just used VisIt in parallel mode.