Running on HPC clusters

Instructions for running on various HPC clusters are given below.

Gadi (NCI Australia)

Use the openmpi/4.1.4 module (or newer), and build with gcc/system or gcc/11.1.0, and use cuda/11.7.0 (or newer).

You can use VisIt in client/server mode with the following server-side patch for the launcher script.

A host file is provided here. You must change the username, project code, and server-side VisIt path.

The recommended build procedure on Setonix is:

source scripts/setonix.profile
mkdir build; cd build
cmake .. -C ../cmake/setonix.cmake
make -j16

Then a single-node test job can be run with:

cd ..
sbatch scripts/setonix-1node.submit

If interconnect issues are observed, it is recommended to add the line

export FI_CXI_RX_MATCH_MODE=software

to your job scripts.