Clara-Parabricks on Ibex

Owned by Mohsin Ahmed Shaikh
Last updated: Jul 28, 2020

Installation

  • requires fakeroot feature of Singularity to be working

    • must build the image on a compute node

Getting the source

  • Goto https://developer.nvidia.com/clara-parabricks and choose appropriate Licensing method:

    • I chose “standard evaluation license” for testing

    • Fill in the requested details and the link to tarball containing the installation script and license will be emailed by NVIDIA

  • It appears that Parabricks is a container image, which when installed using the provided installer.py in the tarball, creates a Singularity sandbox as a installation directory and builds a statically built executable(s) pbrun.

Intall command

salloc --gres=gpu:1 --time=01:00:00

Once GPU node is allocated, the following needs to run on compute node:

export XDG_RUNTIME_DIR=/path/to/some/tmp/space srun --pty ./installer.py --container singularity --install-location /path/to/installation/directory

 

 

To pull a container image from NVIDIA GPU Containers Cloud, you will need to add --ngc to the installer command line. Before using the NGC container image please check the version, its v2.5.0 at the moment whereas the newest in gitlab in v3.0.0.2:

srun --pty ./installer.py --container singularity --ngc --install-location /path/to/installation

 

Testing

I am using the testing recipe from https://ngc.nvidia.com/catalog/containers/hpc:parabricks

Getting data

Typical Jobscript

Output

Benchmarks

NOTES

  • Ran the benchmark with data in local SSD (/tmp of the compute node) with Single GPU. There was no change in compute time of phases.