| Table of Contents |
|---|
Installation
requires fakeroot feature of Singularity to be working
must build the image on a compute node
Getting the source
Goto https://developer.nvidia.com/clara-parabricks and choose appropriate Licensing method:
I chose “standard evaluation license” for testing
Fill in the requested details and the link to tarball containing the installation script and license will be emailed by NVIDIA
It appears that Parabricks is a container image, which when installed using the provided
installer.pyin the tarball, creates a Singularitysandboxas a installation directory and builds a statically built executable(s)pbrun.
Intall command
| Code Block |
|---|
salloc --gres=gpu:1 --time=01:00:00 |
...
| Code Block |
|---|
srun --pty ./installer.py --container singularity --ngc --install-location /path/to/installation |
Testing
I am using the testing recipe from https://ngc.nvidia.com/catalog/containers/hpc:parabricks
Getting data
| Code Block |
|---|
$ wget https://s3.amazonaws.com/parabricks.sample/parabricks_sample.tar.gz $ tar xf parabricks_sample.tar.gz $ ls parabricks_sample Data Ref |
Typical Jobscript
| Code Block |
|---|
#!/bin/bash
#SBATCH --job-name=parabricks-test
#SBATCH --time=02:00:00
#SBATCH --gres=gpu:1
#SBATCH --constraint=gpu_ai
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=4
#SBATCH --mem=20G
module load parabricks/3.0.0.2
export DATA_DIR=/ibex/scratch/shaima0d/scratch/parabricks-testing
pbrun fq2bam \
--ref ${DATA_DIR}/parabricks_sample/Ref/Homo_sapiens_assembly38.fasta \
--in-fq ${DATA_DIR}/parabricks_sample/Data/sample_1.fq.gz parabricks_sample/Data/sample_2.fq.gz \
--out-bam output.bam
|
Output
| Code Block |
|---|
Loading module for Singularity Singularity 3.5 modules now loaded Loading module for parabricks-3.0.0.2 parabricks-3.0.0.2 modules now loaded [Parabricks Options Mesg]: Checking argument compatibility [Parabricks Options Mesg]: Automatically generating ID prefix [Parabricks Options Mesg]: Read group created for /ibex/scratch/shaima0d/scratch/parabricks- testing/parabricks_sample/Data/sample_1.fq.gz and /ibex/scratch/shaima0d/scratch/parabricks- testing/parabricks_sample/Data/sample_2.fq.gz [Parabricks Options Mesg]: @RG\tID:HK3TJBCX2.1\tLB:lib1\tPL:bar\tSM:sample\tPU:HK3TJBCX2.1 ------------------------------------------------------------------------------ || Parabricks accelerated Genomics Pipeline || || Version v3.0.0.2 || || GPU-BWA mem, Sorting Phase-I || || Contact: Parabricks-Support@nvidia.com || ------------------------------------------------------------------------------ [M::bwa_idx_load_from_disk] read 0 ALT contigs GPU-BWA mem ProgressMeter Reads Base Pairs Aligned [16:21:48] 5043564 580000000 [16:22:36] 10087128 1160000000 [16:23:25] 15130692 1740000000 [16:24:13] 20174256 2320000000 [16:25:01] 25217820 2900000000 [16:25:49] 30261384 3480000000 [16:26:38] 35304948 4060000000 [16:27:26] 40348512 4640000000 [16:28:16] 45392076 5220000000 [16:29:05] 50435640 5800000000 GPU-BWA Mem time: 525.787852 seconds GPU-BWA Mem is finished. GPU Sorting, Marking Dups, BQSR ProgressMeter SAM Entries Completed Total GPU-BWA Mem + Sorting + MarkingDups + BQSR Generation + BAM writing Processing time: 570.052968 seconds [main] CMD: PARABRICKS mem -Z ./pbOpts.txt /ibex/scratch/shaima0d/scratch/parabricks-testing/parabricks_sample/Ref/Homo_sapiens_assembly38.fasta /ibex/scratch/shaima0d/scratch/parabricks-testing/parabricks_sample/Data/sample_1.fq.gz /ibex/scratch/shaima0d/scratch/parabricks-testing/parabricks_sample/Data/sample_2.fq.gz @RG\tID:HK3TJBCX2.1\tLB:lib1\tPL:bar\tSM:sample\tPU:HK3TJBCX2.1 [main] Real time: 576.339 sec; CPU: 2121.508 sec ------------------------------------------------------------------------------ || Program: GPU-BWA mem, Sorting Phase-I || || Version: v3.0.0.2 || || Start Time: Mon Jul 27 16:20:45 2020 || || End Time: Mon Jul 27 16:30:21 2020 || || Total Time: 9 minutes 36 seconds || ------------------------------------------------------------------------------ ------------------------------------------------------------------------------ || Parabricks accelerated Genomics Pipeline || || Version v3.0.0.2 || || Sorting Phase-II || || Contact: Parabricks-Support@nvidia.com || ------------------------------------------------------------------------------ progressMeter - Percentage [16:30:24] 0.0 0.00 GB [16:30:34] 47.6 0.27 GB [16:30:44] 94.0 0.11 GB Sorting and Marking: 30.002 seconds ------------------------------------------------------------------------------ || Program: Sorting Phase-II || || Version: v3.0.0.2 || || Start Time: Mon Jul 27 16:30:24 2020 || || End Time: Mon Jul 27 16:30:54 2020 || || Total Time: 30 seconds || ------------------------------------------------------------------------------ ------------------------------------------------------------------------------ || Parabricks accelerated Genomics Pipeline || || Version v3.0.0.2 || || Marking Duplicates, BQSR || || Contact: Parabricks-Support@nvidia.com || ------------------------------------------------------------------------------ progressMeter - Percentage [16:31:05] 9.0 5.95 GB [16:31:15] 22.3 13.18 GB [16:31:25] 38.2 11.80 GB [16:31:35] 52.4 9.11 GB [16:31:45] 64.0 6.49 GB [16:31:55] 76.7 3.97 GB [16:32:05] 87.1 1.24 GB [16:32:15] 100.0 0.00 GB BQSR and writing final BAM: 80.932 seconds ------------------------------------------------------------------------------ || Program: Marking Duplicates, BQSR || || Version: v3.0.0.2 || || Start Time: Mon Jul 27 16:30:55 2020 || || End Time: Mon Jul 27 16:32:16 2020 || || Total Time: 1 minute 21 seconds || ------------------------------------------------------------------------------ |
Benchmarks
| Child pages (Children Display) | ||
|---|---|---|
|
...