OpenFOAM example on Shaheen

Owned by Mohsin Ahmed Shaikh
Jun 19, 2022
2 min read

OpenFOAM can be run as a batch processing job on Shaheen. You can submit your jobscript to SLURM and it will run on your behalf when the requested resources are allocated.

Here we attempt to adapt a tutorial example OpenFOAM into a SLURM workflow.

We copy the motorBike example of OpenFOAM and try to run the step as SLURM job steps within one job. In summary, the jobscript does the following:

It is expected that you are running this jobscript in scratch directory

  1. Load and set the environment

  2. Copies the required files from Tutorial directory to current working directory

  3. Creates the processor directories for parallel run using decomposePar

  4. Pre-process the mesh

  5. Meshing using snappyHexMesh

  6. Validate the integrity of the mesh

  7. Run the solver

  8. Reconstruct the mesh for analysis

 

#!/bin/bash #SBATCH --nodes=1 #SBATCH --hint=nomultithread #SBATCH --time=01:00:00 # Load the envrionment and activate OpenFOAM module swap PrgEnv-cray PrgEnv-gnu module load openfoam/2012 source $OPENFOAM_HOME/etc/bashrc # First copy the example directory provided by OpenFOAM tutorials in your scratch directory cp -r $FOAM_TUTORIALS/incompressible/simpleFoam/motorBike . cd motorBike # Copy the geometry from Tutorial directory to current case mkdir -p constant/triSurface cp -f ${FOAM_TUTORIALS}/resources/geometry/motorBike.obj.gz constant/triSurface/ # rename the decomposeParDict to the default name cp system/decomposeParDict.6 system/decomposeParDict srun -n 1 --hint=nomultithread surfaceFeatureExtract srun -n 1 --hint=nomultithread blockMesh srun -n 1 --hint=nomultithread decomposePar srun -n 6 --hint=nomultithread snappyHexMesh -overwrite -parallel srun -n 6 --hint=nomultithread topoSet -parallel # using the convinience function provided in tutorial runner to copy the 0.orig to 0/ of every processor directory source ${WM_PROJECT_DIR}/bin/tools/RunFunctions restore0Dir -processor srun -n 6 --hint=nomultithread patchSummary -parallel srun -n 6 --hint=nomultithread potentialFoam -writephi -parallel srun -n 6 --hint=nomultithread checkMesh -writeFields '(nonOrthoAngle)' -constant -parallel # Run the solver srun -n 6 --hint=nomultithread simpleFoam -parallel # Reconstruct the solved mesh from decomposed partitions for analysis srun -n 1 --hint=nomultithread reconstructParMesh -constant srun -n 1 --hint=nomultithread reconstructPar -latestTime

 

The output of all the steps will be written in a slurm-#####.out file in the job submission directory.