OpenFOAM-version-7/C3/Parallel-Computing-in-OpenFOAM/English

Title of the script: Parallel Computing in OpenFOAM

Script: Divyesh Variya, Ashuthosh Shridhar

Video & Narration: John Pinto, Payel Mukherjee

Keywords: OpenFOAM, Parallel Computing, Hierarchical, Manual, Scotch, Simple, decomposition, mpirun, Paraview, Video-Tutorial

Visual Cue	Narration
Slide 1: Opening Slide	Welcome to this spoken tutorial on Parallel Computing in OpenFOAM.
Slide 2: Learning Objectives	In this tutorial, we will learn to, Decompose a domain Run a simulation using Simple decomposition method Reconstruct a domain
Slide 3: System Specifications	To record this tutorial, I am using, Ubuntu Linux OS version 22.04 OpenFOAM version 9 ParaView version 5.10.1 and gedit Text Editor
Slide 4: Prerequisites If not, please go through the prerequisite OpenFOAM tutorial on https://spoken-tutorial.org	To practice this tutorial, learner should have, Basic knowledge of OpenFOAM software If not, please go through the prerequisite OpenFOAM tutorials on this website.
Slide 5: Parallel Computing	In parallel computing, a large domain is split into multiple domains of smaller size. Each of the smaller domains is assigned an individual processor. The processors run simultaneously and communicate with each other. OpenFoam uses the OpenMPI library to do this. Thus, we can speed up the simulation.
Slide 6: Processes Involved	There are three main processes involved in parallel computing. Decomposition of the domain. Simulation of the decomposed tasks in separate processors. Reconstruction of the domain.
Slide 7: Decomposition Methods	The four main decomposition methods available in OpenFOAM are: Simple Hierarchical Scotch and Manual
Only Narration	Let us learn about Simple Decomposition method using a 2 Dimensional example.
Slide 8: Simple Decomposition	Simple decomposition method divides the domain in equal parts. The splitting is done according to the user-specified coefficients.
Slide 9: Input Parameter - Simple	Simple decomposition requires two inputs. Decomposition coefficients and cell skewness factor, delta A typical value for delta is 10 to the power of minus 3(10^-3).
Slide 10: Decomposition Coefficients	Decomposition coefficients represent the number of subdomains in a particular direction. Thus, the product of nx, ny, nz must be equal to the total number of subdomains.
Slide 11: Simple Decomposition - n (2 2 1)	This slide shows the decomposition domain for the decomposition coefficients of (2 2 1).
Slide 12: Simple Decomposition - n (4 1 1)	This slide shows the decomposed domain for the decomposition coefficients of (4 1 1).
Only Narration	For more information on decomposition methods, Please refer to Additional Reading material.
Only Narration	Let us set up a case.
Press CTRL + ALT + T keys.	Open the terminal by pressing Ctrl, Alt and T keys.
[Terminal] Type: cd $FOAM_RUN	At the prompt, type this command to go to the run directory.
[Terminal] Type: cp -r $FOAM_TUTORIALS/incompressible/icoFoam/cavity/cavity .	Copy the cavity case from the tutorial directory into the run directory.
[Terminal] Type: cd cavity	Navigate to the cavity case.
[Terminal] Type: blockMesh	Run the blockMesh command to execute the mesh.
[Terminal] Type: cp -r $FOAM_TUTORIALS/multiphase/interFoam/laminar/damBreak/damBreak/system/decomposeParDict system/	Copy a decomposeParDict file from the dam break tutorial to the system folder.
Only Narration	To see the maximum number of processors available in your machine, do the following.
[Terminal] Type: lscpu	Type lscpu on the terminal.
[gedit - terminal] Highlight CPU(s): 24	My machine has 24 CPUs .
[gedit - terminal] Highlight Thread(s) per core…. 2	12 physical cores per socket and 2 threads per physical core.
Slide 13: Number of processors	Multiplication of these 3 quantities gives the number of logical cores. This is equal to the maximum number of processors.
Only Narration	We will run this case in 4 processors.
Only Narration	Note: Please assign at least 4 processors to your Virtual Machine.
[Terminal] Type: gedit system/decomposeParDict	Open the decomposeparDict in a text editor.
[gedit decomposeParDict] Highlight numberOfSubdomains 4;	The number of subdomains is defined as 4.
[gedit decomposeParDict] method simple;	Simple method of decomposition is being used.
[gedit decomposeParDict] Highlight coeffs	Go to coeffs sub-dictionary.
[gedit decomposeParDict: Highlight n	n represents the Decomposition coefficients in each direction. I will modify n so that we have: 2 divisions in x and y directions. And 1 division in z direction.
[gedit decomposeParDict] Save >> Close the window	Save and Close the file.
[Terminal] Type: decomposePar -cellDist	To decompose the domain, type the following command.
[Terminal] Highlight: cellDist	The argument cellDist writes a file named cellDecomposition inside the constant folder. This file contains the details of the processor allocation.
Only Narration	The domain is decomposed.
Only Narration	The command decomposePar creates a directory for each processor. These directories contain the subdomain and the corresponding geometric fields.
[Terminal] Type: ls	Type ls to view the folders assigned to each processor.
[cavity - directory] Highlight Processor0 …..Processor3	Since we use 4 processors, we have 4 processor directories.
[Terminal] Type: ls processor0	Type ls processor0 to view the content of processor0.
[Terminal] Highlight 0 constant	It contains the 0 and constant directories.
Point to 0 directory. Point to Constant directory.	0 directory contains the decomposed geometric field. The Constant directory contains the decomposed mesh files.
[Terminal] Type: gedit constant/cellDecomposition	Open the cellDecomposition file from the Constant folder in a text editor.
[gedit - cellDecomposition] Highlight 400	The number on the first line represents the number of cells in the domain.
[gedit - cellDecomposition] Highlight (.......	The entries within the bracket represent the processor number.
[gedit - cellDecomposition] Highlight 0	0 here indicates the processor number that is assigned to the corresponding cell.
[gedit - cellDecomposition] Close the window	Close the file.
Only Narration	Let’s view the subdomains in paraview.
[Terminal] Type: paraFoam	Type paraFoam on the terminal.
[ParaView] Uncheck: Properties Tab >> Scroll Down - Mesh Parts >> Uncheck fixedWalls & movingWall	Uncheck all other fields other than internalMesh from Mesh Parts.
[ParaView] Click: Properties Tab >> Scroll Down - Fields >> check cellDist	Check all Volume Fields in the Properties tab.
[ParaView] Click: Apply	Click on the Apply button in the Properties tab.
[ParaView] Click on Solid Color >> Click on cell Dist	Click on Solid Color available in the Active Variable Controls. Select cellDist with a block from the menu.
Point to the dark blue block and the light blue block. Point to the beige block and the maroon block.	One can see the divided domain. The dark blue block belongs to processor number 0 The light blue block belongs to processor number 1 The beige block belongs to processor number 2, And the maroon belongs to processor number 3
[paraview] Close the window	Close the ParaView window.
Slide 14: Command - Parallel run	OpenFOAM uses the openMPI library for parallelization. mpirun is the process launcher. The argument np represents the number of processors. In our case we use 4 processors. Then the name of the application entered. We need to specify that the application has to be run in parallel. The output of the simulation is written in a file named log. This is done by using the right angle bracket. The & indicates that the process will be run in the background. This allows us to use the current terminal when the application is running.
[Terminal] Type: mpirun -np 4 icoFoam -parallel > log &	Type this command in the terminal.
[Terminal] Type: tail log	Type the following command to view the end of the log file.
[Terminal] Highlight End	The simulation is complete.
Only Narration	To post-process the case, we need to reconstruct the domain.
[Terminal] Type: reconstructPar	To do that, type reconstructPar in the terminal.
[Terminal] Type: ls	Type ls
[Terminal] Highlight 0.5	The end time directory is reconstructed.
Slide 15: Summary	With this we come to the end of this tutorial. Let us summarize. In this tutorial, we have learned to, Decompose a domain Run the simulation using the simple decomposition method Reconstruct a domain
Slide 16: Assignment	As an assignment: Simulate the same case using, Simple decomposition with 2 processor Scotch decomposition with processor weight
Slide 17: About the Spoken Tutorial Project	The video at the following link summarizes the Spoken Tutorial project. Please download and watch it.
Slide 18: Spoken Tutorial Workshops	We conduct workshops using Spoken Tutorials and give certificates. Please contact us.
Slide 19: Spoken Tutorial Forum	Please post your timed queries in this forum.
Slide 20: FOSSEE Forum	Do you have any general/technical questions? Please visit the forum given in this link.
Slide 21: FOSSEE Case Study Project	The FOSSEE team coordinates solving feasible CFD problems of reasonable complexity using OpenFOAM. We give honorarium and certificates to those who do this. For more details, please visit these sites.
Slide 22: Spoken Tutorial	Spoken Tutorial project was established by the Ministry of Education, Govt. of India This tutorial is contributed by Divyesh Varya, Nishit Pachpande, John Pinto and Payel Mukherjee from IIT Bombay. Thank you for joining.

Contributors and Content Editors

Evan