Codes - Revision history

Abdelsamie: /* General comments on the codes */

2020-09-16T15:26:52Z

‎General comments on the codes

Lartigue: /* YALES2 */

2020-08-26T13:39:28Z

‎YALES2

Lartigue: /* General comments on the codes */

2020-08-25T21:19:54Z

‎General comments on the codes

Xonnainty at 16:21, 25 August 2020

2020-08-25T16:21:01Z

Xonnainty at 16:20, 25 August 2020

2020-08-25T16:20:00Z

Show changes

Xonnainty at 13:38, 25 August 2020

2020-08-25T13:38:20Z

Coria: /* General comments on the codes */

2020-08-23T20:24:32Z

‎General comments on the codes

Xonnainty: /* General comments on the codes */

2020-08-17T20:14:09Z

‎General comments on the codes

Xonnainty: /* General comments on the codes */

2020-08-17T20:05:24Z

‎General comments on the codes

Xonnainty: /* General comments on the codes */

2020-08-17T20:03:36Z

‎General comments on the codes

@@ Line 163: / Line 163: @@
   | Chemistry integration
   | CVODE
-  | PyJac
+  | expl. RK4
   | CVODE
   |-

@@ Line 7: / Line 7: @@
 == YALES2 ==
-YALES2 is a '''massively parallel multiphysics platform'''<ref name="moureauwebsite">V. Moureau. Yales2 public website [https://www.coria-cfd.fr/index.php/YALES2 www.coria-cfd.fr/index.php/YALES2].</ref><ref name="moureau2011"/> developed since 2009 by Moureau, Lartigue and co-workers at CORIA (Rouen, France).
+YALES2 is a '''massively parallel multiphysics platform'''<ref name="moureau2011"/> developed since 2009 by Moureau, Lartigue and co-workers at CORIA (Rouen, France).
 It is dedicated to the high-fidelity simulation of '''low-Mach number flows in complex geometries'''.
 It is based on the '''Finite Volumes formulation''' of the '''Navier-Stokes equations''' and it can solve both '''non-reacting''' and '''reacting flows'''.

@@ Line 127: / Line 127: @@
   |-
   | Connectivity
   | Unstructured
   | Unstructured
   |-

@@ Line 115: / Line 115: @@
 On the other hand, both '''YALES2''' and '''Nek5000''' use an iterative solver with an efficient preconditioning technique; this method is more versatile and should be computationally more efficient for large and complex geometrical configurations.
 '''Nek5000''' employs CVODE to integrate the thermochemical equations without further '''splitting''' of the different terms accounting for convection, diffusion and chemistry.
-The main differences between the three codes are summarized in Table~\ref{Tab:codes}.
+The main differences between the three codes are summarized in the table below.
 Please note that the presented values are those used for the benchmark, even though some other options are available in each codes.

@@ Line 7: / Line 7: @@
 == YALES2 ==
-YALES2 is a '''massively parallel multiphysics platform'''<ref>V. Moureau. Yales2 public website [https://www.coria-cfd.fr/index.php/YALES2 www.coria-cfd.fr/index.php/YALES2].</ref><ref>V. Moureau, P. Domingo, and L. Vervisch. From large-eddy simulation to direct numerical simulation of a lean
+YALES2 is a '''massively parallel multiphysics platform'''<ref name="moureauwebsite">V. Moureau. Yales2 public website [https://www.coria-cfd.fr/index.php/YALES2 www.coria-cfd.fr/index.php/YALES2].</ref><ref name="moureau2011">V. Moureau, P. Domingo, and L. Vervisch. From large-eddy simulation to direct numerical simulation of a lean
 premixed swirl flame: Filtered laminar flame-pdf modeling. Combust. Flame, 158(7):1340–1357, 2011.</ref> developed since 2009 by Moureau, Lartigue and co-workers at CORIA (Rouen, France).
 It is dedicated to the high-fidelity simulation of '''low-Mach number flows in complex geometries'''.
 It is based on the '''Finite Volumes formulation''' of the '''Navier-Stokes equations''' and it can solve both '''non-reacting''' and '''reacting flows'''.
 It can actually solve various physical problems thanks to its original structure which is composed of a main numerical library accompanied with tens of dedicated solvers (for acoustics, multiphase flows, heat transfer, radiation\ldots) which can be coupled with one another.
-YALES2 relies on '''unstructured meshes''' and a '''fully parallel dynamic mesh adaptation technique''' to improve the resolution in physically-relevant zones to mitigate the computational cost<ref>P. Benard, G. Balarac, V. Moureau, C. Dobrzynski, G. Lartigue, and Y. D'Angelo. Mesh adaptation for large eddy simulations in complex geometries. Int. J. Numer Methods Fluids, 81:719-740, 2015.</ref>.
+YALES2 relies on '''unstructured meshes''' and a '''fully parallel dynamic mesh adaptation technique''' to improve the resolution in physically-relevant zones to mitigate the computational cost<ref name="benard2015">P. Benard, G. Balarac, V. Moureau, C. Dobrzynski, G. Lartigue, and Y. D'Angelo. Mesh adaptation for large eddy simulations in complex geometries. Int. J. Numer Methods Fluids, 81:719-740, 2015.</ref>.
 As a result it can easily handle meshes composed of billions of tetrahedra, thus enabling the '''Direct Numerical Simulation''' of laboratory and semi-industrial configurations.
 It is now composed of nearly 500,000 lines of object-oriented Fortran and the parallelism is currently ensured by a pure '''MPI''' paradigm, although a hybrid '''OpenMP/MPI''' as well as a GPU version are under development.
-As most low-Mach number codes, the time-advancement is based on a '''projection-correction method''' following the pioneering work of<ref>A.J. Chorin. Numerical solution of the Navier-Stokes equations. Math. Computation, 22(104):745–762, 1968.</ref>.
+As most low-Mach number codes, the time-advancement is based on a '''projection-correction method''' following the pioneering work of<ref name="chorin1968"/>A.J. Chorin. Numerical solution of the Navier-Stokes equations. Math. Computation, 22(104):745–762, 1968.</ref>.
-The prediction step uses a method which is a blend between a '''4th-order Runge-Kutta method''' and a '''4th-order Lax-Wendroff-like method'''<ref>M. Kraushaar. Application of the compressible and low-Mach number approaches to Large-Eddy Simulation of turbulent flows in aero-engines. PhD thesis, PhD, Institut National Polytechnique de Toulouse-INPT, 2011.</ref>, combined with a '''4th-order node-based centered finite-volume discretization''' of the convective and diffusive terms.
+The prediction step uses a method which is a blend between a '''4th-order Runge-Kutta method''' and a '''4th-order Lax-Wendroff-like method'''<ref name="kraushaar2011"/>M. Kraushaar. Application of the compressible and low-Mach number approaches to Large-Eddy Simulation of turbulent flows in aero-engines. PhD thesis, PhD, Institut National Polytechnique de Toulouse-INPT, 2011.</ref>, combined with a '''4th-order node-based centered finite-volume discretization''' of the convective and diffusive terms.
 Moreover, to improve the performance of the correction step, the pressure of the previous iteration is included in the prediction step to limit the '''splitting errors'''.
 The correction step is required to ensure '''mass conservation''', using a pressure that arises from a '''Poisson equation'''.
-This is performed numerically by solving a linear system on the pressure at each node of the mesh thanks to a dedicated in-house version of the deflated conjugate gradient algorithm, which has been optimized for solving elliptic equations on massively parallel machines<ref>M. Malandain, N. Maheu, and V. Moureau. Optimization of the deflated conjugate gradient algorithm for the solving of elliptic equations on massively parallel machines. J. Comput. Phys., 238:32–47, 2013.</ref>.
+This is performed numerically by solving a linear system on the pressure at each node of the mesh thanks to a dedicated in-house version of the deflated conjugate gradient algorithm, which has been optimized for solving elliptic equations on massively parallel machines<ref name="malandain2013"/>M. Malandain, N. Maheu, and V. Moureau. Optimization of the deflated conjugate gradient algorithm for the solving of elliptic equations on massively parallel machines. J. Comput. Phys., 238:32–47, 2013.</ref>.
-When considering '''multispecies''' and '''non-isothermal flows''', the extension of the classical projection method proposed by Pierce et al.<ref>C.D. Pierce and P. Moin. Progress-variable approach for large-eddy simulation of non-premixed turbulent combustion. J. Fluid Mech., 504:73–97, 2004.</ref> is used to account for '''variable-density flows'''.
+When considering '''multispecies''' and '''non-isothermal flows''', the extension of the classical projection method proposed by Pierce et al.<ref name="pierce2004"/>C.D. Pierce and P. Moin. Progress-variable approach for large-eddy simulation of non-premixed turbulent combustion. J. Fluid Mech., 504:73–97, 2004.</ref> is used to account for '''variable-density flows'''.
-All the thermodynamic and transport properties are provided by the '''Cantera software'''<ref>.G. Goodwin. An open-source, extensible software suite for CVD process simulation. Chemical Vapor Deposition XVI and EUROCVD, 14:2003–2008, 2003.</ref>, which has been fully re-implemented in '''Fortran''' to avoid any performance issues.
+All the thermodynamic and transport properties are provided by the '''Cantera software'''<ref name="goodwin2003"/>.G. Goodwin. An open-source, extensible software suite for CVD process simulation. Chemical Vapor Deposition XVI and EUROCVD, 14:2003–2008, 2003.</ref>, which has been fully re-implemented in '''Fortran''' to avoid any performance issues.
 Each species thermodynamic property is specified by '''5th-order polynomials''' on two temperature ranges (below and above 1000 K).
 The mixture is supposed to be both thermally and mechanically perfect.
@@ Line 29: / Line 29: @@
 Regarding transport properties, several type of models are implemented in YALES2:
-) the default approach is based on the computation of transport properties for each species (using tabulated molecular potentials), then combining those to obtain mixture-averaged coefficients for '''viscosity''', '''conductivity''' and '''diffusion velocities''' (Hirschfelder and Curtiss approximation<ref>J. Hirschfelder, R.B. Bird, and C.F. Curtiss. Molecular Theory of Gases and Liquids. Wiley, 1964.</ref>;
+) the default approach is based on the computation of transport properties for each species (using tabulated molecular potentials), then combining those to obtain mixture-averaged coefficients for '''viscosity''', '''conductivity''' and '''diffusion velocities''' (Hirschfelder and Curtiss approximation<ref name="hirschfelder1964"/>J. Hirschfelder, R.B. Bird, and C.F. Curtiss. Molecular Theory of Gases and Liquids. Wiley, 1964.</ref>;
-) alternatively, simplified laws (for example the Sutherland law<ref>W. Sutherland. LII. the viscosity of gases and molecular force. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 36(223):507–531, 1893.</ref> can be used for the '''viscosity''', while fixed values for '''Prandtl''' and '''Schmidt numbers''' allow the computation of '''conductivity''' and '''diffusion coefficients''';
+) alternatively, simplified laws (for example the Sutherland law<ref name="sutherland1893"/>W. Sutherland. LII. the viscosity of gases and molecular force. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 36(223):507–531, 1893.</ref> can be used for the '''viscosity''', while fixed values for '''Prandtl''' and '''Schmidt numbers''' allow the computation of '''conductivity''' and '''diffusion coefficients''';
 ) a mix of both approaches, for example computing '''viscosity''' and '''conductivity''' with a mixture-averaged approximation and then imposing a '''Lewis number''' for each species.
@@ Line 53: / Line 53: @@
 DINO is a '''3-D DNS code''' used for '''incompressible''' or '''low-Mach number flows''', the latter approach being used in this project.
 The development of DINO started in the group of D. Thévenin (Univ. of Magdeburg) in 2013.
-DINO is a '''Fortran-90 code''', written on top of a 2-D pencil decomposition to enable efficient large-scale parallel simulations on distributed-memory supercomputers by coupling with the open-source library 2-DECOMP&FFT<ref>N. Li and S. Laizet. 2DECOMP&FFT - a highly scalable 2D decomposition library and FFT interface. In Cray
+DINO is a '''Fortran-90 code''', written on top of a 2-D pencil decomposition to enable efficient large-scale parallel simulations on distributed-memory supercomputers by coupling with the open-source library 2-DECOMP&FFT<ref name="li2010"/>N. Li and S. Laizet. 2DECOMP&FFT - a highly scalable 2D decomposition library and FFT interface. In Cray
 User’s Group 2010 conference, Edinburgh, 2010.</ref>.
 The code offers different features and algorithms in order to investigate different physicochemical processes.
@@ Line 69: / Line 69: @@
 ) mixture-averaged,
-) full multicomponent diffusion, by coupling either again with '''Cantera''' or with the open-source library EGlib<ref>A. Ern and V. Giovangigli. Fast and accurate multicomponent transport property evaluation. J. Comput. Phys.,
+) full multicomponent diffusion, by coupling either again with '''Cantera''' or with the open-source library EGlib<ref name="ern1995"/>A. Ern and V. Giovangigli. Fast and accurate multicomponent transport property evaluation. J. Comput. Phys.,
 :105–116, 1995.</ref>.
@@ Line 75: / Line 75: @@
 The I/O operations are implemented using two different approaches: (1) binary '''MPI-I/O''' using 2-DECOMP&FFT for check-points and restart files; (2) HDF5 files used for analysis and visualization.
-'''Multi-phase flows''' can be simulated in DINO using resolved or non-resolved (point) particles and droplets using a '''Lagrangian approach'''<ref>A. Abdelsamie and D. Thévenin. On the behavior of spray combustion in a turbulent spatially-evolving jet investigated by direct numerical simulation. Proc. Combust. Inst., 37(3):2493–2502, 2019.</ref><ref>A. Abdelsamie and D. Thévenin. Nanoparticle behavior and formation in turbulent spray flames investigated by DNS. In M. Garcia-Villalba, H. Kuerten, and M. Salvetti, editors, Direct and Large Eddy Simulation XII, volume 27 of ERCOFTAC Series. Springer, 2020.</ref>. Complex boundaries are represented by a novel second-order immersed boundary method implementation ('''IBM''') based on a directional extrapolation scheme<ref>C. Chi, A. Abdelsamie, and D. Thévenin. A directional ghost-cell immersed boundary method for incompressible flows. J. Comput. Phys., 404:109122–109142, 2020.</ref>.
+'''Multi-phase flows''' can be simulated in DINO using resolved or non-resolved (point) particles and droplets using a '''Lagrangian approach'''<ref name="abdelsamie2019"/>A. Abdelsamie and D. Thévenin. On the behavior of spray combustion in a turbulent spatially-evolving jet investigated by direct numerical simulation. Proc. Combust. Inst., 37(3):2493–2502, 2019.</ref><ref name="abdelsamie2020"/>A. Abdelsamie and D. Thévenin. Nanoparticle behavior and formation in turbulent spray flames investigated by DNS. In M. Garcia-Villalba, H. Kuerten, and M. Salvetti, editors, Direct and Large Eddy Simulation XII, volume 27 of ERCOFTAC Series. Springer, 2020.</ref>. Complex boundaries are represented by a novel second-order immersed boundary method implementation ('''IBM''') based on a directional extrapolation scheme<ref name="chi2020"/>C. Chi, A. Abdelsamie, and D. Thévenin. A directional ghost-cell immersed boundary method for incompressible flows. J. Comput. Phys., 404:109122–109142, 2020.</ref>.
-More details about the implemented algorithms can be found in particular in<ref>A. Abdelsamie, G. Fru, F. Dietzsch, G. Janiga, and D. Thévenin. Towards direct numerical simulations of low-Mach number turbulent reacting and two-phase flows using immersed boundaries. Comput. Fluids, 131(5):123–141, 2016.</ref>.
+More details about the implemented algorithms can be found in particular in<ref name="abdelsamie2016"/>A. Abdelsamie, G. Fru, F. Dietzsch, G. Janiga, and D. Thévenin. Towards direct numerical simulations of low-Mach number turbulent reacting and two-phase flows using immersed boundaries. Comput. Fluids, 131(5):123–141, 2016.</ref>.
-Since DINO has been developed as a multi-purpose code for analyzing many different '''reacting''' and '''non-reacting flows'''<ref>C. Chi, A. Abdelsamie, and D. Thévenin. Direct numerical simulations of hotspot-induced ignition in homogeneous hydrogen-air pre-mixtures and ignition spot tracking. Flow Turbul. Combust., 101(1):103–121, 2018.</ref><ref>T. Oster, A. Abdelsamie, M. Motejat, T. Gerrits, C. R¨ossl, D. Thévenin, and H. Theisel. On-the-fly tracking of flame surfaces for the visual analysis of combustion processes. Comput. Graph. Forum, 37(6):358–369, 2018.</ref><ref>A. Abdelsamie and D. Thévenin. Impact of scalar dissipation rate on turbulent spray combustion investigated
+Since DINO has been developed as a multi-purpose code for analyzing many different '''reacting''' and '''non-reacting flows'''<ref name="chi2018"/>C. Chi, A. Abdelsamie, and D. Thévenin. Direct numerical simulations of hotspot-induced ignition in homogeneous hydrogen-air pre-mixtures and ignition spot tracking. Flow Turbul. Combust., 101(1):103–121, 2018.</ref><ref name="oster2018"/>T. Oster, A. Abdelsamie, M. Motejat, T. Gerrits, C. Rössl, D. Thévenin, and H. Theisel. On-the-fly tracking of flame surfaces for the visual analysis of combustion processes. Comput. Graph. Forum, 37(6):358–369, 2018.</ref><ref name="abdelsamie2019-2">A. Abdelsamie and D. Thévenin. Impact of scalar dissipation rate on turbulent spray combustion investigated
-by DNS. In M. Salvetti, V. Armenio, J. Fr¨ohlich, B. Geurts, and H. Kuerten, editors, Direct and Large-Eddy
+by DNS. In M. Salvetti, V. Armenio, J. Fröhlich, B. Geurts, and H. Kuerten, editors, Direct and Large-Eddy
 Simulation XI, volume 25 of ERCOFTAC Series. Springer, 2019.</ref>, a detailed verification and validation is obviously essential.
@@ Line 88: / Line 88: @@
 The computational domain is decomposed into <math>E</math> conforming elements, which are '''quadrilaterals''' ('''in 2-D''') or '''hexahedra''' (in '''3-D''') that conform to the domain boundaries.
 Within each element, functions are expanded as <math>N</math>th-order polynomials so that resolution can be increased either by decreasing the element size (<math>h</math>-type refinement) or by increasing the polynomial order (<math>p</math>-type refinement; typically <math>N = 7 - 15</math>).
-The grids can be unstructured and allow for '''static local refinement''', while '''adaptive mesh refinement''' has been recently developed<ref>A. Tanarro, F. Mallor, N. Offermans, A. Peplinski, R. Vinuesa, and P. Schlatter. Enabling adaptive mesh refinement for spectral-element simulations of turbulence around wing sections. Flow Turbul. Combust., 105(2):415–436, 2020.</ref>.
+The grids can be unstructured and allow for '''static local refinement''', while '''adaptive mesh refinement''' has been recently developed<ref name="tanarro2020"/>A. Tanarro, F. Mallor, N. Offermans, A. Peplinski, R. Vinuesa, and P. Schlatter. Enabling adaptive mesh refinement for spectral-element simulations of turbulence around wing sections. Flow Turbul. Combust., 105(2):415–436, 2020.</ref>.
 By casting the '''polynomial approximation''' in tensor-product form, the differential operators on <math>N^3</math> gridpoints per element can be evaluated with only <math>O(N^4)</math> work and <math>O(N^3)</math> storage.

@@ Line 175: / Line 175: @@
   | Operator splitting
   | Yes
-  | ???
+  | No
   | No
   |-

@@ Line 115: / Line 115: @@
 First, '''YALES2''' is an '''unstructured code''', designed to handle any type of elements; its main application field pertains to '''LES''' of industrially relevant flows, though '''DNS''' is possible as well.
 On the other hand, both '''DINO''' and '''Nek5000''' are mostly dedicated to '''DNS''' of configurations found in fundamental research.
-Both '''DINO''' and '''Nek5000''' can only deal with '''quads''' or '''hexas''', with the major difference that '''DINO''' is based on a '''structured connectivity''' while Nek5000 can use '''unstructured meshes''' (pavings).
+Both '''DINO''' and '''Nek5000''' can only deal with '''quads''' or '''hexas''', with the major difference that '''DINO''' is based on a '''structured connectivity''' while '''Nek5000''' can use '''unstructured meshes''' (pavings).
 As a consequence, both '''DINO''' and '''Nek5000''' employ '''higher-order numerical schemes''' compared to '''YALES2''', limited at best to a '''4th-order scheme'''.
 All codes rely on dedicated libraries to compute the thermo-chemical properties of the flow, either '''Chemkin''', '''Cantera''', or in-house versions of those.
@@ Line 121: / Line 121: @@
 Regarding the Poisson equation for pressure which must be solved by all codes, '''DINO''' relies on a spectral formulation by performing direct and inverse Fourier transforms, which is possible thanks to its structured mesh.
 On the other hand, both '''YALES2''' and '''Nek5000''' use an iterative solver with an efficient preconditioning technique; this method is more versatile and should be computationally more efficient for large and complex geometrical configurations.
-Nek5000 employs CVODE to integrate the thermochemical equations without further '''splitting''' of the different terms accounting for convection, diffusion and chemistry.
+'''Nek5000''' employs CVODE to integrate the thermochemical equations without further '''splitting''' of the different terms accounting for convection, diffusion and chemistry.
 The main differences between the three codes are summarized in Table~\ref{Tab:codes}.
 Please note that the presented values are those used for the benchmark, even though some other options are available in each codes.

@@ Line 110: / Line 110: @@
 The three '''aforementioned solvers''' are all '''unsteady''', '''high-fidelity codes''' based on the '''low-Mach number''' formulation of the '''Navier-Stokes equations'''.
-They can perform both Direct Numerical Simulations or Large Eddy Simulations of reacting flows, only DNS being considered here.
+They can perform both '''Direct Numerical Simulations''' or '''Large Eddy Simulations''' of reacting flows, only '''DNS''' being considered here.
 However, they differ in a certain number of points, mainly from the numerical point of view.
 The aim of this section is to emphasize those major differences.
-First, '''YALES2''' is an '''unstructured code''', designed to handle any type of elements; its main application field pertains to LES of industrially relevant flows, though DNS is possible as well.
+First, '''YALES2''' is an '''unstructured code''', designed to handle any type of elements; its main application field pertains to '''LES''' of industrially relevant flows, though '''DNS''' is possible as well.
-On the other hand, both '''DINO''' and '''Nek5000''' are mostly dedicated to DNS of configurations found in fundamental research.
+On the other hand, both '''DINO''' and '''Nek5000''' are mostly dedicated to '''DNS''' of configurations found in fundamental research.
 Both '''DINO''' and '''Nek5000''' can only deal with '''quads''' or '''hexas''', with the major difference that '''DINO''' is based on a '''structured connectivity''' while Nek5000 can use '''unstructured meshes''' (pavings).
-As a consequence, both '''DINO''' and '''Nek5000''' employ higher-order numerical schemes compared to '''YALES2''', limited at best to a 4th-order scheme.
+As a consequence, both '''DINO''' and '''Nek5000''' employ '''higher-order numerical schemes''' compared to '''YALES2''', limited at best to a '''4th-order scheme'''.
 All codes rely on dedicated libraries to compute the thermo-chemical properties of the flow, either '''Chemkin''', '''Cantera''', or in-house versions of those.
 Moreover, they also rely at least to some extent on external software to perform the temporal integration of the stiff chemical source terms.