3D Hydro Simulation using Spectral Method and Lattice Boltzmann Method:Accuracy and Performance Analysis
CISC879 Parallelization for Scientific Applications
Project 5
Advisors:
Dr. L. P. Wang
Dr. P. Dmitruk

Members:
Orlando Ayala
Ben Breech
Peng Gao


 

Motivation:

Turbulence is an unstable state of viscous fluid flow and is the most common flow regime in nature and in engineering applications. Air motion in the earth's atmosphere, ship wakes, flows in pipelines and chemical reactors are just a few examples. Turbulence produces rapid mixing and transport of passive pollutants and particulate matters. Analytical solutions to even the simplest turbulent flows do not exist.

Direct numerical simulation (DNS) provides a numerical solution of the flow variables as a function of space and time by numerically solving the exact mathematical equations governing a turbulent flow. DNS has been used effectively as a research tool since its invention in the earlier 70’s (Orszag and Patterson 1972). Significant insight into turbulence physics has been gained from DNS of certain idealized flows which cannot be easily attained in the laboratory (Moin and Mahesh 1998).  For example, for the case of isotropic turbulence, DNS have helped us (a) identify the small scale vortex tubes and their physical properties, (b) examine the hypotheses underlying the classical Kolmogorov-Obukhov theory of the inertial range, (c) explore the locality of the spectral energy transfer, (d) quantify mixing, structure, and turbulent combustion of passive scalar, and (e) reformulate statistical models for transport and coagulation of heavy particles. In quite a few cases, DNS data have been used to evaluate measurement accuracy (e.g., Moin & Mahesh 1998).  We believe that DNS, viewed as novel numerical experiments, continue to be a complementary tool to experiments in turbulence research. 

Given these remarkable contributions made in the recent years through DNS, a known limitation is the relatively low Reynolds number (to date, Rl is about or less than 250). A major effort has been to increase the flow Reynolds number.  This project represents a first step towards this goal by optimizing DNS codes on scalable parallel computers. 
 
 

Problem Statement:

Our faculty advisor, Dr. Wang, has recently developed two MPI codes to simulate homogeneous turbulence on scalable parallel computers. The codes are based on two distinct numerical methods.  The first or spectral code is based on a 3D Fourier representation of the Navier-Stokes equations (the governing equations for macroscopic fluid motion).  The second is based on the discretized  Boltzmann equation and as such is known as the lattice Boltzmann method (LBM).  The two methods solve completely different equations but macroscopically give the same turbulent flow field if they share a same initial condition.

The overall objective of this project is to analyze these codes carefully and test their performance on the DEC Alpha cluster so that their capabilities for simulating turbulence can be explored and compared.  
 
 

Plan and Tasks of Investigation:

The existing codes developed by Dr. Wang are intended for Beowulf clusters. Our first task will be to port the codes on the DEC Alpha cluster.  Dr. Pablo Dmitruk, who is well experienced with the spectral method and parallel computation, is helping us on migrating the codes.  Once the codes are operational on the DEC cluster, the three student members of the team are expected to perform the following tasks:   

1. Performance analysis of each code:  we will measure carefully the computation and communication costs by timing them separately for different problem sizes and computer nodes.  The expected costs will also be analyzed by counting the number of floating point operations and size of the data communications, which can be used to interpret the timing test results.  

2. Comparison of the two codes:  the performance results will be compared together with an analysis of the numerical accuracy of the two methods.  This can help us answer the question whether the LBM can be an alternative to spectral method for turbulence simulation.

3. Improvement of a MPI FFT subroutine in the spectral code: one limitation in the existing spectral code is that the code can only run on 2n nodes when a pairing strategy is used for data communication. In principle, this limitation can be removed.  If time permits, we will modify the code so that it can be run on any even node numbers.

4. A study of different index ordering of data array in the LBM code:  In the LBM code, there are several possibilities for the order of indices of the main data array.  Apparently, the speed of the code can depend on this ordering since the efficiency of data retrieval process depend on the ordering. We may compare the code performance for different index ordering strategies.

The group is expected to meet once a week to discuss the project progress and to enhance communications among its members.
 
 

FINAL WRITTEN REPORT


 
 

References:

Moin P and K. Makesh 1998 Annu Rev Fluid Mech. 30: 539-578.

Orszag, S.A. and Patterson, G.S. 1972 Phys. Rev. Lett. 28: 76-79.
 



Last update: September 20, 2000