SHORT-FORMAT FILES

Created On: 13 July, 1997
Last Modified: 4 November, 1997


Overview


Introduction

The native file format of CleanVec and other software in the Laboratory for Turbulence and Complex Flow is short format. Short format was developed by Prof. Carl D. Meinhart as a compact binary representation for two-dimensional fields of PIV data. However, because of its flexible data storage capability, it can be used for much more than standard PIV data. In fact, it is capable of storing almost any type of 1-D or 2-D data field, including

The only assumption made in short format is that the underlying grid of data is uniform Cartesian, which is not restrictive since most (if not all) interrogation software outputs data onto this type of grid.


File Content

Since short format is a binary format, the content of this file type is outlined below for reference. Each variable is named and described. Their data types and sizes (in bytes) are listed, as well. The order in which the variables appear in the table below is the same as in the file.


Variable Type Size
(bytes)
Description
Columns Integer 4 Number of columns on the grid.
Rows Integer 4 Number of rows on the grid.
Values Integer 4 Number of data values stored at each grid point.
Photos Integer 4 Number of photographs (realizations) used to generate the values in this file. This variable is primarily used for statistical analysis software that performs ensemble averages.
X0 Real 4 x-location of grid origin (lower-left corner).
Y0 Real 4 y-location of grid origin (lower-left corner).
Delta X Real 4 Spacing between grid points in the x-direction.
Delta Y Real 4 Spacing between grid points in the y-direction.
Comments Integer 4 Number of comments contained in the comment history.
Comment History String Array 80 * Comments Comment history. These comments indicate what operations have been performed on the short-format file. Used primarily by statistical analysis software.
Data Values Real Array 4 * Rows * Columns * Values Array of data values stored at each grid point.
Data Weights Integer Array 4 * Rows * Columns Array of data weights at each grid point. Raw data normally has a weight of "1" at each grid point. Used primarily by statistical analysis software.


Sample Code for Reading and Writing Short-Format Files

Below is a listing of source code that contains two functions which can be used to read and write short-format files. These functions are written in C++ and make use of the Microsoft® Foundation Classes (MFC) library (version 4.0 or higher), although it could be easily changed to make use of another library, such as the standard C++ library.

///////////////////////////////////////////////////////////////////////////// // // HOW TO READ AND WRITE SHORT-FORMAT FILES // //--------------------------------------------------------------------------- // // This sample contains two subroutines that illustrate how one could read // and write short-format files. Both routines use the CShortFormatContent // structure, which, as the name implies, is responsible for storing the // content of a single short-format file. // // The sample has been written in C++ using the Microsoft(R) Foundation // Classes (MFC) library. If you are not familiar with MFC, most of the // constructs can be easily extended to other libraries, such as the standard // C++ library. // ///////////////////////////////////////////////////////////////////////////// #include // MFC core components #include // MFC templates ///////////////////////////////////////////////////////////////////////////// // Define Alias Types typedef float REAL; typedef CArray CREALArray; typedef CArray CIntArray; ///////////////////////////////////////////////////////////////////////////// // Define Constants static const int COMMENT_LENGTH = 80; ///////////////////////////////////////////////////////////////////////////// // User-Defined Types ///////////////////////////////////////////////////////////////////////////// // // CShortFormatContent Structure // //--------------------------------------------------------------------------- // // This structure stores the contents of a short-format file. It is used // by the routines ReadShortFormatFile() and WriteShortFormatFile(). // ///////////////////////////////////////////////////////////////////////////// struct CShortFormatContent { // The following attributes represent the components of the short-format // file's header. // int nRows, // Number of rows in the grid nCols, // Number of columns in the grid nValues, // Number of values stored at each grid point nPhotos; // Number of photos (realizations) used to // generate the values in this file REAL rX0, // x-coordinate of origin (real units) rY0, // y-coordinate of origin (real units) rDeltaX, // Grid point spacing in x-direction (real units) rDeltaY; // Grid point spacing in y-direction (real units) CStringArray astrComments; // Comment history // The following attributes represent the data content of the short-format // file. // CREALArray arValues; // Implicit 3-D array that stores all the data values // at each grid point CIntArray anPoints; // Implicit 2-D array of weights for each grid point // (defaults to 1) }; ///////////////////////////////////////////////////////////////////////////// // Function Prototypes BOOL ReadShortFormatFile( LPCTSTR, CShortFormatContent& ); BOOL WriteShortFormatFile( LPCTSTR, const CShortFormatContent& ); ///////////////////////////////////////////////////////////////////////////// // Function Definitions //=========================================================================== // // ReadShortFormatFile() // //--------------------------------------------------------------------------- // // PURPOSE: This function reads in the specified short-format file and stores // its contents in the CShortFormatContent structure specified by // content. // // INPUTS: lpszFileName: Path to the short-format file to be read. // content: Structure which will receive the contents of the short- // format file. // // RETURNS: [BOOL] Nonzero if the file was read successfully; otherwise 0. // //=========================================================================== BOOL ReadShortFormatFile( LPCTSTR lpszFileName, CShortFormatContent& content ) { ///////////////////////////////////////////////////////////////////////// // VARIABLE DECLARATIONS // CFile file; // File used to read object from disk char szBuffer[ COMMENT_LENGTH ]; // Buffer used for reading header comments int nComments, // Number of comments in header nI; // Loop control variable // // ///////////////////////////////////////////////////////////////////////// // Attempt to open the file in read mode if( !file.Open( lpszFileName, CFile::modeRead | CFile::shareDenyWrite ) ) return( FALSE ); try { // Read in the general parameters from the short-format header file.Read( &content.nCols, sizeof( int ) ); file.Read( &content.nRows, sizeof( int ) ); file.Read( &content.nValues, sizeof( int ) ); file.Read( &content.nPhotos, sizeof( int ) ); file.Read( &content.rX0, sizeof( REAL ) ); file.Read( &content.rY0, sizeof( REAL ) ); file.Read( &content.rDeltaX, sizeof( REAL ) ); file.Read( &content.rDeltaY, sizeof( REAL ) ); // Read in the number of comments and set the size of the comment array file.Read( &nComments, sizeof( int ) ); content.astrComments.SetSize( nComments ); // Read in each comment for( nI = 0; nI < nComments; nI++ ) { file.Read( szBuffer, COMMENT_LENGTH ); content.astrComments[ nI ] = szBuffer; } // Allocate memory for the Values and Points arrays and read in each one content.arValues.SetSize( content.nCols * content.nRows * content.nValues ); content.anPoints.SetSize( content.nCols * content.nRows ); file.Read( content.arValues.GetData(), content.arValues.GetSize() * sizeof( REAL ) ); file.Read( content.anPoints.GetData(), content.anPoints.GetSize() * sizeof( int ) ); // Close the file object file.Close(); // File was read successfully return( TRUE ); } catch( CFileException* pE ) { // Do default exception processing pE->ReportError(); pE->Delete(); return( FALSE ); } catch( CMemoryException* pE ) { // Do default exception processing pE->ReportError(); pE->Delete(); return( FALSE ); } } //=========================================================================== // // WriteShortFormatFile() // //--------------------------------------------------------------------------- // // PURPOSE: This function writes the CShortFormatContent structure specified // by content to the file given by lpszFileName. // // INPUTS: lpszFileName: Path to the short-format file to be created. // content: Structure which contains the contents of the short- // format file. // // RETURNS: [BOOL] Nonzero if the file was written successfully; otherwise 0. // //=========================================================================== BOOL WriteShortFormatFile( LPCTSTR lpszFileName, const CShortFormatContent& content ) { ///////////////////////////////////////////////////////////////////////// // VARIABLE DECLARATIONS // CFile file; // File used to write object to disk int nComments, // Number of comments in the header nI; // Loop control variable // // ///////////////////////////////////////////////////////////////////////// // Attempt to open the file in write mode if( !file.Open( lpszFileName, CFile::modeWrite | CFile::modeCreate | CFile::shareDenyWrite ) ) return( FALSE ); try { // Write the general parameters of the short-format header file.Write( &content.nCols, sizeof( int ) ); file.Write( &content.nRows, sizeof( int ) ); file.Write( &content.nValues, sizeof( int ) ); file.Write( &content.nPhotos, sizeof( int ) ); file.Write( &content.rX0, sizeof( REAL ) ); file.Write( &content.rY0, sizeof( REAL ) ); file.Write( &content.rDeltaX, sizeof( REAL ) ); file.Write( &content.rDeltaY, sizeof( REAL ) ); // Write the number of comments nComments = content.astrComments.GetSize(); file.Write( &nComments, sizeof( int ) ); // Write each comment (NOTE: we must make sure each comment is COMMENT_LENGTH // characters long, otherwise we face a possible invalid address problem) for( nI = 0; nI < nComments; nI++ ) { file.Write( content.astrComments[ nI ].GetBuffer( COMMENT_LENGTH ), COMMENT_LENGTH ); content.astrComments[ nI ].ReleaseBuffer(); } // Write the Values and Points arrays file.Write( content.arValues.GetData(), content.arValues.GetSize() * sizeof( REAL ) ); file.Write( content.anPoints.GetData(), content.anPoints.GetSize() * sizeof( int ) ); // Close the file object file.Close(); // File was written successfully return( TRUE ); } catch( CFileException* pE ) { // Do default exception processing pE->ReportError(); pE->Delete(); return( FALSE ); } }

Ordering of Multi-Dimensional Arrays

Both the Data Values and the Data Weights arrays implicitly represent multi-dimensional arrays. Specifically, the Data Values array is three-dimensional, while the Data Weights array is two-dimensional. The ordering of elements in these arrays is described below.


Data Values Array

The elements in the Data Values array are indexed as (column, row, value). Where the first array index (column) varies the fastest. The sample code below illustrates how one would iterate through the Data Values array.

// nCol, nRow, nVal: Loop control variables // nCols, nRows: Size of grid // nVals: Number of values stored at each grid point // nIndex: Equivalent 1-D index given a set of 3-D indices // arValues: 1-D array of data values // rValue: Current data value for( nCol = 0; nCol < nCols; nCol++ ) { for( nRow = 0; nRow < nRows; nRow++ ) { for( nVal = 0; nVal < nVals; nVal++ ) { nIndex = nCols * (nVal * nRows + nRow) + nCol; rValue = arValues[ nIndex ]; } } }
Data Weights Array

The elements in the Data Weights array are indexed as (column, row). Where the first array index (column) varies the fastest. The sample code below illustrates how one would iterate through the Data Weights array.

// nCol, nRow, nVal: Loop control variables // nCols, nRows: Size of grid // nIndex: Equivalent 1-D index given a set of 2-D indices // anWeights: 1-D array of data weights // nWeight: Current data weight for( nCol = 0; nCol < nCols; nCol++ ) { for( nRow = 0; nRow < nRows; nRow++ ) { nIndex = nCols * nRow + nCol; nWeight = anWeights[ nIndex ]; } }
Download Tech Note with Source Code