next up previous index
Next: Create an MPI-IO File Up: Property Lists Previous: Modifying Property Lists

Create a File Family

This simple example code creates a file family and then fills the family with data. Even though physically we are going to write on multiple files, logically, we treat the IO operation like a single HDF5 dataset write.

Here is the code (this example is also taken from the NCSA HDF5 Tutorial):

 *  This example writes data to the HDF5 file.
 *  Data conversion is performed during write operation.  
#include "hdf5.h"

#define DIM0   40
#define DIM1   60

#define H5FILE_NAME        "SDS%d.h5"
main (void)
    hid_t       file, dataset;         /* handles */
    hid_t       dataspace;             
    hid_t       plist_id;
    int         i, j, dset_data[DIM0][DIM1];
    hsize_t     family_size = 1000; 
    hsize_t     dims[2] = {DIM0,DIM1};
    herr_t      status;

    plist_id = H5Pcreate(H5P_FILE_ACCESS);
    status = H5Pset_fapl_family(plist_id, family_size, H5P_DEFAULT);

    file = H5Fcreate(H5FILE_NAME, H5F_ACC_TRUNC, H5P_DEFAULT, plist_id);
    dataspace = H5Screate_simple(2, dims, NULL);
    dataset = H5Dcreate (file, "family-dataset", H5T_NATIVE_INT, dataspace,

    for (i = 0; i < DIM0; i++)
      for (j = 0; j < DIM1; j++)
         dset_data[i][j] = i * 6 + j + 1;

    status = H5Dwrite(dataset, H5T_NATIVE_INT, H5S_ALL, H5S_ALL, H5P_DEFAULT,
    status = H5Dclose(dataset);
    status = H5Sclose(dataspace);
    status = H5Fclose(file);
The program is compiled with:
gustav@bh1 $ h5cc -o h5_family h5_family.c
gustav@bh1 $
and then it is run by simply typing its name:
gustav@bh1 $ ./h5_family
gustav@bh1 $
After the program returns, you can see that it has generated 12 data files:
gustav@bh1 $ ls
SDS0.h5   SDS11.h5  SDS4.h5  SDS7.h5  h5_family
SDS1.h5   SDS2.h5   SDS5.h5  SDS8.h5  h5_family.c
SDS10.h5  SDS3.h5   SDS6.h5  SDS9.h5  h5_family.o
gustav@bh1 $
You cannot view any of the HDF5 data files in this family with a simple call to h5dump followed by an explicit name of any of the files. To view this data do as follows:
gustav@bh1 $ h5dump -f family SDS%d.h5
HDF5 "SDS%d.h5" {
GROUP "/" {
   DATASET "family-dataset" {
      DATASPACE  SIMPLE { ( 40, 60 ) / ( 40, 60 ) }
      DATA {
         1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20,


         277, 278, 279, 280, 281, 282, 283, 284, 285, 286, 287, 288, 289, 290,
         291, 292, 293, 294
gustav@bh1 $
You can also call h5dump without the -f family switch, in which case h5dump will try five different drivers against the file name you have given it, eventually finding that it can read it with the family driver.

Let us have a look at the program itself now. We begin by creating the default property list for file access with H5Pcreate and then modify it by calling  H5Pset_fapl_family:

    plist_id = H5Pcreate(H5P_FILE_ACCESS);
    status = H5Pset_fapl_family(plist_id, family_size, H5P_DEFAULT);
Function H5Pset_fapl_family takes the property list identifier as its first argument. The second argument is the size of each file family member in bytes. The third argument is the property list to be used for each of the file family member, in this case just H5P_DEFAULT.

Now we create the HDF5 logical file itself:

    file = H5Fcreate(H5FILE_NAME, H5F_ACC_TRUNC, H5P_DEFAULT, plist_id);
Observe that H5FILE_NAME is defined as "SDS%d.h5". Once the file family is created %d will be replaced with an appropriate number. The third argument in the call to H5Fcreate is the file create property list, whereas the fourth argument is the file access property list. The specific property we are talking about here, i.e., the property that the file is to be split into the family, is an access property, and so it is the fourth argument that has been cutomized to convey our request.

The next couple of steps in the program are standard. We create a simple data space, then we create the dataset, then we initialize the data in memory and write the data on the dataset.

Finally we close the dataset, close the dataspace and close the file.

At no stage do we have to worry about dealing with multiple files. HDF5 takes care of this. Logically, we have opened a single HDF5 file for writing in this program.

next up previous index
Next: Create an MPI-IO File Up: Property Lists Previous: Modifying Property Lists
Zdzislaw Meglicki