File management

Maia supports HDF5/CGNS file reading and writing, see related documention.

The IO functions are provided by the maia.io module. The low level hdf calls are performed by the python module h5py. Note that both the hdf5 library and the h5py module must have been installed on your computer with parallel support.

All the IO functions accept str or Path objects for filename argument. For write operations, intermediate directories are created if they do not already exist.

Distributed IO

Distributed IO is the privileged way to deal with CGNS files within your maia workflows. Files are loaded as distributed trees, and, inversely, distributed trees can be written into a single CGNS file.

High level IO operations can be performed with the two following functions, which read or write all data they found :

file_to_dist_tree(filename, comm)

Distributed load of a CGNS file.

Parameters

  • filename (str) – Path of the file

  • comm (MPIComm) – MPI communicator

Returns

CGNSTree – Distributed CGNS tree

dist_tree_to_file(dist_tree, filename, comm, links=[])

Distributed write to a CGNS file.

If links are used, the link description list must be identiqual on all ranks.

Parameters

  • dist_tree (CGNSDistTree) – Distributed tree to write

  • filename (str) – Path of the file

  • links (list) – List of links to create (see SIDS-to-Python guide)

  • comm (MPIComm) – MPI communicator

The example below shows how to uses these high level functions:

from mpi4py import MPI
import maia

# Generate a sample tree
dist_tree = maia.factory.generate_dist_block(10, "Poly", MPI.COMM_WORLD)
# Write
maia.io.dist_tree_to_file(dist_tree, "tree.cgns", MPI.COMM_WORLD)
# Read
tree = maia.io.file_to_dist_tree("tree.cgns", MPI.COMM_WORLD)

Finer control of what is written or loaded can be achieved with the following steps:

  • For a write operation, the easiest way to write only some nodes in the file is to remove the unwanted nodes from the distributed tree.

  • For a read operation, the load has to be divided into the following steps:

    • Loading a size_tree: this tree has only the shape of the distributed data and not the data itself.

    • Removing unwanted nodes in the size tree;

    • Fill the filtered tree from the file.

The example below illustrate how to filter the written or loaded nodes:

from mpi4py import MPI
import maia

# Generate a sample tree
dist_tree = maia.factory.generate_dist_block(10, "Poly", MPI.COMM_WORLD)

# Remove the nodes we do not want to write
maia.pytree.rm_nodes_from_name(dist_tree, 'CoordinateZ') #This is a DataArray
maia.pytree.rm_nodes_from_name(dist_tree, 'Zm*') #This is some BC nodes
# Write
maia.io.dist_tree_to_file(dist_tree, "tree.cgns", MPI.COMM_WORLD)

# Read
from maia.io.cgns_io_tree import load_size_tree, fill_size_tree
dist_tree = load_size_tree("tree.cgns", MPI.COMM_WORLD)
#For now dist_tree only contains sizes : let's filter it
maia.pytree.rm_nodes_from_name(dist_tree, 'CoordinateY') #This is a DataArray
maia.pytree.rm_nodes_from_name(dist_tree, 'Ym*') #This is some BC nodes
fill_size_tree(dist_tree, "tree.cgns", MPI.COMM_WORLD)

Partitioned IO

In some cases, it may be useful to write or read a partitioned tree (keeping the partitioned zones separated). This can be achieved using the following functions:

part_tree_to_file(part_tree, filename, comm, single_file=False, links=[])

Gather the partitioned zones managed by all the processes and write it in a unique hdf container.

If single_file is True, one file named filename storing all the partitioned zones is written. Otherwise, hdf links are used to produce a main file filename linking to additional subfiles.

Parameters

  • part_tree (CGNSPartTree) – Partitioned tree

  • filename (str) – Path of the output file

  • comm (MPIComm) – MPI communicator

  • single_file (bool) – Produce a unique file if True; use CGNS links otherwise.

  • links (list) – List of links to create (see SIDS-to-Python guide). Each rank must provide only the links related to one of its partitions.

Example

from mpi4py import MPI
import maia

dist_tree = maia.factory.generate_dist_block(10, "Poly", MPI.COMM_WORLD)
part_tree = maia.factory.partition_dist_tree(dist_tree, MPI.COMM_WORLD)

maia.io.part_tree_to_file(part_tree, 'part_tree.cgns', MPI.COMM_WORLD)
file_to_part_tree(filename, comm, redispatch=False)

Read the partitioned zones from a hdf container and affect them to the ranks.

If redispatch == False, the CGNS zones are affected to the rank indicated in their name. The size of the MPI communicator must thus be equal to the highest id appearing in partitioned zone names.

If redispatch == True, the CGNS zones are dispatched over the available processes, and renamed to follow maia’s conventions.

Important

This function does not perfom the partitioning operation; input file is supposed to contain an already partitioned tree, eg. saved with part_tree_to_file().

Parameters

  • filename (str) – Path of the file

  • comm (MPIComm) – MPI communicator

  • redispatch (bool) – Controls the affectation of the partitions to the available ranks (see above). Defaults to False.

Returns

CGNSTree – Partitioned CGNS tree

Raw IO

For debug purpose, trees can also be read or written independently in a sequential manner. Be aware that information added by maia such as Distribution or GlobalNumbering nodes will not be removed.

read_tree(filename)

Sequential load of a CGNS file.

Parameters

filename (str) – Path of the file

Returns

CGNSTree – Full (not distributed) CGNS tree

Detect the links embedded in a CGNS file.

Links information are returned as described in sids-to-python. Note that no data are loaded and the tree structure is not even built.

Parameters

filename (str) – Path of the file

Returns

list – Links description

write_tree(tree, filename, links=[])

Sequential write to a CGNS file.

Parameters

  • tree (CGNSTree) – Tree to write

  • filename (str) – Path of the file

  • links (list) – List of links to create (see SIDS-to-Python guide)

Example

from mpi4py import MPI
import maia

dist_tree = maia.factory.generate_dist_block(10, "Poly", MPI.COMM_WORLD)
if MPI.COMM_WORLD.Get_rank() == 0:
  maia.io.write_tree(dist_tree, "tree.cgns")
write_trees(tree, filename, comm, links=[])

Sequential write to CGNS files.

Write separate trees for each process. Rank id will be automatically inserted in the filename. If links are used, each rank must provide its own link description list.

Parameters

  • tree (CGNSTree) – Tree to write

  • filename (str) – Path of the file

  • links (list) – List of links to create (see SIDS-to-Python guide)

  • comm (MPIComm) – MPI communicator

Example

from mpi4py import MPI
import maia

dist_tree = maia.factory.generate_dist_block(10, "Poly", MPI.COMM_WORLD)
maia.io.write_trees(dist_tree, "tree.cgns", MPI.COMM_WORLD)