File management

Maia supports HDF5/CGNS file reading and writing, see related documention.

The IO functions are provided by the maia.io module. The low level hdf calls are performed by the python module h5py. Note that both the hdf5 library and the h5py module must have been installed on your computer with parallel support.

All the IO functions accept str or Path objects for filename argument. For write operations, intermediate directories are created if they do not already exist.

A note about 32 characters

CGNS is infamous for its names being limited to 32 characters. However, since the Python/CGNS mapping does not technically enforce this limitation, Maia allows trees having names longer than 32 chars in IO functions. This is achived throught the following hack:

when writing files, nodes having a name longer than 32 chars are shortened, the original name beeing stored in a special Descriptor_t child called ‘FullNameLongerThan32CharactersLi’.
when reading files, nodes having the special ‘FullNameLongerThan32CharactersLi’ child have their name updated with the registered value. The descriptor node is then removed.

The same apply to links: users should provide original (possibly long) paths to write functions, and automatically get original paths when calling read_links().

This ensure a transparent processus for the user, but keep in mind that cross tree references are not updated: for exemple, FamilyName_t values are not updated, even if the matching family is shortened. This can confuse applications not using Maia to read files. In addition, if such application remove the special Descriptor children nodes from file, original names will be lost.

Hint

Users can see the shortened ↔ original names mapping of a hdf file using maia_show_full_names utility.

Distributed IO

Distributed IO is the privileged way to deal with CGNS files within your maia workflows. Files are loaded as distributed trees, and, inversely, distributed trees can be written into a single CGNS file.

High level IO operations can be performed with the two following functions, which read or write all data they found :

file_to_dist_tree(filename, comm)

Distributed load of a CGNS file.

Parameters:

filename (str) – Path of the file
comm (MPIComm) – MPI communicator

Returns:

CGNSTree – Distributed CGNS tree

dist_tree_to_file(dist_tree, filename, comm, links=[])

Distributed write to a CGNS file.

If links are used, the link description list must be identiqual on all ranks.

Parameters:

dist_tree (CGNSDistTree) – Distributed tree to write
filename (str) – Path of the file
links (list) – List of links to create (see SIDS-to-Python guide)
comm (MPIComm) – MPI communicator

The example below shows how to uses these high level functions:

from mpi4py import MPI
import maia

# Generate a sample tree
dist_tree = maia.factory.generate_dist_block(10, "Poly", MPI.COMM_WORLD)
# Write
maia.io.dist_tree_to_file(dist_tree, "tree.cgns", MPI.COMM_WORLD)
# Read
tree = maia.io.file_to_dist_tree("tree.cgns", MPI.COMM_WORLD)

Finer control of what is written or loaded can be achieved with the following steps:

For a write operation, the easiest way to write only some nodes in the file is to remove the unwanted nodes from the distributed tree.
For a read operation, the load has to be divided into the following steps:
- Loading a size_tree: this tree has only the shape of the distributed data and not the data itself.
- Removing unwanted nodes in the size tree;
- Fill the filtered tree from the file.

The example below illustrate how to filter the written or loaded nodes:

from mpi4py import MPI
import maia

# Generate a sample tree
dist_tree = maia.factory.generate_dist_block(10, "Poly", MPI.COMM_WORLD)

# Remove the nodes we do not want to write
maia.pytree.rm_nodes_from_name(dist_tree, 'CoordinateZ') #This is a DataArray
maia.pytree.rm_nodes_from_name(dist_tree, 'Zm*') #This is some BC nodes
# Write
maia.io.dist_tree_to_file(dist_tree, "tree.cgns", MPI.COMM_WORLD)

# Read
from maia.io.cgns_io_tree import load_size_tree, fill_size_tree
dist_tree = load_size_tree("tree.cgns", MPI.COMM_WORLD)
#For now dist_tree only contains sizes : let's filter it
maia.pytree.rm_nodes_from_name(dist_tree, 'CoordinateY') #This is a DataArray
maia.pytree.rm_nodes_from_name(dist_tree, 'Ym*') #This is some BC nodes
fill_size_tree(dist_tree, "tree.cgns", MPI.COMM_WORLD)

Partitioned IO

In some cases, it may be useful to write or read a partitioned tree (keeping the partitioned zones separated). This can be achieved using the following functions:

part_tree_to_file(part_tree, filename, comm, single_file=False, links=[])

Gather the partitioned zones managed by all the processes and write it in a unique hdf container.

If single_file is True, one file named filename storing all the partitioned zones is written. Otherwise, hdf links are used to produce a main file filename linking to additional subfiles.

Parameters:

part_tree (CGNSPartTree) – Partitioned tree
filename (str) – Path of the output file
comm (MPIComm) – MPI communicator
single_file (bool) – Produce a unique file if True; use CGNS links otherwise.
links (list) – List of links to create (see SIDS-to-Python guide). Each rank must provide only the links related to one of its partitions.

Example

from mpi4py import MPI
import maia

dist_tree = maia.factory.generate_dist_block(10, "Poly", MPI.COMM_WORLD)
part_tree = maia.factory.partition_dist_tree(dist_tree, MPI.COMM_WORLD)

maia.io.part_tree_to_file(part_tree, 'part_tree.cgns', MPI.COMM_WORLD)

file_to_part_tree(filename, comm, redispatch=False)

Read the partitioned zones from a hdf container and affect them to the ranks.

If redispatch == False, the CGNS zones are affected to the rank indicated in their name. The size of the MPI communicator must thus be equal to the highest id appearing in partitioned zone names.

If redispatch == True, the CGNS zones are dispatched over the available processes, and renamed to follow maia’s conventions.

Important

This function does not perfom the partitioning operation; input file is supposed to contain an already partitioned tree, eg. saved with part_tree_to_file().

Parameters:

filename (str) – Path of the file
comm (MPIComm) – MPI communicator
redispatch (bool) – Controls the affectation of the partitions to the available ranks (see above). Defaults to False.

Returns:

CGNSTree – Partitioned CGNS tree

Raw IO

For debug purpose, trees can also be read or written independently in a sequential manner. Be aware that information added by maia such as Distribution or GlobalNumbering nodes will not be removed.

read_tree(filename)

Sequential load of a CGNS file.

Parameters:: filename (str) – Path of the file
Returns:: CGNSTree – Full (not distributed) CGNS tree

read_links(filename)

Detect the links embedded in a CGNS file.

Links information are returned as described in sids-to-python. Note that no data are loaded and the tree structure is not even built.

Parameters:: filename (str) – Path of the file
Returns:: list – Links description

write_tree(tree, filename, links=[])

Sequential write to a CGNS file.

Parameters:

tree (CGNSTree) – Tree to write
filename (str) – Path of the file
links (list) – List of links to create (see SIDS-to-Python guide)

Example

from mpi4py import MPI
import maia

dist_tree = maia.factory.generate_dist_block(10, "Poly", MPI.COMM_WORLD)
if MPI.COMM_WORLD.Get_rank() == 0:
  maia.io.write_tree(dist_tree, "tree.cgns")

write_trees(tree, filename, comm, links=[])

Sequential write to CGNS files.

Write separate trees for each process. Rank id will be automatically inserted in the filename. If links are used, each rank must provide its own link description list.

Parameters:

tree (CGNSTree) – Tree to write
filename (str) – Path of the file
links (list) – List of links to create (see SIDS-to-Python guide)
comm (MPIComm) – MPI communicator

Example

from mpi4py import MPI
import maia

dist_tree = maia.factory.generate_dist_block(10, "Poly", MPI.COMM_WORLD)
maia.io.write_trees(dist_tree, "tree.cgns", MPI.COMM_WORLD)