H5PY

HDF5 for Python (H5PY) is a general-purpose Python interface to the Hierarchical Data Format library, version 5. HDF5 was designed to be a versatile, mature scientific software library designed for the fast, flexible storage of enormous amounts of data.

From a Python programmer's perspective, HDF5 provides a robust way to store data, organized by name in a tree-like fashion.

You can create datasets (arrays on disk) hundreds of gigabytes in size, and perform random-access I/O on desired sections. Datasets are organized in a filesystem-like hierarchy using containers called "groups", and accessed using the tradional POSIX /path/to/resource syntax.

H5py provides a simple, robust read/write interface to HDF5 data from Python. Existing Python and Numpy concepts are used for the interface; for example, datasets on disk are represented by a proxy class that supports slicing, and has dtype and shape attributes. HDF5 groups are presented using a dictionary metaphor, indexed by name.

A major design goal of h5py is interoperability; you can read your existing data in HDF5 format, and create new files that any HDF5- aware program can understand. No Python-specific extensions are used; you're free to implement whatever file structure your application desires.

Almost all HDF5 features are available from Python, including things like compound datatypes (as used with Numpy recarray types), HDF5 attributes, hyperslab and point-based I/O, and more recent features in HDF 1.8 like resizable datasets and recursive iteration over entire files.

The foundation of h5py is a near-complete wrapping of the HDF5 C API. HDF5 identifiers are first-class objects which participate in Python reference counting, and expose the C API via methods.

This low-level interface is also made available to Python programmers, and is exhaustively documented.