HDF5 for Python (H5PY) is a general-purpose Python interface to the Hierarchical Data Format library, version 5. HDF5 was designed to be a versatile, mature scientific software library designed for the fast, flexible storage of enormous amounts of data.
From a Python programmer's perspective, HDF5 provides a robust way to store data, organized by name in a tree-like fashion. You can create datasets (arrays on disk) hundreds of gigabytes in size, and perform random-access I/O on desired sections. Datasets are organized in a filesystem-like hierarchy using containers called "groups", and accessed using the tradional POSIX /path/to/resource syntax.
H5py provides a simple, robust read/write interface to HDF5 data from Python. Existing Python and Numpy concepts are used for the interface; for example, datasets on disk are represented by a proxy class that supports slicing, and has dtype and shape attributes. HDF5 groups are presented using a dictionary metaphor, indexed by name.
A major design goal of h5py is interoperability; you can read your existing data in HDF5 format, and create new files that any HDF5- aware program can understand. No Python-specific extensions are used; you're free to implement whatever file structure your application desires.
Almost all HDF5 features are available from Python, including things like compound datatypes (as used with Numpy recarray types), HDF5 attributes, hyperslab and point-based I/O, and more recent features in HDF 1.8 like resizable datasets and recursive iteration over entire files.
The foundation of h5py is a near-complete wrapping of the HDF5 C API. HDF5 identifiers are first-class objects which participate in Python reference counting, and expose the C API via methods. This low-level interface is also made available to Python programmers, and is exhaustively documented.
Here are some key features of "H5PY":
· HDF5 does not limit the size of files or the size or number of objects in a file.
· The HDF5 format and library are extensible and designed to evolve gracefully to satisfy new demands.
· HDF5 functionality and data is portable across virtually all computing platforms and is distributed with C, C , Java, and Fortran90 programming interfaces.
· HDF5 has a simple but versatile data model.
· The HDF5 data model supports complex data relationships and dependencies through its grouping and linking mechanisms.
· HDF5 accommodates many common types of metadata and arbitrary user-defined metadata.
· HDF5 supports a rich set of pre-defined datatypes as well as the creation of an unlimited variety of complex user-defined datatypes.
· Datatype definitions can be shared among objects in an HDF file, providing a powerful and efficient mechanism for describing data.
· Datatype definitions include information such as byte order (endian), size, and floating point representation, to fully describe how the data is stored, insuring portability to other platforms.
· HDF5, through its virtual file layer, offers extremely flexible storage and data transfer capabilities. Standard (Posix), Parallel, and Network I/O file drivers are provided with HDF5.
· Application developers can write additional file drivers to implement customized data storage or transport capabilities.
· The parallel I/O driver for HDF5 reduces access times on parallel systems by reading/writing multiple data streams simultaneously.
· HDF5 employs various compression, extensibility, and chunking strategies to improve access, management, and storage efficiency.
· HDF5 provides for external storage of raw data, allowing raw data to be shared among HDF5 files and/or applications, and often saving disk space.
· HDF5 enables datatype and spatial transformation during I/O operations.
· HDF5 data I/O functions can operate on selected subsets of the data, reducing transferred data volume and improving access speed.
Requirements:
· Python 2.5
What's New in This Release: [ read full changelog ]
Enhancements unlikely to affect compatibility:
· HDF5 1.8.3 through 1.8.7 now work correctly and are officially supported
· Python 3.2 is officially supported by h5py! Thanks especially to Darren Dale for getting this working
· Fill values can now be specified when creating a dataset. The fill time is H5D_FILL_TIME_IFSET for contiguous datasets, and H5D_FILL_TIME_ALLOC for chunked datasets
· On Python 3, dictionary-style methods like Group.keys() and Group.values() return view-like objects instead of lists
· Object and region references now work correctly in compound types
· Zero-length dimensions for extendible axes are now allowed
· H5py no longer attempts to auto-import ipython on startup
· File format bounds can now be given when opening a high-level File object (keyword “libver”)
Changes which may break existing code:
Supported HDF5/Python versions:
· HDF5 1.6.X is no longer supported on any platform; following the release of 1.6.10 some time ago, this branch is no longer maintained by The HDF Gr...
Find us on Google+
