ZarrChunked, compressed, N-dimensional arrays

Description ​

Zarr is motivated by the need for a simple, transparent, open, and community-driven format that supports high-throughput distributed I/O on different storage systems. Zarr data can be stored in any storage system that can be represented as a key-value store, including most commonly POSIX file systems and cloud object storage but also zip files as well as relational and document databases.

See the following GitHub repositories for more information:

• Zarr Python
• Zarr Specs
• Numcodecs
• Z5
• N5
• Zarr.jl
• ndarray.scala

Applications ​

• Simple and fast serialization of NumPy-like arrays, accessible from languages including Python, C, C++, Rust, Javascript, Java and Julia
• Multi-scale n-dimensional image storage, e.g. in light and electron microscopy
• Geospatial rasters, e.g. following the NetCDF / CF metadata conventions

Features ​

• Chunk multi-dimensional arrays along any dimension.
• Store arrays in memory, on disk, inside a Zip file, on S3, etc.
• Read and write arrays concurrently from multiple threads or processes.
• Organize arrays into hierarchies via annotatable groups.

Sponsorship ​

Zarr is a Sponsored Project of NumFOCUS, a US 501(c)(3) public charity.

NumFOCUS Sponsored Projects rely on the generous support of corporate sponsors, institutional partners, and individual donors.

Videos ​

Videos of community members talking about Zarr. If you have a video you'd like us to share, let us know!

Acknowledgements ​

Hero animation adapted from dynamical-zarr-ecmwf in developmentseed/deck.gl-raster