[messaging] Zero-metadata object store

[Peter Parkanyi]

Hi,

My project is not strictly messaging-related, but I think this is still the most fitting forum.

For the past few months I have been working on the design and implementation for an object store that hides as much metadata as practical, while keeping it fast.

Zerostash stores all data in 4MiB blobs that look random from the outside. I use Argon2 and libsodium’s key derivation APIs to make all of this addressable through a user password, and lz4 inside ChaCha-Poly for encrypting the blobs.

The code is very PoC quality, but I’ve focused on giving a performant implementation over cleanliness for now. On a MacBook Pro, I can achieve write speeds of ~300MiB/s, and unpack at around 400MiB/s for a large number of small files. At the moment single file is bound to a single thread, so you will not be blown away by performance on large files.

I’ve uploaded the specification and code to https://github.com/rsdy/zerostash, and would be grateful for some feedback.

Cheers,
Peter