[messaging] NaCl comm relay: file support

[Max Skibinsky]

Hi All,

A while back, we open sourced our relay [1] designed for anonymous,
“dead-drop” messaging built on top of NaCl. For the past year, we have been
using it intensively in production on our main project, finding and fixing
all sorts of bugs in the process. As our project evolved, we had to add
support for a similar async “dead-drop” style large file exchange (when
device needs to send a 1gb folder tree to another, there are a lot of
assumptions to be adjusted from simple app data messages). So we designed a
file exchange protocol around NaCl primitives and extended the Zax relay
with new commands for these file operations.

We just published a big combined update [2] of Zax with all improvements
from the past year and the new file API. Our web platform is upgraded to
the latest Ruby/Rails with significant performance improvements (full web
client test suite running local 200+ tests went down from 120 seconds down
to only 30).

We would love to have more folks take a look at the new protocol and
codebase - any feedback is very weclome!

MIT licensed source is here:
https://github.com/vault12/zax
https://github.com/vault12/glow

Short summary of the file exchange protocol:

- Alice and Bob use the regular Zax messaging channel based on NaCl PKI
keys. To start a file upload, Alice sends an ecrypted message to Bob's
public key with file meta-data and a Nacl `secretbox` symmetric key. Relay
issues unique upload token in response and mandates largest chunk size it
will accept for single POST upload.

- Alice and Bob's interactions are assumed to be asynchronous (it might be
days before Bob’s device gets back online). Alice has no dependency on
Bob's response, and starts uploading chunks of her file to a relay using
the upload token issued by the relay in reponse to the first command.

- Each chunk is encrypted with a symmetric key (Nacl `secretbox`) that
Alice sent Bob previously. Each chunk nonce is encrypted in a message
attached to that chunk encrypted from Alice to Bob's public key.

- Relay data visibility consists of a symmetrically encrypted blob of the
file chunk, and the PKI encrypted blob with chunk nonce.

- When/if Bob gets online, he downloads the initial message with the
symmetric key and all file chunks with associated nonces from the relay.

- The relay uses memory-based redis for storage of messages and metadata.
Encrypted file chunks are stored on hard drive. When the file metadata
expires in redis (default set to one week), the relay cleanup job will
remove expired files.

Full details of the new file protocol are in [2] and [3].

[1] https://moderncrypto.org/mail-archive/messaging/2015/002047.html
[2] https://github.com/vault12/zax#-version-20-updates
[3] https://github.com/vault12/zax/wiki/Zax-2.0-File-Commands

- Max [blog <http://skibinsky.com>]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://moderncrypto.org/mail-archive/messaging/attachments/20170912/3f101667/attachment.html>