[noise] DoS *is* a problem

[Jason A. Donenfeld]

Hi Trevor, Michael,

Thanks for pointing me at the DTLS cookie and spelling it out for me here
too. Now we're onto something!

So, I'm using Noise_IK (with an encrypted timestamp/monotonic-counter in
the first message to prevent replay), which is just 1 round trip to
establish a session. Peers sometimes switch roles of being initiator and
responder; there's no concept of a "server" and "client" - just an
initiator and responder, and sometimes they reverse. That's the setup.

As I understand it, the general idea here is that if the responder is under
load, it doesn't take the handshake. Instead it replies with a cookie
value, which is a HMAC of the src IP and src port, using a key that changes
once a minute. Then, the initiator re-transmits the handshake containing
this cookie. This time, the cookie checks out, and the server processes the
handshake. This does NOT prevent that handshake from being transmitted a
millionbillion times within the one second that the HMAC key is valid, so
the DoS potential is still there. But, it does bind that cookie to an
actual IP address, which means the usual token bucket ratelimiting
mechanisms can then be applied to cookied handshake initiations. (This
means that the DTLS specification claim that it's a "stateless" mechanism
isn't completely true; it just moves the state into a different layer - the
memory demanding [annoyance 1] but venerable hashlimit algorithm in network
stacks.)

This mechanism is cute, but it has two flaws. First [attack 1], it does't
protect against a man in the middle learning the cookie value. It could be
argued that neither does the TCP SYN dance, but that's not a very high bar
to set. Second [attack 2], the cookie response packet from the responder to
the initiator isn't authenticated. This means the initiator could be DoS'd
by a malicious party flooding the initiator with bogus cookie-response
packets, causing the initiator to then send bogus cookies to the responder.

I have a mitigation, but not perfect solutions, to these two attacks. In
the first place, we require any initial messages sent by the initiator to
the responder to be HMAC'd using the responder's public key as a key, and
optionally the PSK mixed in too, as discussed prior in this thread as an
inadequate but still useful mechanism; this prevents against automated
scanners learning about endpoints under load by being able to provoke
cookie responses. The second, and more significant, mitigation is that when
the responder sends the cookie back to the initiator, it
authenticated-encrypts it, taking as a key a combination of the responder's
public key, optionally the PSK too, and the initial HMAC that was sent in
the initiator's first handshake initiation.

This isn't perfect, as it uses a public key as a secret, but it does at
least help mitigate [attack 1] and [attack 2]. It doesn't account for
[annoyance 1], however.

If anybody has a better solution, or feedback on this, please do let me
know!

Thanks,
Jason
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://moderncrypto.org/mail-archive/noise/attachments/20151121/b7aa9d8f/attachment.html>