[noise] Invalid point attacks

[Jason A. Donenfeld]

Hi Trevor,

I suggest that the Noise spec explicitly mandate erroring out early in
the case that DH() returns the null string.

Noise already had an abort path when an AEAD calculation fails, which
means adding in more abort points doesn't introduce a fundamentally
new oracle. Rather, explicitly aborting on null DH() output strictly
adds desirable security properties.

1. It prevents accidents in which a peer might shoot himself in the
foot with a bad RNG for ephemerals and/or bad static key.
2. In NoiseIK, there is an invalid point attack from the initiator to
the responder, wherein the initiator sends a bad ephemeral for message
1. The responder then encrypts his payload in message 2 with {DH(Sr,
Si), DH(Sr, Ei) = NULL, DH(Er, Ei) = NULL, DH(Er, Si)} = {DH(Sr, Si),
DH(Er, Si)}, which may not be desirable.
3. In NoiseIK, there is an invalid point attack from the initiator to
the responder, wherein the initiator sends a bad ephemeral and also
announces his static key out of band to be an invalid point, for
message 1. The responder then encrypts his payload in message 2 with
{DH(Sr, Si) = NULL, DH(Sr, Ei) = NULL, DH(Er, Ei) = NULL, DH(Er, Si) =
NULL} = NULL. While you might claim the attacker has brought this upon
himself and so we shouldn't be sympathetic, one could imagine complex
interactions or sophisticated adversaries who are in a position to
weaken utilized keys like this, with the goal of having weaker crypto
later on that is passively recorded.
4. Allowing for DH to "maybe return null" significantly complicates
the security analysis and formal verification.

If you aren't compelled by all of these reasons, perhaps you will be
compelled by at least one. Can we just mandate erroring out early
after computing null DH output?

Thanks,
Jason