[noise] out of curve points

[Trevor Perrin]

On Thu, Sep 24, 2015 at 9:43 PM, Daniel Kahn Gillmor
<dkg at fifthhorseman.net> wrote:
> On Sat 2015-09-19 18:20:57 -0400, Trevor Perrin wrote:
>> No, IMO - if one party in a session is misbehaving, neither party to
>> that session can expect any security for the session.
>
> Sure, but this is a trivially-detectable misbehavior.  It's one thing to
> continue operations when you can't detect that the peer is misbehaving,
> but it seems like bad form to continue with the connection when you can
> know for sure that the peer is misbehaving, no?

Validating that the point is in the correct subgroup requires a scalar
multiply by the subgroup order.  It's an easy check, but it doubles
the DH computation time.

So while fully checking inputs is the conservative strategy, I
disagree that it's "trivial".

It makes more sense to mandate checks only if they're needed.
Twist-secure curves with single-coordinate ladders, like 25519, were
designed not to need extra checks for DH.

So the current Noise test allows extra checks, but doesn't mandate them:

"""
Performs a DH calculation and returns an output sequence of bytes. If
the public key is invalid the output of this calculation is up to the
implementation but must not leak information about the private key.
Implementations are also allowed to abort on receiving or processing
an invalid public key.
"""

I should add a security consideration about "Implementation
fingerprinting" for anonymity contexts, but I don't see a reason to
mandate other checks.

Trevor