[messaging] Let's run a usability study (was Useability of public-key fingerprints)

[Trevor Perrin]

On Thu, Feb 13, 2014 at 3:27 PM, Daniel Kahn Gillmor
<dkg at fifthhorseman.net> wrote:
> On 02/13/2014 05:58 PM, Trevor Perrin wrote:
>> (A)  Users aren't able to communicate unless they enter each other's
>> public-key fingerprint.  That wouldn't work for a general
>> communication tool, as the high entropy of fingerprints makes them
>> awkward to handle, and the extra security of a manual fingerprint
>> check isn't needed for many conversations.  A tool that required this
>> would be rejected by most users.
>
> to be clear, i'm not proposing that each and every conversation would be
> preceded by tedious fingerprint verification or entry.  This would be
> done once per contact, after which it would get out of the way.  Do you
> think that cost of entry is still too high?

Yeah, I do.  That's still much harder than keying in someone's digits
or entering an email address for an initial contact.

Secure apps need to be as useable as alternatives to have any hope of
wide adoption.

The case of a changed fingerprint is different from an initial
contact.  So if you want to force the user to jump through hoops and
confirm a changed fingerprint, you could argue for doing it there.

But I'd still advocate something gentler, as users will change keys
occasionally.


>> (B)  If the user chooses to check a fingerprint, the tool presents an
>> "entry" UI instead of a "display" UI.  I think I can compare strings
>> faster than I can transcribe them (particularly on a phone or tablet),
>> so a tool that forced me to enter it would be annoying, IMO.
>
> I'm quite sure that comparison is faster than transcription,
> particularly if the comparison is based on well-aligned,
> identically-formatted strings (which is not actually how most
> fingerprint comparisons are done today, in my experience).  But i'm not
> convinced that comparison is more accurate than transcription.  If
> accuracy/correctness is the goal, what sort of tradeoffs are we willing
> to accept?

Considering 125 bit fingerprints:
 - accuracy of 60% = 75 bits security, good against everyone except
well-funded attackers
 - accuracy of 80% = 100 bits security, good against everyone for now

So a bit of inaccuracy isn't the end of the world.


> While we're talking about usability studies, this question (about
> transcription versus comparison, measured by accuracy and speed) seems
> like it's narrowly-scoped enough that it could make for a good
> experiment based on a single fingerprinting scheme.  Comparison across
> fingerprinting schemes would be a natural extension once the other
> details were worked out for a single fingerprint.

Sure, but who has the experiences and resources to do that?  I agree
with Joe, this requires access to a lot of users, careful experimental
design, etc.

I'm hoping someone on this list knows UX researchers, so if we point
out interesting questions maybe we can tempt them, but it feels like
we're running into the limits of what a mailing list can do.


Trevor