[messaging] Let's run a usability study (was Useability of public-key fingerprints)

[Trevor Perrin]

On Mon, Mar 10, 2014 at 1:14 AM, Tom Ritter <tom at ritter.vg> wrote:

> As promised, here's a first-pass at a proposal:
> https://github.com/tomrittervg/crypto-usability-study



Nice, comments -

Fingerprint Types
 - Visual and poetry fingerprints seem worth including.

Comparison Method
 - Business cards can only fit a small amount of text (since most of the
space is taken up with other stuff), and don't typically contain
high-resolution images.  So I'm not sure that comparing things between
screens can be reduced to comparing things on business cards.

Approaches
 - I suggest giving users X seconds to perform a comparison between a pair
of values that are either identical or close, then seeing whether they
correctly distinguish these cases.  X can be calibrated by performing some
preliminary tests, then choosing a number that's likely to produce a
variable error rate (i.e. not so low that subjects are always guessing
randomly, not so high that they're always getting it right).  This is
modeled after "character legibility" studies, e.g.

 http://psychology.wichita.edu/surl/usabilitynews/81/legibility.asp

 - I don't see a reason to fake-out the user by having her perform
extraneous tasks.  Just seems like it would slow things down.

Modulating Speed
 - For the "Spoken Aloud" test, why not just have pairs of subjects compare
the fingerprints by speaking to each other?

Error Rates
 - I'm not sure about the '"One Subtle Flaw" case, because the fingerprints
have different notions of "tokens" so this will be hard to compare between
formats.  Also, it doesn't model a realistic attacker.

 - For the computationally-chosen flaw, I think you should just assume an
attacker that can consider 2^80 random candidate fingerprints, and choose
the closest-matched fingerprint this attacker could find (but of course
don't actually do 2^80 hashes, just set 80 bits of the fingerprint equal).


Trevor
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://moderncrypto.org/mail-archive/messaging/attachments/20140311/fd4ef93c/attachment.html>