[messaging] Let's run a usability study (was Useability of public-key fingerprints)

[Trevor Perrin]

On Sun, Mar 9, 2014 at 8:10 PM, Joseph Bonneau <jbonneau at gmail.com> wrote:

> I'm interested in helping out as well if I can, glad to see there's some
> movement here! In particular I can help with data analysis/significance
> testing if that's something needed, though sounds like the sample size will
> be low.
>
>
>> For the tests, I could imagine giving users pairs of fingerprints which
>> are either identical or a close match, and have them choose same/different
>> after X seconds, where X is tuned to produce a significant error rate.  I'd
>> also try having one value on a screen, and the other in different formats
>> that might be used for fingerprint exchange:  e.g. printed on the front of
>> a business card, displayed on a separate screen, read aloud, written on a
>> napkin, etc.
>>
>
> I think I've made this point before but I think the main challenge is
> seeing how users perform not just in a quick check time wise, but one in
> which they have no reason to suspect an error, because most of the time
> most users don't think they're being attacked so they just check the
> beginning for a gross error then click through. If you tell users to check
> for errors, it may not represent very well how they'd do in practice.
> Perhaps the only way around this is to show users fingerprints which match
> in 99% of cases and see if they catch the 1% when they are mind-numbingly
> bored and their prior is low.
>

There might be two different questions here:

1) How much effort do users spend comparing fingerprints?
2) For a given amount of effort, which fingerprint format works the best?

Question (2) seems testable in the lab.

Question (1) depends on the user and her context:  Does she believe herself
to be a surveillance target?  Are these sensitive communications?  This
seems a question for fieldwork.


Trevor
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://moderncrypto.org/mail-archive/messaging/attachments/20140310/36aeeab8/attachment.html>