[messaging] Let's run a usability study (was Useability of public-key fingerprints)

[Daniel Kahn Gillmor]

On 02/13/2014 09:06 AM, Tom Ritter wrote:
> I think that's a common use case, the one I would put that is slightly
> more common is I need to check a fingerprint against nothing. I have a
> signed email, or I have an email I'm about the encrypt, or I have a
> new instant message and I need to figure out if this fingerprint is
> genuine.  I *could* go google for a web page to assert validity that
> way (or query that PGP-in-DNS thing) - but that's asking too much of a
> regular user.  We should remember this use case.

I also run into this use case regularly, and i have habits to work
around some of the standard craziness

> Do we know studies saying that PGP fingerprints are effective?

I don't know of any.  I'm actually concerned that none of this is
relevant without a major UI overhaul that requires user transcription
(or at least copy/paste from some other source) instead of user
comparison.  Transcription requires active participation *in order to
get to the activity that they want to do*, instead of just "click yes to
confirm", or any sort of after-the-fact steps (which will probably never
get taken).

I'd also be interested in a study about whether people even try to check
fingerprints *at all*.

A certain well-known technologist who deals with cryptography (who shall
remain nameless) gave me their business card with what they thought was
their OpenPGP fingerprint on it.  I tried to verify it, and discovered
that the "fingerprint" was not even properly formed -- instead of 40 hex
characters broken into the usual two groups of 5 double-octets, it was
36 hex characters broken into 9 groups of double-octets.  It was missing
one of the double-octet sequences somewhere in the middle entirely (but
the remaining characters matched the correct fingerprint).  I was the
first person (including the keyholder) to have noticed this in over a
year of use of this business card.

> I agree.  Let's run one.  I've participated in them - it's really not
> that hard, especially if we can find a professor in the field who's
> willing to advise/review our proposal.

I like the idea of trying to run such a study.  I'm also interested in
studies that compare specific interaction modes against one another,
though.  A tool that says "you can't send person X an encrypted e-mail
until you have typed or pasted or QR-scanned their fingerprint" (which
is remembered by your mail user agent thereafter for future sessions) is
radically different than one that says "is this fingerprint correct for
this person?"

Is it possible that a good, usable tool could avoid ever showing
fingerprints (or parts of fingerprints) of unverified keys, to ensure
that the user has to actively confirm them from some external source?
Testing how well a user can compare things might be irrelevant if UI can
be structured that the user is never presented with a comparison task,
just a transcription task.

I don't know how to structure such a comparative usability study, though.

	--dkg

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1010 bytes
Desc: OpenPGP digital signature
URL: <http://moderncrypto.org/mail-archive/messaging/attachments/20140213/37143398/attachment.sig>