<div dir="ltr"><div><br></div><div>Very cool!</div><div><br></div><div>Do you already have a good set of evaluators and judges, and you're mainly looking for advice on metrics? Or are you also looking to get more people / orgs involved in the evaluation process?</div>
<div><br></div><div>I guess I'd encourage a larger / diverse judging panel so that you're tapping different perspectives, and also to increase the contest's prestige and objectivity.</div><div><br></div><div>I also think you might consider multiple prizes, since there are different categories of apps, and having more winners means the benefits of this process get spread more widely.</div>
<div><br></div><div><br></div><div>Trevor</div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Mar 11, 2014 at 2:30 PM, Joseph Bonneau <span dir="ltr"><<a href="mailto:jbonneau@gmail.com" target="_blank">jbonneau@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div>Of possible interest to many on this list, the EFF is hoping to offer a prize for the most usable end-to-end encrypted communication tool this year. This is still in the early stage of planning but we have set aside a one-day workshop on July 9 in Menlo Park, CA to discuss and debate the process for awarding the prize. The workshop is attached to SOUPS, the biggest event for academic usability researchers interested in security, so for security-minded folks this is a great way to get in touch with UI experts.</div>
<div><br></div><div>As a co-organizer I'd be interested to hear ideas on the best structure to adopt for the workshop. I'd also encourage anybody interested in participating more to submit an abstract on a talk they'd like to give at the workshop (due date is May 15 so lots of time to think about ideas until then).</div>
<div><br></div><div>Full details below.</div><div><br></div><div>Cheers,</div><div><br></div><div>Joe</div><div><br></div><div><div><br></div><div>************************************************************</div><div>2014 EFF Crypto Usability Prize (EFF CUP) Workshop</div>
<div>************************************************************</div><div><br></div><div>CALL FOR PAPERS</div><div><br></div><div>Submission Deadline: May 15, 2014, 5pm PDT</div><div>Notification Deadline: May 30, 2014 5pm PDT</div>
<div>Anonymization: Papers are NOT to be anonymized</div><div>Length: 500 words</div><div>Formatting: PDF</div><div>Submission site: email to <a href="mailto:effcup@eff.org" target="_blank">effcup@eff.org</a></div><div>Workshop Date: Wednesday, July 9, 2014</div>
<div><br></div><div>SCOPE AND FOCUS</div><div><br></div><div>The Electronic Frontier Foundation is evaluating the feasibility of offering a prize for the first secure, private end-to-end encrypted communication tool. There is currently tremendous interest in this area, with several dozen new projects trying to make encrypted email, instant messaging, text messaging, VOIP and video chat a reality. It is not yet clear which of these tools is best-suited to meet real-world usability challenges.</div>
<div><br></div><div>We believe a prize based on objective usability metrics might be an effective way to determine which project or projects are best delivering communication security to vulnerable user communities; to promote and energize those tools; and to encourage interaction between developers, interaction designers and academics interested in this space.</div>
<div><br></div><div>The EFF CUP workshop aims both to establish suitable metrics and criteria for the prize, and to introduce developers working on open source encryption tools (likely contestants) to the privacy and security research community. EFF CUP will be held in conjunction with the Symposium on Usable Privacy and Security (SOUPS) in July 2014 in Menlo Park, CA. We are seeking talk abstracts and position papers on the following topics:</div>
<div><br></div><div>USABILITY AND SECURITY METRICS: Holding an open competition for secure communication tools is a new undertaking and requires new thinking about measuring security and usability tools. We are seeking position papers on what metrics can be used to most objectively evaluate quality, including:</div>
<div><br></div><div>*Security metrics: Identifying the types of attacks that at-risk groups (journalists, activists, lawyers) are subject to, and how we can reliably measure the resistance which cryptographic communications tools provide.</div>
<div>*Indirect usability metrics: Metrics which can be evaluated analytically, such as backwards compatibility with existing tools, integration into existing tools, or demonstrated adoption by N million users.</div><div>
*Direct usability metrics: Metrics which can be evaluated through user studies, such as the percentage of users who can quickly start using a tool and survive various classes of real-world attack.</div>
<div><br></div><div>CURRENT TOOL SUMMARIES: Developers of secure end-to-end communication tools are invited to submit a short (100-500 word) abstract describing their project. We aim to have a series of short presentations (followed by discussion) on the state of various projects, including a description of the project's security and usability goals, current development status, installed user base and supported platforms, known usability challenges and vulnerabilities, and experiences (if any) with user testing.</div>
<div><br></div><div>EXPERIENCE FROM PAST CONTESTS: Organizers or competitors from other technology contests, particularly but not exclusively in the areas of security and/or usability, are invited to submit a short (500 word) abstract describing lessons from those contests. We aim to have a series of short presentations including a brief overview of past contest's goals, setup and rules, and outcomes. Example competitions may include cryptographic primitive competitions (eg. AES, ESTREAM, SHA3, PHC), Darpa contests, Capture the Flag contests, Crack Me If You Can, VoComp or the Netflix Prize.</div>
<div><br></div><div>ORGANIZERS</div><div><br></div><div>Lorrie Faith Cranor, </div><div>Associate Professor of Computer Science and of Engineering and Public Policy at Carnegie Mellon University, Director of the CyLab Usable Privacy and Security Laboratory (CUPS). Member, Electronic Frontier Foundation Board of Directors.</div>
<div><br></div><div>Peter Eckersley, </div><div>Technology Projects Director, Electronic Frontier Foundation.</div><span class=""><font color="#888888"><div><br></div><div>Joseph Bonneau, </div><div>Postdoctoral Fellow, Center for Information Technology Policy, Princeton University</div>
</font></span></div><div><br></div></div>
<br>_______________________________________________<br>
Messaging mailing list<br>
<a href="mailto:Messaging@moderncrypto.org">Messaging@moderncrypto.org</a><br>
<a href="https://moderncrypto.org/mailman/listinfo/messaging" target="_blank">https://moderncrypto.org/mailman/listinfo/messaging</a><br>
<br></blockquote></div><br></div></div>