Description

You are required to complete a semester long project related to usability security and privacy. Projects will be completed in teams of 2-3 students. You may discuss your project with others outside of your team, but the work of the project itself should only be that of the team members. While it is not necessarily required, it is expected that all projects have a user study that can either be qualitative or quantitative.

All papers submitted for the project (proposal, final report) should use the SOUPS formatting template (available for MS Word and LaTeX) and be submitted as PDFs.

---

Project Proposal

Your specific project topic is left to the discretion of the team. A list of suggested topics are at the end of this document. I encourage you to read through them and think of a few that interest you before deciding. You can talk to your instructor to help winnow down a list.

Eventually, you must submit a initial project proposal for approval on one topic. The instructor may approve the proposal as is, or request changes and an updated proposal after discussing with the team.

If you would like to work on a topic not listed below, I recommend you contact the instructor (a short email describing the idea and general approach) prior to submitting a full proposal. This will help you avoid unnecessary work if it is idea is deemed out-of-scope for the course.

---

Proposal Requirements

Your proposal should include the following information (approximately one paragraph per-point):

• What research questions/hypothesis do you plan to answer/test?
  • Why are these important? What is the motiviation behind this study?
• What methods will you use in your investigation?
• What is your target demographic and recruitment plan?
• What is the timeline for your work?
• What are the ethical considerations?
• What is your analysis plan for any data you collect?
• What are the limitations of your approach?

Here is an example proposal for reference. Note, this is meant to be an example to give you an idea of the content included in the propsal. You can ignore the formatting in this example because it was built using a different template.

Following submission of your proposal, your group will schedule meetings with the instructor to get feedback and provide updates. Once the topic is approved, you are expected to make edits on your initial proposals for a final proposal.

Submit your Inital Project Proposal here: https://canvas.tufts.edu/courses/27293/assignments/142775

---

Ethics Review

As part of your project, you will be required to complete an IRB Application that contains the following information:

• IRB application form
• Consent forms
• Survey questions, interview protocol, recruitment messages -- these do not need to be final, but should cover the kinds of things you are asking about. Examples of each can be found here

We expect you to uphold the same standards as an IRB approved research. This includes:

• Treating all participants ethically and fairly
• Protecting the confidentiality of any links between study data and personal information.
• Informing participants that they are participating in a research project for a class.
• If deception is used, providing post-procedure information to participants.

---

Status Updates

You are required to provide one status update for your project. This should include all the same material as the proposal, but with substantive updates on work completed and any changes that occurred since submission.

---

Presentations

You will make two presentations about your project in class. The first is a "Lightening Talk" which will be a short 5 minute talk introducing the topic and your methods. This will enable all class members to learn about the work of everyone else.

The second presentation will be your final project presentation, which will be 12 minutes in length with 5 minutes of questions. These presentations should be inline with conference/workshop style presentations and provide some depth of the methods, analysis, and conclusions.

---

Final Report

Your final report should be between 6-8 pages, not includeing bibliography and appendix. Your paper should follow an outline similar to the following:

• Abstract
  • One paragraph description of your research questions and motivations and a brief summary of the conclusion
• Introduction (containing the following details)
  • Motivation (one/two-paragraph)
  • Research Question (…)
  • Method (…)
  • Results (…)
  • Conclusions/Contributions (…)
• Related Work
  • Enumerate what’s come before but also(!) include how that related work matters to this research
• Methodology
  • What was the method of investigation (survey/interview)
  • Description of the survey/interview
  • Who do you recruit and from where
  • Limitations
• Results
  • How did you analyze it?
  • What did you find?
  • Address each hypthosis/research question, what is their answer?
• Discussion
  • Interpretation and place in the context
  • Now that you know the answer to a RQ, what does that mean?0
  • How could we apply your results?
  • Future work/future directions
• Conclusion
  • Rehashing of the motivations/research-question/methods/results/contributions
• Appendix
  • Entire survey/interview instrument
  • Codebook (qualitative)
  • Any additional figures and material is relevant for me to review

---

Project Schedule

• 3/9: Initial Project Proposal is Due
• 3/10-12: Proposal Feedback meetings (to be scheduled with the instructor)
• 3/16: Approved Project Proposal is Due
• 3/23: IRB Application Due
• 3/30: Lightening Talks and Project Status Updates due
• 3/31-4/2: Status update meetings (to be schedule with the instructor)
• 4/14-16: Status update meetings (to be schedule with the instructor)
• 4/29, 5/4: Project Presentations Due
• 5/14: Final Project Reports Due

---

Grading

• Initial Proposal: 5%
• Final Proposal: 10%
• Ethics Document: 10%
• Lightening Talk: 5%
• Final Presentation: 20%
• Final Report: 50%

Potential Project Topics

Below is a non-exhaustive list of topics and some general research questions that you can use to build a proposal. You may also propose your own topics. Note that you will need to develop your own more specific research question and methods of investigation for your research.

• Developer Studies
  • So much development happens in the open on github---that's public information!---so take a look at all the open issues for security related issues. How quickly are they closed? Do developers care about them?
  • Developers regularly rely on automated analyses to alert them to potential issues in their code. Github's addition of third-party static analysis tools makes it even easier all developers to incorporate these into their workflow. There is also a growing trend of employing dyanamic analysis for code scanning. What kind of output do these tools produce? Do developers understand what these tools are doing? Do they have a broken mental model that could lead to misinterpretation of results?
  • This is a great short read for some inspriation about human factors for programming langauge security.
• Security Professionals
  • Replication: Apply methodology from An Observational Investigation of Reverse Engineers' Processes to live reverse engineering sessions on YouTube.
  • Reverse engineers write analysis plugins (e.g., for IDA Pro, Ghidra, and BinaryNinja) to support basic tasks in their process and make these available for other users. What tasks do these plugins perform? What user interactions do they allow? Are they usable?
  • Hackers commonly report that they participate in Capture-the-flag competitions to develop their ability to find vulnerabilities. What are participant perceptions of these types of exercises? Are they welcoming to everyone? If not, what do beginners struggle with?
  • Whenever a hacker discloses a vulnerability, the produce a report that describes the vulnerability and how it can be exploited. In some cases, these are made public, especially for new vulnerability classes or vulnerabilities with unique characteristics. These reports provide an opportunity for beginners to learn from the experiences of others. What information is typically provided in bug reports? Is it sufficient to be able to reproduce the issue? What is typically missing?
• Authentication
  • Biometric Authentication
    • As more and more devices use biometrics, how does this new convenient method of authentication impact knowledge based methods of authentication choices and the perceptions therein?
    • Do people choose weaker PINs/Passwords if they have a biometric? What are people's opinions comparing biometrics with knowledge based authentication?
  • Mobile Authentication
    • How do users manage passwords on their mobile devices, how does that differ than how they manage them in traditional settings? Does it differ between applications and browsing?
    • Some secure messaging services, like Signal, ask users to select a PIN, do users do this? Do they understand how the PINs are used? Also there was a lot of reminders about entering your PIN on signal, did they work or annoy?
    • Even if the phone is locked, there are some items that are visable. What settings do users use for "locked access" and how do they choose them?
  • Password Managers
    • Why do users or why do they not use a password manager in different settings? Mobile vs. Desktop?
    • What are users preferences and features for password managers?
    • How well do Password Managers actually work at auto filling? Can you empirically measure auto-filling success and failures of password managers in different settings?
    • Many password managers have features for users to review and update passwords. How well do these work? Do users user them? If they do use them, do they understand them?
    • What if we asked users who weren't previously using a password manager to set one up? What do they do? How does it affect them?
• Digital Sharing
  • Users want/need to share secrets online, such as a password or PII. If so tasked, how would they do it? Would they use email or text message or something else? What are the threat models and security understanding of users when they share secrets online?
  • We share a lot of documents in the cloud, how often have you gone back to actually review all of that sharing? If users were to reflect on documents they previously shared, would they change anything by adding more restrictions, or just leave it be? What are the threat models of sharing documents this way?
• Security of Signal/Whatsapp/SMS
  • More and more users are using texting applications that provide end-to-end encryption. Do users user all these features? How do they understand the security provided?
  • Can two users properly establish secure channels using these apps?
  • (see the mobile authentication one for PINs in signal)
• Sharing of Venmo payments
  • What kinds of social payment sharing is acceptable and how do people make these choices?
  • What if you made people go back and look at their Venmo sharing history, what do they think? Is there anything they would want to make private?
• Voice Assistants
  • Creepy or necessary? How do people understand and use voice assistant technology?
  • Voice assistants often record conversations even when users are not aware, but you can go a look at these recordings. How do people feel about these un-aware recordings?
• Two Factor Authentication
  • It's clearly better, but what might stop users from using it? How many accounts do ussrs actually have 2fa installed on?
  • If there is two-factor, do people end up making worse choices elsewhere because they have a false sense of security?
• Video Conferencing Privacy and Security
  • So many more people are using video conferencing, what are their concerns? How do they mitigate them?
• Private Browsing Mode
  • What do people actually use this for and what is their expectations of privacy when using private browsing mode?
• Breaches and Identity Theft
  • We have all probably been a part of a breach, but how do we understand what happens when we are a pat of breach? Do we do anything about it? You can study participants responses after looking up their information in a breach database.
  • You may have signed up for identify theft, perhaps in response to a breach or other notification--- what then? How often do you use? Is it helpful? How usable are these services in the first place?