Phase 1: Getting ready for user testing 🪞⌗

Planning is a big part of user testing because it sets you up for success. If possible, kick off the process with a team meeting to decide and document the: what, why, how, who, how many, how long, and where of user testing.

What 🔎⌗

Deciding what to test depends on your team’s agreed upon roadmap. If a user testing phase is already built-in to your development process, you can skip this step. But if you’d like to do a user test and you don’t know what to test (kudos to you!), there are a number of options.

Ideas for what to test with users:

New features
Known pain-points or bugs
Something you’re curious about

Feeling anxious?

Typically features or parts of a tool that are in development are a great choice because they’re already the focus. However, your team may be apprehensive because user testing might uncover critical errors or misunderstandings. If testing reveals that something can be done better, that doesn’t mean you have to stall development. It means that you now have more information for future improvements.

A short 3 panel comic about how development and design of a feature can be done at the same time and user insight can be used to make improvements rather than hold up development work

Don’t try to test every part of your software - that’s overwhelming for you and the participant. Instead, choose a small task that you’re curious about or that isn’t working so well.

Examples:

Can the user add a new entry?
Can the user sign up?
Can the user send a message?
Can the user upload their first document?
Can the user fill out their profile?
Does the user understand the message?

Why 🏷️⌗

What is your goal? What do you hope to learn? This helps the team remember why you are testing. Examples:

We want to know if the messaging is clear before we commit.
We get a lot of questions about the signup process so we want to know what to change.

How 🤖⌗

Depending on what you decided to test, you may share a prototype with your tester, a beta version, or your current version. Keep in mind that it can be challenging for users to install software that they don’t already have. Is it possible to share a staging version of the tool? Or a live web version if what you want to test is functional? You can also create a simple prototype. To mimic real software, you can simply put a sketch on Google Slides or a PDF. You and your tester just need a common frame of reference.

Who (Testers) 👋⌗

Who are you going to recruit to be a tester? It’s best to find people who are similar to your target users, but if that’s not possible, you can still test for usability with almost anyone (except people on your team - they know too much!). Inviting the same testers who participated in a previous session won’t produce the best results - you want people with a fresh perspective. Try to get different types of testers so that you’re learning from and about a wide range of users and being inclusive.

Examples:

People who are power users. These are users that know every aspect of your tool and have been using it or something similar for years.
People who are new users of your tool or have never used the feature or part of the tool you’re testing.
People who are high risk users. Regardless of the nature of your tool, it’s always good to test with people who are at risk if security and privacy is connected to their safety. (Resources for conducting high-risk user research: High-Risk User Research, USABLE Personas)
People who consider themselves ‘techy’ and people who consider themselves ‘not techy’.
People with malicious intentions (or people that can pretend to have malicious intentions).
People who are distracted as they use the tool. Think about people with children, jobs where they are ‘on call’, etc.

A note on compensation: In an ideal world, you’d compensate people for their time, expertise, and feedback. However, we understand that this is a lean user testing process. Can you offer them something other than money, such as swag, account upgrades, etc? If you can’t offer anything, try to keep the tests on the shorter side and of course, offer your thanks.

Who (Facilitators) 🧑‍✈️⌗

Depending on your personality, setting up and conducting tests can be tiring. One to two people can lead the task, but others should jump in to help with notes or observation. Decide how to split up the task in the beginning, keeping in mind everyone’s availability. Consider who in your team feels the most comfortable talking to new people and ‘building rapport’ with users as sometimes the interactions can feel awkward until you get into the testing.

Example:

Testing organizer (emails with testers, sends calendar events, etc): Malek
Test 1 interviewer: Joon
Test 1 note-taker: Trina
Test 2 interviewer: Trina
Test 2 note-taker: Malek

How Many 🧺⌗

How many tests will you do? We recommend two to four tests if you’re just getting started. One test is also good (and better than none). Most UX professionals agree that five tests are sufficient in order to gain information that trends a direction, although this can be challenging for small teams without a dedicated designer.

How Long 🧵⌗

Deciding the duration of each test helps the team and your participants manage expectations. We like to do 20-minute, 30-minute, or 45-minute tests, depending on the size of the task. Remember to give yourself some buffer time and end the test promptly at the time limit to respect the participant’s time. Don’t feel the need to schedule test sessions back to back - give your team a break between sessions.

Example:

Test duration: 30 minutes

Where 🏢⌗

Testing can be done in-person and remotely. We recommend that everyone’s face is visible (unless anonymity is crucial) and you all can see what’s being tested. If you give a phone or a laptop to the user, sit next to them (if culturally appropriate) so that you can watch the process. If the testing is remote, you can share your screen (the participant verbally gives commands while you perform the operations) or ask them to share their screen (making sure to get their consent in advance).