Back to main page
How to do remote user testing during COVID-19
Designing great products was hard enough before the Covid-19 crisis hit, with a market oversaturated by competition for limited resources. In a post-pandemic world preparing to face the virus’ second wave, success will be a matter of survival of the fittest.
In a situation like this, companies need a competitive advantage. At Mooncascade, we recommend UX as a weapon of choice. Why? Because great UX won’t just guarantee your survival, it’ll also give you a serious shot at market dominance.
Data tells us the same thing. Research shows that, on average, every $1 invested in UX brings $100 in returns. That’s an ROI of 9,900%. On the other hand, 91% of users who don’t complain just leave, and 13% of them tell 15 other people about their bad experience.
The secret to great UX lies in an iterative design process and user testing. In this article, I’ll give you advice on what to keep in mind and which tools to use when doing remote user testing for your business. Let’s dig in.
What do you want to find out?
Designing a user test is basically like conducting research on a smaller scale. What’s the first thing you do when you’re researching something? You define a research problem, research question(s), and a methodology.
This is something that’s often skipped over – researchers jump to the methodology part straight away, because conducting tests can quickly become routine. Still, I’d argue that it’s necessary to follow these steps every time you do user testing. It’ll help focus your project and improve the chances of you actually getting the answers you want.
Let’s take an example. Say you’re designing an app that will allow users to send money abroad. You’re familiar with the field, maybe you even have a family member living abroad yourself, and you’re able to define your research problem like this: “When it comes to money, people carefully choose who they’ll let handle it, and they don’t easily trust apps with their finances.” You’ve talked to your users, gotten some ideas for features, and built a clickable prototype. Before going into development, you want to know two things:
- RQ1: will your users trust the app enough to use it?
- RQ2: is the app easy and enjoyable to use?
When asking the first question, you’re interested in the why, the what, and the how:
- Why and what the users are afraid of = how you can address and mitigate your users’ fears in your app
- What kind of experiences your users have had with similar services and products = what you can do better, which mistakes to avoid
- How you can help users solve their problems = which features you need to have and how they should be prioritized
This list of questions can go on, but what’s important to note is that this line of questioning will give you deep insight and a thorough understanding of your users.
The second research question to ask is about usability: is your app understandable and easy to use? Here’s where you should focus on users’ interactions with the app. The most common method to test usability in our example case is task-based testing, which involves writing out tasks for the users, letting them accomplish these, and asking them to comment on everything they saw, thought, and felt while doing so. This will give you an overview of things that need to be improved in your product’s design.
Once you’ve established a research problem, research questions, and methods (in our example, the methods you’re using are interviews for RQ1 and task-based testing for RQ2), it’s time to choose your tools. This is especially important as you’ll be conducting your tests remotely.
Tools
Before we get into which tools to use, let’s have a look at user testing from another angle. There are basically three types of testing possible: moderated user testing, unmoderated user testing, and quantitative testing. All of these can easily be done remotely.
Moderated user testing
Moderated user testing most resembles in-person testing. If you’d normally invite the person you want to interview to your office or lab, in the case of remote testing, you’d make all of this happen using technology.
Since you’re using the interview as a method to answer RQ1, you’ll be most interested in the tools that allow you to speak with users remotely. If this process involves testing, you’ll want to see what users are doing on their screens, so remember that screen sharing and session recording will be important functions to have access to.
- Zoom, Google Meet, Skype – these all allow for screen sharing and session recording, and are all free and easy to use.
- Professional user testing and research products, like Lookback – this allows for screen sharing and session recording and allows you to invite other team members to watch the tests in real-time (or watch the recordings); it also allows for inserting timestamped comments and data exporting. Very easy to use, and offers a free trial. Here are some similar tools.
- Products that combine testers’ databases with testing and research capabilities, like Sharewell. This means that, in addition to doing research using their environment, you’ll also be able to recruit users from their database.
Unmoderated user testing
Say you’ve carried out your interviews and fine-tuned a prototype, and now you want to let as many users as possible play around with your product and comment on what they see, think, and feel. For that, you’d use unmoderated user testing, where tests are performed by the participant on their own time, without a moderator.
There are plenty of websites (UserTesting, UsabilityHub, and Userfeel, to name a few) that allow you to do just that. You can upload anything from sketches to clickable prototypes and ask users to express their thoughts and feelings about their experience, accomplish tasks, or do anything in between.
One important thing to keep in mind with unmoderated testing is that the websites with the biggest user pools are usually U.S.-based. So, if your target audience is from another country, finding a website that has enough relevant users in their database might be tricky. Then again, most of the options out there let you invite users yourself, so if you have your own database to work with, you won’t have any problems.
Quantitative user testing
While most user testing is used to answer “how” and “why” questions, you can also use testing to explore users’ behavior quantitatively. For example, if you upload your prototype to one of the above-mentioned platforms, users will carry out the tasks you’ve asked them to accomplish and comment on the process. But the platform will also be monitoring their mouse usage, which can provide you with plenty of additional information. You could use this quantitative data to generate a heatmap, for example, which would give you valuable insight into your users’ actual behavior and unconscious attitudes when interacting with your product.
Quantitative user tests are unmoderated and usually focus on metrics like user satisfaction and perceived difficulty (SUS), time spent on a task, and task success rate. Some quantitative testing platforms are even specialized in domains like information architecture, which means they’ll include heatmaps, scrollmaps, and site maps (tree testing) as well.
Recruiting users
Once you’ve selected your tools, you’ll need to find users to start conducting tests on. If you opted for a platform that has a user base, you can start picking suitable users from there. As their profiles might not include everything you need to know to ensure they fit your testing criteria, it’s wise to create a small survey to screen them more thoroughly before choosing anyone.
If you’re not working with a user pool and need to find users yourself, there are several ways to do this (see the graph above). The cheapest, easiest, and quickest way is always to handpick suitable people from your personal and extended network. Ask your coworkers and friends whether they know people who match your criteria and reach out to them.
Giving them something for their time also helps. Although Amazon gift card is the easiest, I recommend supporting local, sustainable, and small businesses and giving out gift cards to vegan restaurants, ethical coffee shops, concerts, and other similar initiatives.
Pick 5 people that best fit your criteria and in principle, you’re ready to go.
Watch out for these common mistakes
Before you jump into things, let’s look at a few common mistakes to avoid.
The first mistake is starting user research without determining what you want to find out beforehand. It’s as important to stay focused on this when choosing methods as it is when putting one into action: listing your tasks, preparing interview questions, and recruiting users all have to be based on your research problem and questions. Otherwise, you’ll get results that are vague at best and false at worst, resulting in potentially disastrous decision-making.
As qualitative methods are all based on communication with your users, poor active listening skills are another factor that can ruin the reliability of your tests. Making assumptions and asking leading questions will influence the user into giving you information that you want to hear. Not asking followup questions or reflecting the users’ terms back to them will leave gaps in the information you’ve gathered. Both options are a no-go.
Finally, remember that testing isn’t the only thing that needs to be accomplished. You have to focus on data analysis, too. This starts with data collection and organization. Not only should you always take notes, but you should categorize and code these notes systematically and digitally. This will make data analysis much easier and create transparency and iterability in whatever you’ve learned.
If you haven’t yet, start with remote user testing now
Of course this article won’t be enough to walk you through testing if you have no previous experience in the field. But I do think it can lay the groundwork for a successful set of user testing processes applicable to a variety of use cases. Remember, talking to users – even when you’re not a specialist – is a million times better than letting yourself be deceived by talking to no one at all.
That said, if you do need help with user testing or any other kind of user research, we have plenty of experience and would love to help. Don’t hesitate to get in touch!
