28,14 €
Mehr erfahren.
- Herausgeber: Packt Publishing
- Sprache: Englisch
Gain actionable insights from qualitative remote studies to improve user experience
Key Features
- Understand the different usability testing methodologies and their strengths and weaknesses
- Master the execution of remote studies
- Learn how to analyze and present study results
Book Description
Usability testing is a subdiscipline of User Experience. Its goal is to ensure that a given product is easy to use and the user's experience with the product is intuitive and satisfying. Usability studies are conducted with study participants who are representative of the target users to gather feedback on a user interface. The feedback is then used to refine and improve the user interface.
Remote studies involve fewer logistics, allow participation regardless of location and are quicker and cheaper to execute compared to in person studies, while delivering valuable insights. The users are not inhibited by being in a new environment under observation; they can act naturally in their familiar environment. Remote unmoderated studies additionally have the advantage of being independent of time zones.
This book will teach you how to conduct qualitative remote usability studies, in particular remote moderated and unmoderated studies. Each chapter provides actionable tips on how to use each methodology and how to compensate for the specific nature of each methodology. The book also provides material to help with planning and executing each study type.
What you will learn
- Choose the most suitable remote study methodology
- Establish a clear goal for the study
- Plan the study execution
- Understand recruitment logistics, expectations and compensation
- Set up and moderate remote studies
- Write good tasks and questions for each methodology
- Analyze and document the study results
- Deliver results that align with the goal for the study
Who this book is for
This book is for user experience (UX) professionals familiar with traditional in-person usability testing methodologies, or for UX designers with no prior exposure to user research and usability testing. Customer experience professionals or product managers who want to understand remote usability testing will also find this book useful. No knowledge of remote usability testing is needed.
Das E-Book können Sie in Legimi-Apps oder einer beliebigen App lesen, die das folgende Format unterstützen:
Seitenzahl: 259
Veröffentlichungsjahr: 2018
Ähnliche
Remote Usability Testing
Copyright © 2018 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author(s), nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
Commissioning Editor:Amarabha BanerjeeAcquisition Editor: Siddharth MandalContent Development Editor: Mohammed Yusuf ImaratwaleTechnical Editor: Shweta JadhavCopy Editor: Safis EditingProject Coordinator: Hardik BhindeProofreader: Safis EditingIndexer: Pratik ShirodkarGraphics: Jason Monteiro, Nic GruchotProduction Coordinator: Shraddha Falebhai
First published: August 2018
Production reference: 1170818
Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK.
ISBN 978-1-78899-904-5
www.packtpub.com
Mapt is an online digital library that gives you full access to over 5,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.
Why subscribe?
Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals
Improve your learning with Skill Plans built especially for you
Get a free eBook or video every month
Mapt is fully searchable
Copy and paste, print, and bookmark content
PacktPub.com
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.
At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.
Contributors
About the authors
Inge De Bleecker has been designing and testing web, mobile, and voice experiences for more than 20 years. She builds and leads UX teams and evangelizes UX throughout organizations. She is fascinated by the communication between humans and devices. Her mantras are "design for everyone"and "test early and often." Inge has run over 200 remote studies across different industries, languages, and regions. She finds remote studies a powerful and effective way to gather user feedback that would otherwise be difficult to collect.
Rebecca Okoroji has been working in UX since 2000. She is passionate about the need for providing exceptional digital experiences and has since expanded her focus to encompass customer experience. She believes that there is no such thing as a user error, only badly designed interfaces. Customer/user feedback is essential to building a good UX, and remote unmoderated usability testing is a cost-effective and efficient way of collecting this. Rebecca has conducted over 100 such studies in globally dispersed projects for a large variety of industries and has a wealth of experience to draw on.
About the reviewers
UX blogger, author of Fixing Bad UX Design (Packt Publishing), author of the chapter about UX for Conversion for a Brazilian book on digital marketing, reviewer of the book UX Mobile (Packt Publishing), Lisandra Maioli is an Italian-Brazilian journalist with a certification in UX (General Assembly LA), a post-graduate diploma in Marketing (UC Berkeley), in digital marketing (UCLA), in Interactive Digital Medias (Senac SP), and in Digital-Cultural Journalism (PUC SP). She has about two decades of international and multidisciplinary experience in digital communications in different roles, working for different companies and clients based in Brazil, the US, Italy, Ireland, China, Germany, and the Netherlands.
Jens Jacobsen created his first web page in 1995 with Notepad and tested it in NCSA Mosaic. Soon, he shifted his focus from coding to conceptual work and wrote a book on the conceptual design of websites, Website-Konzeption, published in German by Addison-Wesley in 2001. It is now in its 8th edition. Jens writes for several German expert blogs about usability, UX, and creating successful websites. He works as a freelance UX consultant for mid- and big-sized companies.
Packt is searching for authors like you
If you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.
Table of Contents
Title Page
Copyright and Credits
Remote Usability Testing
Dedication
Packt Upsell
Why subscribe?
PacktPub.com
Contributors
About the authors
About the reviewers
Packt is searching for authors like you
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the color images
Conventions used
Get in touch
Reviews
Why Everyone Should Run Remote Usability Studies
Usability testing methodologies
In-person or lab usability testing
Remote usability testing
Advantages of remote methods
Extended reach
Typical devices
No travel required
No travel costs
No lab costs
No lab environment
Familiar equipment
User's natural environment
In the wild testing
Larger number of participants
Disadvantages of remote methods
Distributing the product under testing
Other disadvantages
Remote moderated usability testing
Advantages of remote moderated studies
Body language
Tailored follow-up questions
Disadvantages of remote moderated studies
Remote unmoderated usability testing
Advantages of remote unmoderated studies
Natural behavior
Time zone independent
Less effort for the UX researcher
Eliminates influence
No schedules to manage
Testing with minors
Faster turnaround
Disadvantages of remote unmoderated studies
Guidance
Other disadvantages
Types of usability studies
Formative and summative usability studies
Qualitative and quantitative usability studies
Longitudinal and single-session studies
Comparative and single test object studies
Other study types
Summary
What Not to Forget When Planning Your Study
Who are the stakeholders?
The product being tested
The goal of the study
Determining the status quo
A regular study to measure against a baseline
Why? (Finding answers to questions about increasing conversion/decreasing drop-off)
Understanding the users
Global suitability
Comparing design options
Comparing to competitors
Feature validation
Concept validation
Product validation
What to do next
Predicting future behavior
Classic usability goals
The budget
The scope of the study (the tasks and questions)
Objective benchmark scores
Task completion rate
Time-on-task
Subjective benchmark scores
Customer Effort Score
Single Ease Question
Task satisfaction rate
Net Promoter Score
System Usability Scale
USERindex
The test environment
Which are the target devices for the study?
What state is the product in?
How can the product be accessed?
Are special credentials required?
Are there any known bugs?
Any other requirements
The participants
Demographic requirements
Device requirements
Other criteria
The number of participants
Participant incentives/compensation
The study methodology
Moderated or unmoderated study
Video-based and survey-based studies
The schedule
When will the product be available for testing?
Are there any hard deadlines?
Are there any update cycles that need to be taken into consideration?
Do the participants need to be equipped with physical devices?
What time of year is the study planned for?
Is a pilot run with participants/stakeholders necessary?
Is external approval required?
Do external factors impose a schedule on the UX study?
The deliverables
Study framework for sign off
Participant screener
Discussion guide/script
Report
Next steps
Summary
How to Effectively Recruit Participants
Who to recruit
Multiple target user groups
When the target user (almost) doesn't exist
Recruiting for specific target user groups
Minors
Seniors
People with disabilities
Subject matter experts
Who is really participating in your study?
How to find participants
Self-recruiting
Coworkers
Friends and family
Recruiting Ads
Customers
Panel companies
Representations of target users
Expectations
Screening the participants
Screener methods
The number of participants
Challenges
Informing the participants
Time commitments
Location
Online versus offline
Devices and tools
Expectations
Compensation
Monetary compensation
Cash
Gift cards
Donations
Non-monetary compensation
Lottery drawings
Promotional items
Sheer gratitude
How much is the right amount?
When to provide compensation
Extra costs
Next steps
Re-screening participants
Building a database for future recruiting
Summary
Running a Remote Moderated Study
Discussion guide
Anatomy of a discussion guide
Introduction
Warm-up
Tasks
How many tasks and questions?
Task descriptions
Topics
Questions
Post-session questions
Wrap-up
Writing tips
Tone
Style
Preparing for the study
Number of participants
Days for the sessions
Product being tested
Internal team
Number of sessions per day
Time of day
Dry run
Backup slots
Floaters
Sample schedule
Communicating
Communicating with observers
Communicating with participants
Communication tools
Being prepared for changes
Pre-session tech setup
Running the study
Running the sessions
When to abort a session
The participant is not getting it
Mismatches in participants
Someone other than the recruited participant joins the session
Participant misrepresented themselves
Debriefing
When enough is enough
Blocking usability issues
Consistent feedback
Moderator techniques
The rules of the game
Understanding your participants
Level of engagement
Participant disposition
Managing observers
Moderator note-taking
Basic tool functionality
Next steps
Summary
Running a Remote Unmoderated Study with User Videos
About user videos
Screen capture
Audio capture
Think-aloud protocol
Formulating thoughts
Continued thinking aloud
Strengths of user videos
Observation of user actions
Think-aloud verbal cues
Facial expressions
A picture is worth a thousand words
Suitable throughout the development cycle
Limitations of user videos
Privacy of user data
Digital rights management (DRM) 
Less deliberate feedback
Session organization
Session duration
Number of participants
Number of tasks
Number of videos
The script
Introduction
Instructions
Think-aloud
Give examples
Greeting and task description
Video recording
Video file format, size, and delivery
Access to the product under testing
Device to be used
Read task instructions
Tasks
Tasks that are easy to complete
Tasks that are difficult to complete
Complex tasks
Long tasks
Goal-oriented tasks
Post-task or post-session questions
Writing tips
Provide a clear starting point for each task
Tone
Preparing for the study
Scheduling participants
Running the study
Next steps
Summary
Running a Remote Unmoderated Study with a Survey
About surveys
Write-down-what-you-think protocol
Strengths of surveys
Long tasks
Deep dive questions
Suitable throughout the development cycle
Larger number of participants
Limitations of surveys
Not ideal for path analysis
Fraud
Self-reported data
Survey questions
Open questions
Closed questions
Single-answer multiple-choice questions
Multi-answer, multiple-choice questions
Semantic differential scale
Ranking questions
Grouping questions
Study organization
Session duration
Number of participants
Number of tasks
The script
Introduction
Instructions
Write-down-what-you-think protocol
Access to the product under testing
Device to be used
Tasks versus questions
Tasks
Task context
Task flow
Tasks that are easy to complete
Tasks that are difficult to complete
Complex tasks
Long tasks
Goal-oriented tasks
Post-task questions
Time-on-task measurement
Post-session questions
What makes for a good question?
What is the desired answer?
One question per question
Avoiding leading questions
Meeting the goals
Clarity
Tone
Unblocking participants
Providing clear anchors
Providing a clear starting point for each task
Facilitating sentiment expression
Follow-up questions
Obvious questions
Preparing for the study
Dry run
Scheduling participants
Running the study
Next steps
Summary
Running a Remote Unmoderated Study with a Hybrid Approach
About the hybrid method
Strengths of the hybrid method
Self-reported data informed by user videos
Device and product version confirmation
Reducing fraud
Limitations of the hybrid method
Increased analysis effort for the UX researcher
Study organization
Session duration
Number of participants
Number of tasks
The script
Introduction
Instructions
Standalone responses
Tasks
Recording start and end
Balance between open questions and user videos
User videos for relevant tasks only
Post-task or post-session questions
Writing tips
Preparing for the study
Scheduling participants
Running the study
Next steps
Summary
What to Consider When Analyzing and Presenting the Study Results
Analyzing the data
Preparing the raw data
Remote moderated studies
Remote unmoderated studies with videos
Remote unmoderated studies with surveys
Hybrid remote unmoderated studies
Compiling the findings
Observed or recorded data
Self-reported data
Interpreting the findings
Visualising the data
(Stacked) column/bar chart
Line chart
Pie chart
Table
Word cloud
Lists
Best practices when representing data
Identifying issues
Where did the study participants struggle with a task?
Where did the participants voice (or respond with) insecurity?
Were there any big emotions, whether negative or positive?
Are there any outliers?
Are participants consistently using different terminology?
Are there any inconsistencies in the responses?
Were there any "false positives"?
Did anyone encounter bugs?
Identifying recommendations
Make recommendations constructive and direct
Provide detail and illustrate
Address only the original usability problem
Speak the readers' language
Provide alternatives
Solve the problem
Assign a severity
Focus on the user
Reporting the data
Audience
Content of the report
Cover page
Summary
The goal(s)
Any critical issues
The bottom line
Optional details
Study context
Benchmark scores
Findings
Improvement/remediation recommendations
Verbatim participant comments
Conclusion and next steps
Appendix
Summary
Thanks! And What Now?
The debrief session
Next steps
Summary
Sample Material and Further Reading
Sample material
Further reading
Other Books You May Enjoy
Leave a review - let other readers know what you think
Preface
Does the world really need yet another book about usability testing? We believe it does as there are no books available that are focused solely on the very exciting realm of remote usability testing. Hopefully, by the end of this book, we will have convinced you too.
Please take the time to read this preface because we want to ensure that you do not go into the book with expectations that we cannot fulfill. We promise to keep it short.
This book is about remote usability testing, nothing else. It is targeted at anyone interested in understanding why remote usability testing is becoming more ubiquitous in the UX portfolio and how best to plan, run, and wrap up this type of study.
Some housekeeping before we dive in:
In this book, we use the terms
usability test,
usability study
, and
UX study
in the broader sense of user testing, which comprises evaluating product acceptance, the perceived value of features and functionality, usefulness, and much more, and is thus not restricted to the pure
evaluation of
ease-of-use.
We use the terms
product
,
interface
, and
digital interface
interchangeably throughout the book when referring to the product under testing.
Throughout the book, we refer to the person executing the study as the
UX researcher
. The person triggering the study will be referred to as the
study sponsor
.
We understand that the actual setup of a usability test with regard to the involved stakeholders may differ from reader to reader. The person running the study may be part of the UX team designing the product under testing or an external consultant contracted only to run the test; t
he study sponsor and the UX researcher may be one and the same person, and so on. We will stick to these terms and you can translate them into your specific context.
The book is tool agnostic. There are many tools available that support remote usability testing, but we do not want to recommend any specific software considering how quickly new, updated products are made available. We also believe that UX researchers should use the tools they are most comfortable and familiar with in order to be able to focus completely on the study instead of the tool itself.
That's all! We've had a great time writing this book, and we hope you find it useful.
Inge and Rebecca
Who this book is for
This book targets both User Experience (UX) professionals who are familiar with traditional in-person usability testing methodologies, and UX designers who have had no prior exposure to user research and usability testing. This book may also be of use to customer experience professionals, product managers, or frontend developers who are interested in understanding remote usability testing.
What this book covers
Chapter 1, Why Everyone Should Run Remote Usability Studies, explains why there are only very few situations in which a remote usability study would be ill-advised.
Chapter 2, What Not to Forget When Planning Your Study, describes how to plan a study, comprising all aspects that will influence its successful execution.
Chapter 3, How to Effectively Recruit the Right Participants, outlines how to determine whom to recruit, how to recruit and ensure that they are properly informed.
Chapter 4, Running a Remote Moderated Study, describes how to successfully run a remote moderated study.
Chapter 5, Running a Remote Unmoderated Study with User Videos, describes how to successfully run a remote unmoderated study using user videos.
Chapter 6, Running a Remote Unmoderated Study with a Survey, describes how to successfully run a remote unmoderated study using surveys.
Chapter 7, Running a Remote Unmoderated Study with a Hybrid Approach, describes how to successfully run a remote unmoderated study using a hybrid approach of user videos and surveys. This chapter builds on the previous two chapters.
Chapter 8, What to Consider When Analyzing and Presenting the Study Results, gives guidelines on writing reports that will help the audience understand the study results and empathize with the users' experience.
Chapter 9, Thanks! And What Now?, talks about possible next steps once the study results have been evaluated and, optionally, documented.
To get the most out of this book
The reader does not need to have any prior experience with usability testing but should be familiar with the concepts of user-centered design.
No software is needed for this book.
Download the color images
We provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: http://www.packtpub.com/sites/default/files/downloads/RemoteUsabilityTesting_ColorImages.pdf.
Conventions used
There are a number of text conventions used throughout this book.
Bold: Indicates a new term, an important word, or words that you see onscreen. For example, words in menus or dialog boxes appear in the text like this. Here is an example: "Select System info from the Administration panel."
Get in touch
Feedback from our readers is always welcome.
General feedback: Email [email protected] and mention the book title in the subject of your message. If you have questions about any aspect of this book, please email us at [email protected].
Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details.
Piracy: If you come across any illegal copies of our works in any form on the Internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.
If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.
Reviews
Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at Packt can understand what you think about our products, and our authors can see your feedback on their book. Thank you!
For more information about Packt, please visit packtpub.com.
Why Everyone Should Run Remote Usability Studies
If you are reading this book, then you probably already know what usability testing is, but we don't want to make any assumptions about that, which takes us right to the essence of this book: usability testing is about not making assumptions. In fact, it takes the opposite approach. Usability studies are executed in order to gain concrete, actionable insights. Instead of assuming that users will use an interface in a certain manner, usability tests are run to actually monitor their use of the interface, identify where they stumble and what they appreciate, hear their thoughts, understand their decisions and ultimately use this information to improve the product.
Let's take a step back and talk about usability. Usability refers to how easy an interface is to use. It is a quality that every interface inherently possesses. Issues arise when this quality is not very pronounced. ISO 9241-11 defines usability as the "extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use".
Effectiveness (how well the user achieves their goal) and efficiency (how much effort is required from the user to achieve their goal) are unsurprising in the ISO description. Satisfaction, however, is interesting: even if an interface is effective (meaning that it does what it needs to do) and efficient (meaning that it does not require a lot of effort to do it), how the user feels about using the interface is also relevant. For every app someone uses, there is probably a plethora of competing apps that do exactly the same thing. So how does the user choose? Oftentimes, they go by the very subjective feeling of being satisfied or not. The evaluation of the usability of an interface is therefore based on both very objective measures, such as the task completion rate or the time spent on a task, but also on the very subjective perception of the individual user using the interface. Did the user like the design? Are the colors pleasing? Does the interface make the user feel good about themselves? Understanding what users like or dislike, where they struggle, and which tasks come easy to them, helps with the following:
Evaluate the usability of the interface
Identify areas to fix and patterns to avoid
Determine a usability baseline across product iterations
Compare an interface with the competition
A more formal approach to evaluate the usability of a product is to run a usability test. Usability testing refers to "evaluating a product or service by testing it with representative users" (https://www.usability.gov/how-to-and-tools/methods/usability-testing.html). Usability testing is a subdiscipline of User Experience (UX). Its goal is to ensure that a given product is easy to use and that the user's experience with the product is intuitive, useful, and satisfying. Essentially, users, who are representative of the target users, are monitored while using an interface to complete tasks that are relevant to their needs. A usability test can be run at any stage of the product development lifecycle. Usability tests in the early design stages can be used to validate a navigation concept using wireframes, for example, or test the usability of new features using early prototypes, or benchmark the ease-of-use of the final product. The feedback is then used to refine and improve the user interface, ideally in an iterative process.
We want to point out that usability testing is not market research. While usability testing is mostly focused on the interaction with a particular product, market research is usually less specific; usability testing is about ease-of-use, whereas market research is mostly about user opinions or past experience. Usability testing does not require a large number of participants in order to generate valuable results, whereas most market research tools depend on large, statistically relevant sample sets.
Usability testing is also not Quality Assurance (QA). Quality assurance is performed by qualified testers, whereas usability testing is preferably run with non-QA testers. Quality assurance is aimed at finding bugs, whereas usability testing is preferably run on bug-free implementations. Usability testing usually involves a user completing a task the way they normally would, while quality assurance testers will repeat that same task multiple times in order to mimic every possible permutation a real user might encounter.
Usability testing methodologies
Usability tests can be run in different ways and for various types of studies. Let's first look into the different ways (or methodologies) of executing a usability test. These are mainly differentiated by the following characteristics:
In-person versus remote
Moderated versus unmoderated
In-person or lab usability testing
This is the traditional way to run a usability test: users who meet the user profile requirements are invited to participate in a usability study at a lab. This methodology is almost always moderated, meaning that there is always a UX researcher present who moderates the user sessions.
We are listing this methodology for completeness, but this book only covers remote methodologies.
Remote usability testing
While in-person usability testing has been around for some time, remote usability testing has only more recently become feasible due to advances in technology and the availability of new tools.
Connectivity and the internet have made it possible for us to run usability studies remotely using software that allows the moderator to view and capture the participant’s screen, hear their audio input, and view their faces. Enhanced distribution and recording tools have also made it possible to target participants offline and receive high-fidelity user videos and feedback.
Remote usability testing allows companies to gain insights into user behavior in their natural environment and on their own schedule. It involves fewer logistics, allows participation regardless of location, and is quicker and cheaper to execute compared to in-person studies, while still delivering valuable insights and feedback. In today's globally connected world, quickly getting local insights is imperative for providing the same standard of user experience while remaining culturally specific.
Advantages of remote methods
We are raving fans of remote usability testing (you might have guessed this by the fact that we wrote a book about it) for the following reasons.
Extended reach
Remote studies make it easy to include participants from different geographic locations in the same study. Essentially, anyone with a smart device and an internet connection can participate. Imagine that the product under testing is a social app aimed at allowing people to connect with each other when in proximity of another person using the same app. The typical users are city dwellers in densely populated capitals around the globe. Setting up in-person studies in each of the major cities would be logistically complex and expensive. This is where remote studies provide the biggest benefit.
Typical devices
In a remote study, study participants use their own devices. That variety of devices combined with operating system versions (and, optionally, browser versions) and connectivity speeds is near impossible to match in a lab setting. Imagine a wayfinding app that provides directions to a selected target which is meant to be rolled out globally. A remote, globally distributed study will allow the study sponsor to gather realistic feedback with regards to how the app deals with the connectivity available locally on typical devices.
