Experiments Handbook E-Book

Britta Ullrich

• Herausgeber: Books on Demand
• Kategorie: Geisteswissenschaft
• Sprache: Englisch
• Veröffentlichungsjahr: 2016

Seitenzahl: 44

Beschreibung

In the Experiments Handbook, we have collected the approaches we apply to learn faster in UX and Product at XING.

Leseprobe

This handbook and its experiments.

Rather than ‘just’ providing a list of tools, or a checklist for projects, we acknowledge the complexity of product management and the differences between projects by providing first a description of how we think about Experiments @ XING.

Be sure to also check out the overview and the reasoning behind why we picked experiment methods for you before jumping into the detailed descriptions of our tools.

Table of Contents

Experiments at XING. When and why.

Think. Build. Measure. Learn. A Framework.

Tools for Experiments. An overview.

Tool Descriptions. What you came for.

More Experiments – Touchpoints.

Special Purpose Experiments..

Acknowledgements & References.

Introduction

Intent & Purpose.

Learning is important.

Failure is how we learn.

Fail small, learn fast.

Introduction

Focusing an organisation on learning is difficult. Focusing yourself on learning in an organisation that does not is painful. We need you to keep us on the right path. This handbook is a small contribution to learning culture aimed at both you and XING.

We provide some snippets and pointers that can help you make a difference. Test yourself: when did you last make a decision that was not based on facts, or even one based on despite contradictory evidence? Let’s change this.

The [email protected] team.

Britta, Sarah, Steph, Cord and Hartmut.

Experiments at XING.

When do experiments make sense? We believe: all of the time.

However, when you think about doing an experiment now, it is most likely your task is to come up with a solution for a customer problem.

This means that a strategic frame exists to work from – data supporting the strategic decision to invest in a business opportunity. Your task is to refine and reframe this opportunity.

Learn before you ship.

Experiments should become a natural part of your work.

Experiment to mitigate risk as you discover the customer problem, and experiment to optimise as you develop and iterate on your products.

A ‘Thinking Experiments’ Framework

Think. Build. Measure. Learn.

Opening up. Closing down.

Experiments can be used to learn – both in the sense of opening up possibilities and in the sense of closing down by eliminating false assumptions. As you move towards the definition of a solution, opening up helps you to identify the areas where the greatest opportunity for business and users meet, while closing down helps to understand what we cannot or should not build. Digital technology is powerful, so it is often more a question of what we should build rather than what technology we can afford.

Think first.

Make sure everyone in the team understands the strategy and business needs – and make it clear what assumptions you might have regarding user needs and pain-points. If you identify any gaps in your understanding of the problem, plan your first experiments in order to learn more about the why – before moving to the solution space.

Then build for learning.

As you start designing the artefacts that help you gather feedback, think about what you want to learn, and use that as a basis for deciding how to build – both in terms of design choices and technology choices.

Measure the right thing.

If you are lucky, you can close the loop early on and directly measure impact on the KPIs you aim to move. You often need to take smaller steps; while these might not directly move a needle, be clear as to how they contribute to the bigger picture. Make sure to close the loop eventually – aim to cover all the steps towards your defined goal.

Think. Experiments in the Problem & Solution Space.

In the problem space, experiments can lead you beyond where you wanted to go. With experiments, you can find out whether you are focusing on the right problems – or if your assumption that there is a problem holds at all.

In the solution space, experiments are difficult to get right but rather obvious to define: Which of these options is better? Is my idea working? Finding the right tool to get the right kind of feedback is difficult.

Think. What makes a good hypothesis?

Quality criteria for hypothesis.

A good hypothesis expresses assumptions that

focus on most impact on outcome – or on the most critical risks,

build on existing learnings (business needs, validated user needs and solutions),

when validated can be immediately acted upon,

are clearly measurable and allow an experiment that can be safely generalised,

Impact & Risk.

What problem are you trying to solve? What value are you trying to create? Make sure it’s real. Which of your (implicit) assumptions will make you fail?

Three Steps to the measurable hypothesis.

The general hypothesis states the general relationship between the major variables.

The directional hypothesis refines the general hypothesis by stating the direction of the difference or relationship.

The measurable hypothesis makes direction more specific. If…then…because is used as a frame, and words like I, think, believe, all, never, and sometimes are avoided.

Assuring actionability.

Does a validated or invalidated hypothesis help in your definition of a solution? An experiment should cut down possible paths of action but allow for alternatives.

Untangle your hypotheses for clear measurability.