Frameworks to Write and Prioritise Testable Hypotheses in CRO
CRO, or conversion rate optimisation, is the process of improving the percentage of visitors to a website or app who take a desired action, such as signing up for a newsletter or making a purchase. One of the most important steps in CRO is developing and testing hypotheses.
A hypothesis is a statement that predicts what will happen if you make a change to your website or app. For example, you might hypothesise that changing the colour of your call-to-action button will increase conversions.
To be testable, a hypothesis must meet the following criteria:
It must be specific and measurable. For example, instead of saying "changing the colour of the call-to-action button will increase conversions," you should say "changing the colour of the call-to-action button from red to green will increase conversions by 10%."
It must be based on data or evidence. For example, you might have conducted A/B tests in the past that showed that green call-to-action buttons generally perform better than red call-to-action buttons.
It must be falsifiable. This means that it must be possible to prove the hypothesis wrong. For example, if you hypothesise that changing the colour of the call-to-action button will increase conversions by 10%, then you should be able to design an A/B test that will disprove this hypothesis if it is not true.
There are a number of different frameworks that you can use to write testable hypotheses in CRO. Two popular frameworks are the PIES framework and the RICE framework.
PIES framework
The PIES framework stands for:
Prediction. What do you predict will happen if you make a change to your website or app?
Impact. What impact will the change have on your business goals?
Evidence. What data or evidence supports your prediction?
Synergy. What specific change will you make to your website or app? How the changes integrate with other initiates and overall strategy?
Implementing PIES Scoring system
To implement the PIES scoring system, you will need to:
Identify the experiments or ideas that you need to prioritize.
For each experiment or idea, rate each of the four factors (Potential, Importance, Ease, and Synergy) on a scale of 1 to 5.
Multiply the four ratings together to calculate the PIES score.
Rank the experiments or ideas based on their PIES scores.
Implement or test the experiments or ideas in order of priority, starting with the highest PIES scores.
Here is an example of how to use the PIES scoring system for a new landing page design for the company's website.
Potential: 4 (The new landing page design has the potential to increase conversions by 10%.)
Importance: 5 (The company's website is the main source of leads and sales.)
Ease: 3 (The new landing page design can be implemented with a moderate amount of effort.)
Synergy: 4 (The new landing page design is aligned with the company's overall marketing strategy.)
PIES Score: 60 (4 * 5 * 3 * 4)
Based on the PIES score, the new landing page design should be a high priority experiment.
Tips for Implementing the PIES Scoring System
Here are some tips for implementing the PIES scoring system:
Use a consistent scale for rating each of the four factors. This will make it easier to compare the PIES scores of different experiments or ideas.
Get input from other stakeholders when rating the four factors. This will help to ensure that the PIES scores are objective and accurate.
Review the PIES scores regularly and update them as needed. This will help to ensure that you are prioritizing the experiments or ideas that are most important to your business or team.
The PIES framework is a simple and effective way to prioritize experiments and ideas. It can be used by businesses and teams of all sizes to make better decisions about how to allocate their resources.
RICE framework
The RICE framework stands for:
Reach. How many people will be affected by the change?
Impact. How much of an impact will the change have on users?
Confidence. How confident are you that the change will have a positive impact?
Effort. How much effort will it take to implement the change?
Implementing PIES Scoring system
To implement the RICE scoring system, you will need to:
Identify the features, initiatives, or projects that you need to prioritize.
For each feature, initiative, or project, rate each of the four factors (Reach, Impact, Confidence, and Effort) on a scale of 1 to 5.
Multiply the four ratings together to calculate the RICE score.
Rank the features, initiatives, or projects based on their RICE scores.
Implement the features, initiatives, or projects in order of priority, starting with the highest RICE scores.
The RICE Method ranks items by multiplying Reach (the number of users the item affects) by Impact (the result the item has on users) and Confidence (how much validation you have for your estimates). This resulting number is divided by Effort (the amount of work it will take to implement the item) to obtain an item’s final score.
Here is an example of how to use the RICE scoring system for a new onboarding tutorial for new users.
Reach: 5 (All new users will be affected by the new onboarding tutorial.)
Impact: 3 (The new onboarding tutorial is expected to help new users get started with the product more quickly and easily.)
Confidence: 4 (We are confident that the new onboarding tutorial will have a positive impact on new users, based on the results of user testing.)
Effort: 2 (The new onboarding tutorial can be implemented with relatively little effort.)
RICE Score: 60 (5 * 3 * 4 * 2)
Based on the RICE score, the new onboarding tutorial should be a high priority feature.
Tips for Implementing the RICE Scoring System
Here are some tips for implementing the RICE scoring system:
Use a consistent scale for rating each of the four factors. This will make it easier to compare the RICE scores of different features, initiatives,or projects.
Get input from other stakeholders when rating the four factors. This will help to ensure that the RICE scores are objective and accurate.
Review the RICE scores regularly and update them as needed. This will help to ensure that you are prioritising the features, initiatives, or projects that are most important to your business.
Examples
Let's say that you want to increase the number of people who sign up for your newsletter. You could use the PIES framework to develop the following hypothesis:
Prediction. Changing the colour of the newsletter sign-up button from blue to green will increase the number of people who sign up for the newsletter by 5%.
Impact. Increasing the number of people who sign up for the newsletter will increase the number of leads and sales that we generate.
Evidence. A/B tests have shown that green buttons generally perform better than blue buttons.
Solution. We will change the color of the newsletter sign-up button on our website from blue to green.
You could then use the RICE framework to evaluate your hypothesis and to prioritize it against other hypotheses.
Reach. All website visitors will be affected by the change.
Impact. The change is expected to increase the number of newsletter sign-ups by 5%.
Confidence. We are moderately confident that the change will have a positive impact, based on the results of previous A/B tests.
Effort. The change will require a small amount of effort to implement, as we simply need to change the colour of the button.
By using a framework to write testable hypotheses in CRO, you can develop better hypotheses and conduct more effective A/B tests. By following the PIES or RICE framework, you can ensure that your hypotheses are specific, measurable, impactful, and falsifiable. Additionally, by clearly identifying the dependent and independent variables, you can make sure that you are measuring the impact of the change on the correct variable.
Additional tips for writing testable hypotheses in CRO
Here are some additional tips for writing testable hypotheses in CRO:
Focus on a single change per hypothesis. This will make it easier to identify the impact of the change on the dependent variable.
Use data and evidence to support your hypotheses. This will help you to develop more accurate and realistic hypotheses.
Make sure that your hypotheses are falsifiable. This will allow you to disprove your hypotheses if they are not true.
Prioritise your hypotheses based on the potential impact on your business goals and the effort required to implement the changes.
By following these tips, you can write testable hypotheses that will help you to improve the conversion rate of your website or app.
