Top 5 things to consider when starting an experimentation program

By Josh Panebianco, Optimisation Lead

Experimentation is one of the most talked about topics in eCommerce right now. With shopping online becoming the go-to channel, everyone is looking to kick-start a CRO program, but how do you get started and what should you focus on in the first place?  Below are the top five things you need to consider when starting a CRO program.

1) Can you clearly define the upside for your business?

Rising ad costs, increased competition heating up, and the artful dance of scaling the business sustainably. With all this on your mind, it can be hard to focus on what truly matters. 

If you are deep in this game and playing to win I’m sure you’ll understand more traffic is easy, more customers are not. It’s likely that you don’t have a traffic problem, but a conversion problem. How much impact can more traffic truly have if they are not converting? For those in the know, traffic has almost become a vanity metric. Traffic is up 10%, but how often do you see a 10% increase in revenue in parallel?

Conversion is what truly matters. Yes – increasing traffic is great but not at the expense of profits. So with conversion rates now your full focus, what question are we looking to answer?

The golden question in CRO is this, for every one percent increase in conversion rates what is the net impact on the profitability of your business? Knowing this will create a clear and compelling goal that you can focus on when embarking on your first experimentation program. If you lead a large company, maybe even half a percent increase in conversion would transform your business. 

The second habit in Stephen R. Covey’s book The Seven Habits of Highly Effective People is, to begin with the end in mind. Focus on the end benefit of running an experimentation program and bumps and challenges along the road will all be worth it.

2) Are you only tracking the last click in the purchase process?

There’s a great chance that you already have your eCommerce goals set up in Google Analytics. This is a great start but that only tells you a very small part of the overall picture. Tracking the end conversion rate is what gets everyone excited, what gets that number to increase is tracking all the smaller steps that happen before that end conversion takes place.

As a general rule, you should move away from only tracking end conversion rates and graduate to tracking and measuring conversion events that take place across the site. This increased level of tracking can answer some very important questions about how customers are interacting with your site. 

For example:

• What is the primary user journey from the first visit to purchase?
• Which call to action button is underperforming and why?
• Are there any major drop-off points during checkout?
• How are mobile visitors interacting with the bag page compared to visitors from desktop?

3) Are you ready to focus on 70% of your customers?

Many eCommerce brands have over 70% of their traffic come from mobile and this number is only going to continue to grow. The funny thing is very few companies are making mobile-first decisions when making changes to their site. The main reasoning behind this is that traditionally, conversion rates on desktop are higher than on mobile.

The difference in conversion rates on mobile vs desktop is partly based on user behaviour but also on the fact many companies have settled for mobile responsive experiences when they should really be aiming for mobile optimised. In order to get the most out of your conversion rates on mobile, you must create a version of the page specifically designed for mobile users in mind. If you are not designing with mobile as your first consideration, you are not listening to your customers. 

4) Are you maximising your ROI from the money you spend on development?

The hidden advantage of taking a more data-driven approach to the changes you make to your online store is that as your experimentation program grows, it should then become a vehicle for how you validate sprint tasks.

Before you spend all this time, and money making changes to the site do you have any data to support these changes? Or are you making these updates based on your gut? Don’t feel bad, the vast majority of people do the same thing. The important thing is to register that pattern and move forward from it. 

With this in mind, revisit your current list of sprint tasks and ask yourself do we have any data to support this decision? If not, maybe it’s worth gathering some data to validate each of the tasks to make sure you are making the right decision.

5) Are you willing to learn from trial and error and put the right structure in place? 

Optimisation is not a straight-forward path, it takes time to learn best practices and how each stage of the CRO program is completed properly. Since you are often dealing with high-traffic areas of the site, it can affect metrics when a hypothesis doesn’t pan out. 

Depending on the structure of your company you may be able to establish an experimentation program with your current team, chances are there will be up-skilling that needs to take place. If you need some recommendations for CRO training, please reach out and we can point you in the right direction. Please keep in mind that this up-skilling process could take 6 months to a year and during that time they will need dedicated learning time as well as space to experiment.

On the other hand, if you don’t want to wait or are not willing to take the risks that can be made with a team that is still learning we would recommend you reach out to us and book a time to speak with our optimisation team that can give you a detailed overview of how we can help you increase conversion rates.

Our team have been running experimentation programs for over 10 years so they can definitely give you detailed insights on how customers are currently making purchases on your site and how to increase the overall conversion rates. In addition, you can watch our webinar as we look at some of these basics with Australian Powerhouse, Showpo and Optimizely.