Going from Sporadic A/B Testing to Structured Conversion Rate Optimisation

Gareth Williams, Senior Optimisation & Analytics Consultant, takes control of today's blog, discussing how companies can go from sporadic A/B Testing to Structured Conversion Rate Optimisation. Here are some tips to help you deliver a great CRO programme...

All A/B testing contain a variable degree of probable chance – you can never predict with 100% confidence the outcome of any test, often resulting in more misses than hits. You should also disregard vendors’ double-digit conversion uplifts case studies. Whilst those do occasionally happen they are the exception to the norm. Instead look to develop a programme that delivers small gains frequently. The cumulative effect of these gains will result in a significant uplift over time.

Success with Conversion Rate Optimisation

Organisations successful with Conversion Rate Optimisation (CRO) apply a structured, continuous and iterative methodology to their testing and targeting programme. In this short post I describe in general terms what is required to deliver a great CRO programme, outlining the key disciplines and processes. 

First it is important to make a distinction between 'testing' and 'optimisation'.
Testing is the practice of conducting A/B tests which are episodic. Most tests share a common business objective (e.g. improving a website’s conversion rate) but they are normally conducted separately of each other. Whether through lack of resources or pressure from management – follow on tests are rare. Instead the business will move on to test other pages on the site rather than continue optimising the same page.

Optimisation is a continuous programme of scheduled tests delivering incremental improvements to overall conversion. The results of each test and the insight gained from the analysis of test results are used to inform future tests. Testing prioritisation is based on potential commercial benefit. So for example, the business will continue with follow on testing on the same page as long as the commercial benefit exceeds other potential tests. In addition, a CRO programme will have a stated commercial objective which could be defined as a percentage increase in conversion or product-specific sales uplifts.

Team building: the right people

Even the smallest CRO programme requires a team of people with specific skills and experience. The key people and their responsibilities are:

• Programme Manager - is responsible for the delivery of the CRO programme and hitting the commercial targets set for the programme. This includes liaising with management as well as the necessary approvals from stakeholders such as legal, brand and governance.

• CRO Specialist - is responsible for the tactical direction of the programme, ensuring that the tests are aligned to support the programme’s stated commercial objectives. This is achieved by deciding on which tests/ideas to progress, managing the testing schedule, analysing and communicating test results, providing testing recommendations and keeping senior stakeholders up-to-date.

• Content Marketer - creates the content and copy for tests and works with the brand and legal departments to comply with digital brand guidelines and legal restrictions

• User Experience (UX) Designer - provides customer focused solutions to test hypotheses, creating the wireframes and journey maps for the copywriter to build the content around
• Developer - supports the implementation of test versions and management of all technical aspects of the testing platform

Businesses looking to start small could merge some of these roles. In some cases were the CRO specialist has strong managerial experience she or he could also act as the programme manager. A UX designer could also assume the role of the copywriter if they have some relevant experience.

Obviously, it is tempting to start small with reduced costs. However, you must take into account that a smaller team could take longer to deliver the uplift required to make the programme successful. At worst case scenario they may never be able to deliver what a slightly larger team could do.

Management tips for your conversion rate optimisation programme 

Weekly CRO meetings (as a minimum) are critical for the success of the programme. These sessions, which should include representation from other teams such as marketing, sales, IT, brand and legal, are the forum to discuss new testing ideas, agree testing prioritisation, ensure deliverables are on time and discuss results. 

Providing regular updates to senior stakeholders is essential for maintaining momentum. If your senior stakeholders do not see the value the programme is delivering daily and weekly, you will quickly find it harder to secure the necessary resource to run your CRO programme. The role of management communication cannot be understated and it is the responsibility of the project manager to ensure management is constantly engaged (especially when results are not going well).

The management of a continuous schedule is what differentiates optimisation from testing. The CRO specialist ensures that there is a constant flow of potentially high value tests aligned to the programme’s stated commercial objectives. Equally important is to ensure that the learnings from every test are fed back in to the decision making process for subsequent tests. For instance, if you have discovered that leading with ‘Next Day Delivery’ on one product group generated uplift, it would make sense to test whether this was true for other products. This iterative element of testing is very often lost.

For more information and to learn how AEP Convert could help you turn testing into a sustainable conversion rate uplift visit us on www.aepconvert.com, call us on 020 3282 0459 or email us on hello@aepconvert.com.