By: Paras Chopra
Many people believe that A/B testing goes against the basic sentiment behind usability. After all, a usable website should have high conversion rate and so you don’t require A/B testing? Well, answer to that question is both yes and no. Every highly converting website should be usable, but every usable website need not be highly converting. That is because the same website can be made usable in multiple different ways each of which may have different conversion rate. Finding what set of usability techniques work best for the website or landing page is what you need A/B split testing for.
Case Study
In this particular case study (also featured in WhichTestWon), you will see how following a usability “best-practice” (of having a navigation menu) increased conversions by 34% for Slideshop.com, an eCommerce website selling powerpoint templates. This test was conducted by our certified partner agency Optimeria. They specialize in using Visual Website Optimizer for conversion rate optimization with a tight focus on analysis of browsing behavior and psychology of customers v/s non-customers.
The goal of this A/B test was to increase user engagement (on site), and also create a better flow from categories to subcategories and between main categories. In original version of Slideshop.com, all pages had a promotional bar on the right which had special offer, customer testimonials, etc. See how original version looked like:
Original design of Slideshop.com
Hypothesis
As is true for all successful A/B tests, Optimeria didn’t randomly come up with variations to test. In fact, they analyzed the browsing behaviour of the visitors, and saw that they weren’t as committed to staying within a category or subcategory, and heatmap and clickflow analysis of multiple key pages indicated that visitors did not sufficiently engage the top menu – specifically subcategories. This let to the thesis that by improving engagement we would see more activity in the more focused subcategories. The aim was to also improve usability on the site, and see how it affects clickflow patterns.
Variation
Optimeria decided to test a version with better usability. They replaced the promotional right sidebar by a left navigation bar that did not have the heavy focus on single visual elements (VWO team helped them setup this test, see details of how left v/s right A/B test was setup). See below how variation with left navigation menu bar looked like:
Navigation menu on the left: 34% increase in Add to Cart clicks
Note that all subcategory pages are neatly displayed in left navigation bar and it is much more intuitive to browse around different pages on the ecommerce store.
Results
The variation design for Slideshop saw 8.9% lift in user engagement and a staggering 34% lift in people adding products to cart. Besides that a lot of key patterns emerged when looking at the new clickflow patterns. They were impressed with the engagement lift, as Slidshop already had a good engagement rate, but they was very surprised about the lift in adding to cart (as it was not expected – they simply added a navigation bar, why should clicks on add to cart increase?).
Role of A/B testing tool
The increase in conversion rate, of course, made Optimeria and Slideshop.com very happy. Karsten (of Optimeria) gives a lot of credit for that success to tool used for testing:
What I like the most about Visual Website Optimizer is specially the fact it saves me a lot of time in the actual execution of the test -phase, so that I can focus a lot more on the hard analysis, thesis and reporting phase. My clients don’t necessarily see the difference, but they get much more learning value from me using VWO because I can focus on what I do best.
Also when I am dealing with a client that has an overstressed IT department (Slideshop does not, but it happens more often than not) it’s great that I can plan, setup and execute tests and deliver results for decision makers without having to rely on a sometimes slow and tough IT process.
Decision makers often like the fact they can get feasible results before they need to involve IT department in doing changes to the website.
Lesson Learnt: usability is a must for better conversions
When asked for the lessons learnt, Karsten Lund (CEO of Optimeria) said:
One should try to minimize browsing obstacles for visitors. Sometimes more clicks are better as long as they indicate a move forward in the conversion funnel. Better usability creates more motivated visitors.
The counter-intuitive idea of “more clicks may lead to higher conversion rate” was also proved in another A/B testing case study where it was found out that removing a signup form on landing page actually increased signups by 60%.
Seems like A/B testing usability techniques is an important part of conversion rate optimization!
Source: VWO BLOG
Note: this article does not represent the viewpoint of this website.