A CMO's Guide To Website A/B Testing

A/B testing will help you better understand your audience and grow conversions.

By Zach Rozell

Conversion Optimization

Being successful at A/B testing means hypothesizing potential problems on your site and testing these ideas. With the right tools, you’ll be able to better understand who your audience is and increase conversions across your site. Remember: the more hypotheses tested, the more successes you will have.

A/B testing is a way to test multiple ideas on your website and see which ideas increase conversion rates. A/B testing ideas can come from insights formed from tracking users’ activity across your site or survey responses from your target market.

In order to be successful at A/B testing, you must first find the friction-causing elements. Then you must define the visitors’ concerns that are keeping your customers from converting.

After defining top site issues, create and test new hypotheses based on the data you uncover. You can use a number of research methods to pinpoint problem areas and test those areas to increase conversion rates. Below is a sample of helpful tools.

Usability Studies

A usability study involves:

  • Recruiting study participants to visit your website.
  • Providing participants with a list of tasks.
  • Asking for feedback as the participants navigate your website.
  • Recording the whole process for analysis.

Brands have been conducting traditional usability studies since the 1990s and focus groups since the ’50s. However, in recent years usability studies have become more popular through technologies like UserTesting.

UserTesting has taken the traditional model of usability studies and brought it to life in an online platform. The site allows you to recruit participants and manage your usability study videos; all while promising to deliver your study results in 24 hours.

When conducting a usability study you should test the effectiveness of your site by asking the right questions to the participants on your site. You must be careful in how you structure questions and be sure you are keeping an eye on the behavior of your participants.

To avoid leading the user into an answer, the questions must be written in a way that can’t be misinterpreted as direction. It’s also important not to hint at the answer you’re looking for since this, too, will skew the results.

Cart Abandonment Surveys

Cart abandonment surveys allow you to target site visitors who are showing intent to leave the page where they currently are. If you want to know why the exit rate is so high on your cart or checkout pages, the cart abandonment survey is the easiest way to gain customer feedback.

The answer format can be multiple choice or open answer. With this information, you can grasp an understanding of the site usage, the types of visitors, the visitor’s mentality/mindset or the reasons for abandoning the site during a crucial step.

Below is an example of what a site survey question looks like when a user is exposed to a survey, and the results for that specific question:

Here are eight tips to improve conversion rates.

Consumer Surveys

Mass consumer survey tools, such as Google Consumer Surveys, allow you to target the audience by age, location, income, etc. This tool helps you gather data quickly by segmenting your survey target audience and creating a wide range of surveys that can be conducted though A/B tests, brand tracking studies, market trend analysis, concept development, product development, market design, and campaign measurement.

As soon as you receive your results you also have multiple ways to segment survey responses so you can quickly glean insights down to a segmented demographics level.

For example, you may notice that consumers who live in rural areas have a higher chance of owning a pet or that younger audiences are more open to mobile apps than your older audience. The chart below shows Google Consumer Surveys’ various options to filter your survey results.

Heat Maps

Heat maps reveal where your website visitors are most frequently looking and clicking. This information will help you understand the eye path of the user and what attracts the most attention.

Heat maps can be used in navigation flow tests to verify that the user is clicking in the correct spot on the page to continue their path to purchase.

Scroll Maps

Scroll maps are similar to heat maps in that they indicate where the customers are scrolling to and spending most of their time. This data usually indicate what stands out to the customer and where information needs to be moved.

You Can Learn From Losing A/B Tests, Too

Not all test results from these studies are going to be positive. Yet, a negative result doesn’t have to stay negative.

Some of your best performing A/B tests could come from the insights gained from a losing test variation. For example, you might notice that one of your headlines does well while the other one fails. The failing headline will teach you just as much about your audience as the successful one.

Are you taking advantage of usability studies, surveys, heat maps or other tools to improve conversion rates? What has worked best for you?