Ask just about any Internet marketing professional how important testing is and they’ll quickly tell you it’s essential. But ask them how much effort they actually put into testing and their response may be a bit more equivocal. Conducting effective marketing tests on a regular basis is like flossing your teeth:

everyone knows they should do it, but many people skip it all too often.

It’s no wonder. Designing, running and analyzing valid A/B split or multivariate tests the right way can take a real bite out of your day. After you’ve isolated the best variables to test and ensured you have a large enough sample size, then you have the bigger job of actually setting up your pay-per-click (PPC) ads, email campaigns, landing pages and all the other mechanics of the test field. And when the test is complete, it isn’t just a matter of spitting out the results. You need to read between the lines and try to get down to the “why” in order to get the real value from your test.

In a profession with virtually unlimited channels, technologies and techniques to occupy our time, it’s very easy to take shortcuts with testing. Too often we make assumptions or inferences about the results we see, or just take the gurus’ word for it.

But every business and every audience is different, and there is no one-size-fits-all answer. There simply is no substitute for well-executed tests.

In this article I'll describe a landing page test we conducted as one example of how to conduct an effective test, as well as to offer some motivation to those who have started to relax their testing schedule a bit. This test was run against our search engine marketing (SEM) campaign, but the subject could just as easily be email marketing, social media or any other marketing channel. And the point is not to recommend a particular action based on our results. As mentioned, every business is different. The point is to illustrate the significant bottom-line gains you can make by testing some simple changes.

Is the Right Form Right? Or Should it be Left on the Left?

We had noticed that about half of our competitors placed their lead capture form on the right-hand side of their SEM landing pages, while we were among a minority who placed our form in the main body section on the left side of our page. (Others had no form at all.) We could have just assumed that since the majority was using a right-side form, that must be the best practice. Instead we decided to put it to a test.

Controlling the Variables

Our first step was to design an alternative landing page with the form on the right. It wasn’t as simple as it sounds. We didn’t make the right-side form as narrow as we might have liked because that changed the configuration of the form itself, which would have introduced another variable to the mix. The goal was to design a test page that we felt had only one variable: the placement of the form.

Because the ad linking to our left-side form page had been running for some time, it had an established performance history with Google that would give it an advantage over our right-side test page in terms of the ad display frequency. So to control the click volume for the two landing pages we created a clone of the original PPC ad that was just as new as the test ad.

We ran the test within several ad groups to ensure we collected a sufficient amount of data during a test period of only a few days. Even though the ads were running side by side, it’s always a good idea to conduct tests as quickly as possible so changes in the environment over time don’t factor in.

And of course, our campaigns were set to rotate evenly in the AdWords settings. In fact, because we are constantly testing something we always leave them in the “rotate” mode rather than the tempting “optimize” mode.

Ultimately, we believe we eliminated any variables that might have corrupted our findings. Now all we needed was to collect enough data to be confident in our results.

Analyzing the Test Results

We collected 168 leads in total, which was ample for us to be confident in our test results. Statistically speaking, to have any degree of confidence in your results you need a minimum sample size of 30; by the time you get up to a sample size of 300 you’re getting diminishing returns.

So with our test factors tightly controlled and a respectable sample size collected we tallied up the results. And sure enough, we had roughly equal numbers of impressions and clicks, but the number of conversions was different - about 10 percent different.

And the winner? With a 10 percent lift in conversion rate and a 13 percent drop in cost per conversion, the right-side form was a clear winner. It seems the prevailing wisdom was correct in this case. And switching to that format will help us really stretch our PPC budget.

Does this mean you should switch your landing page form to the right? Not necessarily. Your audience may be different than ours. It may be that your particular audience will respond better to a form on the left, or to a big button saying “Learn More” instead of a form, or to any number of different possibilities. Don't simply take someone else’s word for it - test to see what works best for you.

Testing to Answer the Question "Why?"

What is it about having the form on the right that made it convert better for us? Answering this question is not so easy. We could try to design a test to zero in on the answer, or we might just form some theories. In this case, for example, we may theorize that because most of our competitors put the form on the right, our visitors are used to seeing this format, so they’re more comfortable with it.

Either way, only to the extent that you can answer the “why” question can you free yourself from the need to test. That may sound rather philosophical, so let’s be a little more concrete.

If we actually knew that the reason our visitors responded better to the form on the right was because it was standard for our audience, we could skip that test and focus on other tests of different variables. We might even extend our knowledge about the conversion-inducing properties of standardized interface design to other elements of our page (for example, credibility elements) and dispense with those tests, too. Indeed, there have been many useful articles written about landing page best practices based on others’ past tests. We shouldn’t disregard this accumulated knowledge, but rather use it to guide our testing plans.

But the “why” of visitor behavior will always be elusive, so plan to keep on testing. As you’ve seen from our results above, it pays to test.

###

About the Author

Dane Christensen is the SEM Manager for Lyris. He is responsible for optimizing the company's PPC bid management across seven different search engines.

Related Resources:

• Subscribe to Inside Lyris HQ our monthly online-marketing newsletter.
• Download free online-marketing guides, tools and webinars in the Lyris Online-Marketing Resource Center.