A recent study by Adobe shows that 42 percent of marketers say analyzing the A/B test results is the hardest part of conversion optimization.

The majority of conversion optimization strategies today are behavioral ones that fix on testing elements and not strategies. During these type of tests marketers duplicate their landing pages and test different elements such as call-to-action buttons (color or text), a different title or maybe a different image. An additional way of testing elements is “flipping” the landing page and moving the same elements around on the page.

The emphasis being, of course, on having the exact same landing page as a control with one change made to it.

From my experience in the last 5 years, the reason marketers find analyzing their test results extremely hard is because they’re testing the wrong thing. Incremental changes like these can only lead to small increases in conversion rates, leaving the majority of your tests with uninteresting results and the inability to scale.

Testing strategies & concepts versus testing elements can make a huge difference in your results. We call it emotional targeting which focuses on why people buy a product or a service and not why they should. Behavioral targeting (testing elements) generally focuses on features and pricing, whereas emotional targeting focuses on the emotional reasons people purchase products/services.

What I’m suggesting is while testing you don’t have the same landing page with different elements, but you actually have different landing pages altogether — each landing page representing a different concept and idea of your audience. These types of tests have much higher results in conversion but most importantly help understand who the audience is and how to continue testing and improve.

You can find some case studies and more information about this methodology:
http://www.conversioner.com/blog/emotional-targeting/
or
http://www.semrush.com/blog/e-commerce-2/42-marketers-ab-testing-wrong-thing/

I’d love to get your feedback on this.

Even the button colors are extremely light – the color isn’t really brought out much because the buttons are too light / semi-transparent.

Going for solid, brighter colors and dropping the gradients should see a clear winner. Hell, even just changing the color of the button to a solid bright version of the same color probably would.

It never ceases to amaze me how so many websites choose design > usability.

Anyhow, good luck with the next iterations!

The entire goal with A/B testing is to try markedly different strategies to see what gets traction. Every example there the changes were minute, so minute you would have been in the last seconds of A/B refinement. They would yield nothing. From the boot strap button changes, to the micro text changes, to the 2 sets of logos neither of which has any major brands that are recognizable and thus would have no affect on user behavior.

The clear lesson here is that businesses need to learn what real AB testing is about, where value is in changes, and how to differentiate pointless changes versus meaningful changes.

Please note A/B testing works incredibly well, but you have to think about your changes, and comparative strategies. The reason the above failed is there was no real thought or changes made. In the above you tested A/A there was not enough viably different for it to be useful. Remember A/B testing should be comparing Apples and Bananas not Apple and Apple.

And in closing: read Zac Aghion’s comments, they were better than mine 😉 – cheers.

It’s refreshing to read a feedback like this one which is so close to my situation – the one of a lot of startups. So many tests without results.

Testing is important, but it is really depending of your business and the growth you have. There is plenty of things to do to build your business and A/B tests is only one of them.

One additional thought: my big takeaway here was more about how important it is for us, (startuppers), to remember that things aren’t often going to work and that’s okay. I’ve read countless articles, even just today, about “the button” you referred to several times in this post, and constantly find myself feeling discouraged. (Case and point, the last two articles I read tonight: http://bit.ly/1qdxpTt & http://bit.ly/1qdu9aD) I know we, as a community, are constantly stressing the importance of celebrating failure and the difficult path/journey that running a startup can be, but posts like this with tangible bite-sized reminders are lifeblood to the doer’s out there.

You have already done an above average job on your landing pages, design, headlines, etc. and it is entirely possible that you are at 60-90% of what the theoretical maximum might be. At some point you can’t improve some metrics in a meaningful way and you have to focus on a different area.

For example, early on in your business A/B testing a landing page is pointless because you don’t have enough traffic to see a meaningful change in a reasonable period of time. Once you get to the point where you have enough traffic going to a suboptimal page, you can probably improve it and help your overall conversion rate. Then, you might go back and try and get more traffic again.

I think there’s a natural back and forth cycle of optimization of different parts of your business as you grow.

In the manufacturing world, they will do a project and set a benchmark of improving efficiency by say 75% and hit it. Then, they come back to the same line in a year or two and improve it again. That process repeats over and over again at companies like Toyota.

Your A/B tests might be in a good place right now, but maybe you should come back in a year and try again.