20bits - Latest Comments in An Introduction to A/B Testing | 20bits

Re: An Introduction to A/B Testing | 20bits

Tim Cederman — Thu, 23 Oct 2008 02:16:25 -0000

Great article as always (and the preamble and follow up ones). As someone with a qualitative research background, I've really come around to the importance of data-driven design and taking the bias out of design decisions. However, while you qualified your statement "the landing page with the higher conversion rate is better", I think it's more than just unsustainable incentives (and the like) that need to be taken into account.

Two pages may yield the exact same conversion rate - however they may offer a vastly different quality of the conversion (demographic of the user) and experience for the user. I'm not sure why there is often a rift between qualitative and quantitative research as they are really powerful when used in a complementary fashion. Without being able to actually talk to the user and getting a sense of who they are and what their experience was like it's difficult to infer this information otherwise.

Looking forward to the next two articles!

Re: An Introduction to A/B Testing | 20bits

Jesse — Wed, 15 Oct 2008 13:24:52 -0000

Frank,

Give GWO a try, it might do what you need. I usually write my own.

Re: An Introduction to A/B Testing | 20bits

Frank Licht — Wed, 15 Oct 2008 06:13:33 -0000

Nice article :) Really helped me with the basics. You allready stated the Google Website Optizer is farly limited, but which tool would you recommend then? Thanks in advance!

Re: An Introduction to A/B Testing | 20bits

Jesse — Sat, 11 Oct 2008 20:25:04 -0000

Prakash,

Google has the Google Website Optimizer, which supports both A/B and multivariate testing, but it's fairly limited. It only measures "conversion" events.

You can't, say, segment Google Analytics reports based on GWO experiments.

Re: An Introduction to A/B Testing | 20bits

Prakash S — Sat, 11 Oct 2008 18:34:43 -0000

Does google analytics/ other analytics tool help you with A/B testing? Good intro to A/B testing.

Re: An Introduction to A/B Testing | 20bits

Jesse — Mon, 06 Oct 2008 16:44:50 -0000

Douglas,

Thanks for the kind words. Will do!

Re: An Introduction to A/B Testing | 20bits

Douglas Vos — Mon, 06 Oct 2008 13:00:11 -0000

I like the direction you are going in with this article... keep it going.
Andrew King covers similar topic in his new book called Website Optimization.