20bits - Latest Comments in Statistical Analysis and A/B Testing

Re: Statistical Analysis and A/B Testing

lutyu319 — Tue, 17 May 2011 21:38:41 -0000

Atpresent. on get older. goodness me form welcomes be only a little more low-key. I personally start to wearsophisticated basic color during. on some courtly pandora metallic bracelets .
From time to time. I’d would rather pandora charms salepurchase an enormous red-colored hand-bag that i won`teconomic utilization most probably. as well as toss directly towards water closetbecause various years. purchase a manufacturer new neutral-colored ladies handbag pandora bracelets on salepossessing pandora silver valuable metal charms beat would appear more from significance while have the abilitypandora bracelets outlet to pat it it does not create a difference throughout the evening or possibly overnight.

Re: Statistical Analysis and A/B Testing

social media marketing — Fri, 04 Feb 2011 07:51:31 -0000

asas

Re: Statistical Analysis and A/B Testing

jump analysis — Thu, 30 Dec 2010 07:05:26 -0000

hi,

it's a great information guys about Statistical Analysis and A/B Testing.

Thanks

Re: Statistical Analysis and A/B Testing

peng_s — Wed, 27 Jan 2010 05:47:33 -0000

hi friends,
I am new to R.I would like to know R-PLUS.Does any know where can I get the free training for R-PLUS.

Regards,
Peng.

Re: Statistical Analysis and A/B Testing

jtregister — Wed, 26 Aug 2009 13:19:14 -0000

This is a great resource; thanks. I'm beginning to look into the statistics behind A/B testing and have some questions. This is well after the initial post, so hopefully Jesse and others will see this.

For the distribution of the conversion rate, it seems like it should be a binomial distribution, which can be approximated by the normal distribution (as Jesse asserts in the comments) with scale.

But how about if we take this one step further and look to measure this on an e-commerce website, where there's not just conversion rate but also average order value to consider? (Really, we want to look at the contribution margin, but let's assume -- admittedly incorrectly -- that we have a 100% margin on the shopping cart.) This considers contribution per visitor, a broader metric of an e-commerce website than simply conversion rate. (And of course the subsequent step is to follow the impact on lifetime customer value, but let's not go there for now.)

Now if you consider the distribution of average order value on a typical e-commerce website, often ~95% do not convert. Of those who do convert, there's typically a normally distributed range of average order values. But if you plot the entire range of AOV, including those who don't convert, there's a huge 'peak' at zero followed by a normal bell curve. This is a more complicated distribution than a simple normal distribution.

Does anyone have insights on how to analyze the A/B results for contribution per visitor given this type of distribution? Seems like perhaps a compound Poisson, or something similarly complex. Or can someone perhaps provide a good justification of why this level of complexity is unnecessary in the analysis?

Thanks,
Jonathan

Re: Statistical Analysis and A/B Testing

Loïc d'Anterroches — Tue, 25 Aug 2009 08:06:37 -0000

Thanks a lot! You helped me getting a 50% boost in conversion rate for my website:
http://www.ceondo.com/ecte/...

I really recommend everybody to do some AB testing. I am linking to the PHP code to the tests from my article if people are interested.

Re: Statistical Analysis and A/B Testing

Traveller — Sun, 02 Aug 2009 00:28:05 -0000

This is quite impressive, I am pleased to read this post, keep posts like this coming, you totally rock!
Cheers,
Blog Review

Re: Statistical Analysis and A/B Testing

Yvonne — Mon, 19 Jan 2009 10:19:09 -0000

If for example, you had a treatment D which had a z-score of -2.94 - would you then be 95% confident that treatment D is worse than the control?

Re: Statistical Analysis and A/B Testing

Jesse Farmer — Sun, 16 Nov 2008 12:55:48 -0000

"The conversion rate for each treatment approximates a normally distributed random variable" is more correct.

Re: Statistical Analysis and A/B Testing

Hadley Wickham — Sun, 16 Nov 2008 10:40:34 -0000

"The conversion rate for each treatment is a normally distributed random variable" - are you sure??

Re: Statistical Analysis and A/B Testing

Hadley Wickham — Sun, 16 Nov 2008 10:39:32 -0000

If you're going to use R, why not actually use the appropriate test - in this case it would be prop.test() for testing the different between two proportions.

Re: Statistical Analysis and A/B Testing

Jesse Farmer — Sat, 15 Nov 2008 16:24:27 -0000

Oh, that was weird.

Re: Statistical Analysis and A/B Testing

Jesse Farmer — Sat, 15 Nov 2008 16:24:16 -0000

Where did my comments go? :(

Re: Statistical Analysis and A/B Testing

Brendan O'Connor — Thu, 13 Nov 2008 18:33:39 -0000

The most important thing to know is a software package to use -- you don't want to muck around coding this yourself. R's t.test() is a good choice. (I guess Excel is good too.)

Re: Statistical Analysis and A/B Testing

Brendan O'Connor — Thu, 13 Nov 2008 18:33:21 -0000

The most important thing to know is a software package to use -- you don't want to muck around coding this yourself. R's t.test() is a good choice. (I've heard Excel can do it too I suppose.)

Re: Statistical Analysis and A/B Testing

Chris — Thu, 13 Nov 2008 01:21:20 -0000

They were using an Excel spreadsheet and were kind enough to make it public.
http://blog.socialmedia.com...

I just tried it and had to watch the video a couple of times to understand how to use the spreadsheet.

Re: Statistical Analysis and A/B Testing

Jesse — Thu, 13 Nov 2008 00:36:05 -0000

Chris,

That's a nice video. I wonder what software they were using?

Re: Statistical Analysis and A/B Testing

Chris — Thu, 13 Nov 2008 00:29:23 -0000

Nice post. Social Media had a similar blog entry a while back.
http://blog.socialmedia.com...