20bits - Latest Comments in Statistical Analysis and A/B Testing

Re: Statistical Analysis and A/B Testing

Yvonne — Mon, 19 Jan 2009 10:19:09 -0000

If for example, you had a treatment D which had a z-score of -2.94 - would you then be 95% confident that treatment D is worse than the control?

Re: Statistical Analysis and A/B Testing

jfarmer — Sun, 16 Nov 2008 12:55:48 -0000

"The conversion rate for each treatment approximates a normally distributed random variable" is more correct.

Re: Statistical Analysis and A/B Testing

hadley — Sun, 16 Nov 2008 10:40:34 -0000

"The conversion rate for each treatment is a normally distributed random variable" - are you sure??

Re: Statistical Analysis and A/B Testing

hadley — Sun, 16 Nov 2008 10:39:32 -0000

If you're going to use R, why not actually use the appropriate test - in this case it would be prop.test() for testing the different between two proportions.

Re: Statistical Analysis and A/B Testing

jfarmer — Sat, 15 Nov 2008 16:24:27 -0000

Oh, that was weird.

Re: Statistical Analysis and A/B Testing

jfarmer — Sat, 15 Nov 2008 16:24:16 -0000

Where did my comments go? :(

Re: Statistical Analysis and A/B Testing

Brendan O'Connor — Thu, 13 Nov 2008 18:33:39 -0000

The most important thing to know is a software package to use -- you don't want to muck around coding this yourself. R's t.test() is a good choice. (I guess Excel is good too.)

Re: Statistical Analysis and A/B Testing

Brendan O'Connor — Thu, 13 Nov 2008 18:33:21 -0000

The most important thing to know is a software package to use -- you don't want to muck around coding this yourself. R's t.test() is a good choice. (I've heard Excel can do it too I suppose.)

Re: Statistical Analysis and A/B Testing

Chris — Thu, 13 Nov 2008 01:21:20 -0000

They were using an Excel spreadsheet and were kind enough to make it public.
http://blog.socialmedia.com/wp-content/uploads/...

I just tried it and had to watch the video a couple of times to understand how to use the spreadsheet.

Re: Statistical Analysis and A/B Testing

Jesse — Thu, 13 Nov 2008 00:36:05 -0000

Chris,

That's a nice video. I wonder what software they were using?

Re: Statistical Analysis and A/B Testing

Chris — Thu, 13 Nov 2008 00:29:23 -0000

Nice post. Social Media had a similar blog entry a while back.
http://blog.socialmedia.com/crafting-a-statisti...