All About Website Usability Blog – Holly Phillips

Simple yet increcibly effective A/B testing with Google Website Optimizer
August 20, 2009, 4:25 am
Filed under: multivariate testing

We just completed an incredibly successful A/B test using Google’s free Website Optimizer tool (yes, they do full multivariate testing too).  I HIGHLY recommend it! 

The tool is simple to use, and Google provides an impressive number of help documents, examples, and FAQs.  And did I mention it’s FREE?  Of course, you still need to create your page variations which may require external resources, but having a no-cost tool to test the variations is great.

We tested an existing P4P ad and four variations of it.  After just two weeks we had a clear winner that was getting over three times the clickthru rate of the original.  We were able to easily turn off the low-performing variations at any time, and could have just as easily followed up with a new test of newly-iterated designs if we wanted (in this case we were happy with the results and pressed for time, so simply implemented the winning design from that single round of testing and moved on).

GWO has easy-to-follow instructions on how to tag the pages, provides the code to add to the pages, and has a very simple UI for tracking results.  They do all the significance testing behind-the-scenes and only declare a winner once the statistics support it.

There are other multivariate testing tools out there, and the for-pay ones certainly have more bells and whistles than GWO.  But for basic testing like this GWO works great.  This will definitely be a standard tool we use more often.  It’s not often we can achieve 3.5x improvement for free 🙂


Leave a Comment so far
Leave a comment

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: