What is the Taguchi method?

I had no idea about Taguchi methods of statistical analysis until I heard a splendid talk from David Bullock at the System Seminar a few months ago, and even then, I kept thinking that it was somehow related to the popular Japanese Tamagotchi toys.

But more seriously, the Taguchi method was developed by statistician Genichi Taguchi to improve the quality of manufactured goods and it proves to be more widely applicable to a variety of situations where you're testing more than one variable simultaneously. The key question that Genichi was trying to address was: how many different conditions do you need to experience to be able to reliably ascertain whether "a" is better than "b" or not.

An interesting article on this subject is at Offermatic, where they offer this explanation:

"The Taguchi Method ... has been employed with great success to optimize automobile and product manufacturing. Genichi Taguchi devised a methodology for testing multiple variables in the manufacturing process using a subset of the number of tests required by traditional testing methods. In large-scale production scenarios, every test is expensive so the Taguchi Method proved extremely valuable. In addition to reducing the number of sequential test, the Taguchi Method discriminates between signal and noise and estimates the positive or negative value of each element (factor) in each alternative (level). The Taguchi Method works so well in manufacturing that it is now being applied to traditional advertising, Internet PPC advertising and Landing Pages."

They break down application of the Taguchi method to landing page design into four steps:

1. Select elements on the landing page that we believe will influence sign-up
2. Create alternatives for each element
3. Create test "recipes" that combine these elements according to the Taguchi Methodology
4. Set up and run a concurrent multi variable test using the Offermatica ASP testing framework.

What can you test? The most obvious is headlines, but really just about any element of the page you might want to vary, from graphics to font, colors to background images, ordering of bullet list items, whatever. Indeed, the most elements you seek to test, the more valuable that Taguchi testing proves to be, because you really don't want to a/b test, say, twenty variables in less than six months!

David, who I learned about Taguchi from in the first place, recommends Vertster as his choice of tool for automating Taguchi testing. He offers the following benefits:

• Nothing to load on your server.
• Template driven experiment design. (Walks you through step by step)
• Great online documentation
• Enterprise Level Application
• Many of the features and functions that I require for my testing are built in.
• Real time stats
• Ease of use. You can click and create a creative after you setup your template.
• You can change the configuration of your test with a few clicks of your mouse.
• This is a hosted solution.
• SEO protection - you can test without degrading your search engine rankings.
• Profit protection
• Monthly subscriptions are available.

Again, the method itself is really focused on helping you test a variety of different elements simultaneously, so you can ascertain the best possible combination in the shortest possible time. And, if you're testing, isn't that just what you want to do anyway?