Some site designs are more efficient and productive than others, whether your goal is to maximize pages per visit, time on page or even revenue per visitor. Problem is, how do you efficiently test possibilities? Sure you can A/B test two designs, but what if you could test dozens or hundreds of designs simultaneously to quickly identify the best layout for your goal?
This type of testing is known as multivariate or Taguchi testing, and it can be quite mind-numbingly complicated. Imagine keeping track of dozens of variables simultaneously!
That’s why I’m really intrigued to be running the Ezoic UI testing system…
I like experimenting with things, whether they’re technological or just recipes in the kitchen. There’s something about exploring possibilities and stumbling upon something that’s an improvement to the baseline. Heck, even knowing that you tried something and proved that it was worse than what you have now is a success because it helps confirm that what you have is good and helps you keep refining your results.
The problem is, it’s darn hard. That’s why I signed up for Ezoic and am letting them deliver the layout of the pages here on AskDaveTaylor.com. I specified that my goals were to maximize time on page, pages per visitor and revenue per page and they do the rest of the work, parsing my pages to identify blocks of information then rearranging them — and redesigning pages on the fly — to improve these particular results.
Thing of it is, I don’t actually know what your’e seeing at any given time, and given the nature of Ezoic, there are also dozens of different designs competing in the race at any given time.
It’s best demonstrated with this fascinating graph of designs and results over time:
You can see how bad designs spike in percentage of visitors who see it then quickly vanish, to be replaced by more successful ones. The winners are those that show up and stay around for a while, eventually supplanted by yet another design that’s even better at attaining the desired results. Right now, dark purple seems to be winning, doesn’t it?
Looking along the top, you’ll see that Ezoic is also working simultaneously with mobile, tablet, desktop and iOS app (if you have one) results. For me, it’s just the first three, but it’s really interesting to be able to pull out mobile design, for example, and see how those perform versus desktop visitors.
I can, of course, drill in further, at which point you can see some of the statistical results by design:
Here you can see that Macaw has the best page views/visit value, ties with Ling for time on site (TOS), and is the most valuable in terms of effective value per thousand visitors (EPMV, where “M” is the Latin “mil”, for thousand), and that just about all of them have a sufficiently high degree of statistical confidence (e.g., have collected enough data to be statistically meaningful) that the system has moved on the next designs.
In reality, this graph shows me that Garfish was doing great, with 429k visits, but it was supplanted by Ling, which has 85k visits. Macaw should be replacing both of them, however, since it has better results in all the important measures.
Like I said, very cool mad scientist results.
The designs, however, are not always particularly aesthetically pleasing because they seek to maximize specific results, not balance everything, and that’s something that I have had to get used to. Here are a few examples. Let’s start with the official “real” site design:
Pretty attractive, I think.
But push it through the Ezoic black box and it can end up looking like this:
Notice more white space and, of course, more advertisements.
Why the white space? Here’s what Ezoic had to say when I asked the team: it’s a large ad unit – called a 970×250 ‘billboard’ ad – especially for large monitors – which will get filled with the highest paying ad, no matter what the size – but if there isn’t an advertiser to to take that bigger slot, it’ll get filled with a smaller ad. As the first ad of a user session is the most valuable, the system has done the right thing and followed Google best practice for best revenue, and filled that with another ad slot – which does create white space, but it’s not necessarily representative of what a typical user is actually seeing. Fuller explanation on why bigger ad slots get filled with smaller ones can be found in the AdSense help area.
A different user might well see something that’s a hybrid of those two and no white space at all, just a tight, attractive layout like this:
Or how about this version, as displayed on an Apple iPad?
One more, an iPhone:
This one ends up showing a lot of ads, but that’s just Ezoic doing the right thing when I tell it to maximize my ad revenue. What’s a bit hard to fully understand is that there are dozens of different versions of the site design being tested at any given moment, so without going into the Ezoic backend and previewing them all, I have no idea exactly what you’re seeing when you visit this page.
Truth be told, I don’t often look at my Web site so I can just pretend it’s all good, but with its mono-focus on the statistical results that are important to me, Ezoic can sometimes product page layouts that aren’t particularly attractive. But then again, my site’s seeing substantially improved revenue, pages-per-visit and time-on-page, along with a substantial increase in Google traffic (it doesn’t hurt that Ezoic is one of a handful of certified Google AdSense partners, I’m sure).
One person doing design, even tapping best practices, can only accomplish so much in terms of improving a Web site. On Ezoic I can even see how the basic “standard” design is doing too, and it’s nowhere near as good as the Ezoic designs. Which leads to what is perhaps the most important question: how much are you willing to sacrifice in terms of traffic and revenue at the altar of aesthetics and design?
Me? I’m happy to have Ezoic help me maximize results on AskDaveTaylor.com.
Disclosure: Ezoic sponsored this post, but I wouldn’t have written this article in the first place if I didn’t find that Ezoic was producing beneficial results for my site.
Hey Dave, thanks for the Ezoic writeup – any update for 2020?
Hi Katie. Biggest update is probably their speed tools that have dramatically improved my Google speedtest numbers. Is that producing more traffic / revenue? Hard to quantify, but I like a faster site, that’s for sure.
This is Great. Please I Have tried using them but looks odd with ads placement but hope the tester finally optimize a static position.
Ezoic ads not displaying on my website : http://www.sebelasstudio.com, can u explain how to fix them? I saw the ads on this site is make the user experience decreased. Sorry for my bad english
Interesting post. I completely understand you not wanting to let go of your design but still wanting to see increased ad revenue and improved ux metrics. That’s why I don’t understand why you decided to use Ezoic of all the adsense testing tools out there that don;t touch your page design and provide a self-serve platform for you to play with your ad layout without touching the content layout. I’ve used one that has provided significant lift in my adsense RPM.
I have been using Ezoic on one of my technology websites (http://www.networkstraining.com/) for around 5 months now. Here is my review:
My adsense revenue did increase compared to my original layout and also website visitors increased as well (I don’t know if this is due to Ezoic though)
The only disadvantage is website speed. With Ezoic is much slower than before although they use Amazon cloud to serve your website. Before Ezoic I was using a free CDN and the speed was much faster.
Hello there Taylor, I recently started using Ezoic on my site http://www.cruisewithgambee.com/. While the first week was good, that was not case for the following weeks. I am still struggling on wether to leave it running or not to. My earnings this month are close to 40% down. I do get much less traffic than you do, so my testing period is much longer.
I have a question, and I was wondering if you could help me out. Ezoic has not been of much help in this department. Would you mind sharing your current EPMV values, for mobile, table and desktop independently? I am simply trying to get an idea of a possible achievable number.
Thanks for the help 🙂
I’ve been trying Ezoic on my site for about a month now, and although http://www.fromoldbooks.org gets many fewer visitors than http://www.askdavetaylor.com (ask him what? What colour socks are you wearing?), like you I found it hard to let go of site design. I’ve also been slowly fixing things that were broken – the “purchase” and “donate” buttons are the biggest so far! – but it’s not too bad. Did you have any technical problems?
On the other hand revenue is actually slightly down for me, but looks like it may improve over time. Maybe I should move the “revenue” slider up further.
Have you found a way to preview specific experiments? I’m guessing they’d prefer to make that difficult as otherwise you could pick one you liked and ditch Ezoic…
Thanks for your note, Liam. I’m wearing grey socks. Just FYI. 🙂 And in terms of ditching Ezoic, if it works, then there’s no reason it shouldn’t keep working forever, continually making the site design better and better, right?
Thanks for replying.
One danger with continuous experiments of Web site colours and layout and style may be a drop in brand recognition. If I found a particular layout and style that was significantly better I’d like to know what it was and stick with it, rather than keep changing, I think, but Ezoic seems to make that difficult.
And, try shopping for more interesting socks 🙂
I bet they sponsored that made up EPMV too. If you were earning over $10 per thousand views and have an Alexa of under 50,000 you wouldn’t be asking for people to buy you a coffee. You’d probably have your own office overlooking central park and slicking your hair back for no good reason.
Interesting calculation, but incorrect, Matthew. Remember, the ePVM is just for that design, right? 🙂 And I really like the idea of slicking back my hair. Hmm….
Ezoic sponsored this post!
You old dog you.
and, for the record, yes, Ezoic did sponsor the post, as I disclosed. No secret there.