Kalzumeus Software

I am going hog wild with A/B testing now that I have an easy way to do it.  I thought I’d share some of the results:

• Login as guest link vs. No login as guest link:  There is no significant difference in the number of trial signups I get if I put a discrete login as guest link on the registration form.  There is, however, a massive decrease in the number of guests (obviously, they have to decline by at least 50%, right?).  While I don’t get nearly any economic value out of guests, I reverted to allowing them.
• Asking for permission to email versus not : There is no significant difference in the number of trial signups I get if I show or hide the two checkboxes for signing up for the mailing list.  Note that I still ask people for their email address (which I use for a username, figuring this is easiest for non-technical people to remember) and that I, stupidly, included the “We do not spam you” verbiage in the block excised for the test.  That should have been left in both alternatives.  Ahh well, I’ll do it again properly later.
• Reordering buttons:   Reordering the bingo card pages (see example) to include the Make Your Own Cards (which leads to a trial registration dialogue) over the Download These Cards buttons increases the conversion to the trial from 11.90% to 19.82%, which was significant at the 90% confidence level.  Yay, a positive result!  I ended the test and adopted that behavior as the default.

What I’m testing currently:

• A new version of the purchasing page, which is specific to the online version only, for people who are logged into the online version (versus the current purchasing page).  Obviously I’m looking for actual sales as a conversion here.
• The effect of the text “It is quick and easy to get started!” versus “There is nothing to download and no credit card required.” on the landing page.  (Quick and easy are two concepts that have worked very well in my AdWords before.)

Recently on the Business of Software forums I gave someone advice regarding web design.  (I seriously, seriously envy his design prowess — the website for his VST host software is one of the best I have ever seen for a microISV.)  One issue I identified with his site was that at some points it lacked clear direction as to what to do next.

On reflection, there is a gaping hole in my own site design for this: the big screenshot on the front page.  It is ginormous and consistently picks up over 10% of the clicks on the page.  Clicking on it currently brings up a maximized version of the screenshot in a lightbox.  The lightbox is a wonderful effect, but what is the user supposed to do with the maximized version?

Well, ideally, they’re supposed to close that screenshot, then find the Free Trial or Try Online buttons and download.  Blech!  That is two more clicks, bringing to three the total of clicks they have made to my page!  Clearly, there is an opportunity to eliminate a click. All I need to do is bring the buttons to them, in the constrained environment of the lightbox.  (I’ve always wanted to do this but didn’t have any convenient way to do it with my previous lightbox technology, and never got around to doing it after moving to a less constrained lightbox script.)

I have pushed the new and improved version of the page live.   If this post was published recently, you can see the old version here.  For obvious usability wins like this one, I’m not going to bother A/B testing prior to pushing it live.  (Plus I have a lot on my plate tomorrow in terms of non-obvious things, and running multivariate testing scares me.)

Today after reading a thread or three and watching a Google-produced movie about landing page optimization (over an hour long and worth your time) I was inspired to make some changes to my site.  Recently I have been doing far more development than A/B testing, and truth be known I don’t really, um, know how my funnel is working these days.  So I went back into the archives of my CrazyEgg tests and looked around to see if there was anything that jumped out at me. The fresh look caused me to pick up on a few things:

Plug Your Leaking Funnel

If you’ve ever read anything about conversion optimization you’ve heard the funnel metaphor : the steps towards buying can be thought of as a funnel, where a lot of people come in at the top and as they move forward you see less and less people until a mere trickle actually succeed in giving you money.  You want your funnel to avoid leakage, to the maximum extent possible.

Bingo Card Creator has a fairly typical funnel:

Prospect sees me in search engine or ad -> Prospect clicks to homepage -> Prospect downloads trial (or signs up for online trial) -> Prospect plays around with trial -> Prospect likes what they see -> Prospect clicks on purchasing page -> Prospect interacts with shopping cart -> Prospect gets redirected to Paypal or Google Checkout -> Prospect fills out form -> I get money.

Yikes, saying it like that makes it sound even longer than it actually is.  Anyhow, every step is crucial because the effects are multiplicative: a 5% improvement anywhere essentially results in 5% straight to the bottom line.  (Hat tip to Steve Pavlina for this insight, which is probably one of the three most important things I know about selling software.)

Anyhow, I picked a step in the funnel where I see a lot of dieoff, the purchasing page, and took a gander to see if there were any obvious ways to improve it.  Looking at several experiments in CrazyEgg, which I collected over the course of the last few years, something jumped out at me.  See if you can spot it in these heat maps:

Early 2007:

Early 2008:

Early 2009:

Did you spot it?

Far too many users interact with the main navigation bar to go backwards in the funnel.  Despite the fact that the overwhelming majority of people on this page are here because they are satisfied users of the free trial and are seriously considering giving me money, the Free Trial and Printable Bingo Cards buttons capture almost as many clicks as any other element on the page.

I love giving users free things that make them happy, but there is a time and place for it, and that time and place is not right in the middle of me asking them for money.  So I eliminated the distracting free options (and, while I was at it, the redundant purchasing option):

(You’ll note, by the way, that a lot of online retailers take a weedwacker to their entire navigation system when you’re getting into the purchase funnel.  Check out how much cruft Amazon cuts out next time you checkout there.)

Don’t Violate User Expectations

As an engineer I love elegance… probably to a fault.  For example, I always wanted to have exactly the same top-level navigation on all my pages.  This resulted in having a button to the home page actually on the home page.  OK, perhaps that wouldn’t be the worst thing in the world — except in a fit of stupidity way back in 2006 I titled that button “Information”, and despite four people telling me “You know, people seeing a button titled Information will click it thinking that it offers Information and will get upset when it just causes their screen to blink”, I never got around to changing that.

Sell What You Sell

Recently I released an online version of Bingo Card Creator, which I strongly hope will increase my sales substantially by decreasing the funnel dropout associated with downloading and installing BCC.  However, my site is still largely written to promote the downloadable version… to an almost painful degree sometimes.  For example, if you were to click on the purchasing link from within the upsell in the free trial of the online version, you’d find:

(Incidentally, that phrasing used to be “Purchase a single copy for only $24.95″.  I changed it at the suggestion of Aaron Wall, to emphasize the benefit — you get what you want instantly — over the action.  I think that sentence has been worth over a thousand bucks.)

However, when you’re selling something which is specifically designed to avoid the hassles associated with being downloaded, offering instant download is perhaps not the best idea!  So I rewrote it again to be a bit more neutral among my products:

If I had an absolutely infinite development budget I’d code up one version of the site to only sell the online version, one to sell both, and one to sell only the downloadable version, and then start split testing like a madman.  However, that is just not in the cards in the near future.  If I have some time later I’ll consider rewriting this page for people who are logged into the online version, though.

Optimizing Offsite Payment Processors

If you’re a member of SEOBook and have not yet read leeds’ post “Great conversion tactic for you”, do so now and implement the suggestion it makes immediately. (He asked me not to repeat the advice publicly.  I’m going to respect his wishes on that one.)

However, it did get the creative juices flowing:

Usual Disclaimer

I have confidence that the above examples are good ideas but confidence loses to evidence every day and twice on Sunday.  Conversion optimization is a science, not black magic: test, test, and test again to see what works in your particular market and circumstances.

Disclaimer: I am a moderator at SEOBook, but am not compensated for plugging it.  That said, it costs about $100 a month and I made that back on the first bit of advice I got.  It gets my two thumbs up.  CrazyEgg would require minimally an extra seven thumbs to express how awesome it is for visualizing user behavior — see the above heat maps.

If you found this post interesting, you may like my other posts about web design, conversion optimization, and the like.  (I don’t mean design in the sense of pretty things: I have all the artistic skill of a blind mole rat.)  Also, try searching the blog for “conversion rate” and the like, there have probably been a few posts which escaped my uneven categorization efforts.

Everybody paying attention to this blog recently knows I have an unhealthy fascination with Lightboxes, particularly my new favorite iBox.  I like them because they work really well at increasing conversions, most recently in my shopping cart.  (Incidentally, guys, I went back and reset the test.  It really is about 100% improved over the old version.)

So this week I removed the old Lightbox lightbox from my home page and replaced it with an iBox.  I haven’t looked in any rigorous fashion whether that improved things or not — I just did it to cut a few kilobytes off all the Javascript people have to download to use my site, and also because I think the iBox looks cooler out of the box.

Then I noticed that the iBox introduces a loading screen, which the old lightbox did as well.  I hate load screens.  When I see them I think “This is me, losing money”.  Because if two seconds to load the shopping cart cost me half of my potential sales then 2 seconds to look at a screenshot must likewise be suboptimal, and MANY people look at the screenshot (something like 20% of the people who view the page, if CrazyEgg is counting right).

Also, unlike Lightbox, iBox can cause you to get dumped direct to image if you click before the script loads.  Bad news, bears, that means a bounce for most of my users.  I got around this by putting onclick=”false” on the relevant links.

Many, many moons ago (in late 2007!) I experimented with adding blue instructory text to the lightboxed image, exhorting people to download the trial.  That was a crashing failure — it looked like a link but if you clicked it, wham, the image vanished.  But I sort of like the general idea, so it is time for a new split test — take a gander at the screenshot behavior here and tell me if you like it.

You could see the old version of the behavior on my homepage but, well, only if you flip heads the first time you load it.  It is basically the stock lightbox effect — brings up an image, click anywhere to dismiss the image.

I’m hoping for a lift in trial downloads, particularly from clicks coming from advertising.  My worry is that the people are going to get stuck in the lightbox and bounce, since it is much harder to dismiss now.  Oh well, that is what testing is for.  Bring on the visitors!

I have a deep, abiding affection for e-junkie, which is my payment processor of choice.  So much so that I’ve been called their Local Sales Rep on the Business of Software forums.  Probably true — have I mentioned they’re the best $5 a month you will EVER spend in your life?  Sorry, had to do some shillin’ like a villain to make up for what I’m about to say.

I’m thinking of not using their shopping cart anymore.  Not because it isn’t an amazing piece of technology which manages to work for thousands of people without writing code, because it is all of that and more.  The lift I got in conversions from using it has made me thousands, which for about 5 minutes of integration work means the hourly effective wage is sweet indeed.  Its just that my web programming abilities are improving these days, and I’m starting to chafe at a few of the necessary restrictions from using a third party solution.

My big issues:

1)  Forcing people who buy a CD to put in their zip code twice — once in the cart, once at the actual checkout screen.  I’m working on the assumption that that probably kills CD conversions, and I’m not sure those conversions come back

2)  It is slow.  Clicking on the cart gives you a loading message for two or three seconds.  I still think it beats the pants off of a pageload when updating a shopping cart, I just want the perceived speed to be “bugs in my teeth fast”.  (This is where people ask me “Well, why do you need a shopping cart at all when you can just have a Buy Now button?” and where I say “I’ve tested both alternatives and the cart makes me oodles of money by simplifying the customer experience.”)

I was partially inspired to do this by my buddy Yakob at Mixed In Key, who a) has something really cool coming down the pipe which if you are reading this blog will interest you intensely and b) has a really sweet shopping cart implementation.  Try it, you’ll like it.  That’s what I want mine to work like.

I think I can repurpose some of the Widget Factory Javascript code to achieve it fairly easily.  The actual transaction will still be handled by e-junkie, but I want the cart constructed locally.  If I can do it fairly cleanly, I’ll open source this so that anybody else can apply it to their sites — otherwise, I’ll just do the usual kludging and split test it against the genuine e-junkie cart on mine.  I’m thinking the lift in conversions is going to be sizeable enough to justify a few hours of fighting with Javascript (once).

Sidenote: My dashboard informs me that I might hit $3,000 in sales this month.  Whee for new records!  Recession, what recession, 73% year over year growth.  (Watch me jinx myself.)  

What’s doing it?  Same old same old:

• AdWords doing well as it always does near secularized holidays (Valentine’s Day bingo is always hot, and while I’m not winning that SERP the people who are thankfully slap AdWords all over the place)
• organic SEO efforts on website paying off (~1,200 visitors from Google on average weekday these days)
• version 2.51 converts pretty decently (subtle changes, big difference in effects, tell you about it later)
• a few tweaks to the website are paying nice dividends

It has been a really long time since I did any testing of webpage elements and I was thinking “Hmm, what am I paying $20 a month for CrazyEgg if I haven’t logged in since summer?”  So I sat down for a moment and thought of something I could add.  Ahh, I know — maybe I’ll try adding another download button after my benefits spiel and before the ginormous 566×545 pixel sample bingo card, whose scroll-inducing enormity comes before the one visual download button in my main body.

Four days later I come back to CrazyEgg and see the result of 4,200 visitors:

86 clicks, or 10.4% of clicks on the page, on that one stinking extra button.  This increases the conversion of my main page by about 20% (some clicks are presumably “stolen” from other ways to reach the same goal).  Since my main page is also my AdWords landing page this started decreasing the amount I was paying for conversions instantly, and of course increasing the number of trials downloaded will hit the bottom line pretty darn directly.  The naive assumption is that 20% more trials is 20% more sales.

Sadly, my main page does not account for 100% of conversions, but my quick back-of-the-envelope calculation is that the main page causes about 40% of trial downloads directly, which implies that it drives something like $800 of revenue a month, so a 20% increase in that is $160 added to the bottom line (plus assorted cost savings from AdWords which I will not get into because the math gets heady).

Putting a button there is kind of jawdroppingly obvious… I’m not sure why I didn’t do it before.  Sure, there is one button visible to the left when you open the page, and the first text link in the content area is to the download (I have previously tested this and highly recommend it, because the first text link in the content area gets a huge portion of user interaction).  But if someone scrolls down to actually read the benefits list, then they’re left high and dry with nowhere to click unless they scroll past the huge image or scroll upwards.  Neither of those activities “flows” well after you’ve just read a benefits list, not nearly as well as “click here” does.

Well, lesson learned. 

I don’t think I’ve said this in a while: I love you, CrazyEgg.

I have the good fortune to be on a fast, fast, fast Japanese connection and as a result don’t typically wait too long to view webpages.  This makes me forget sometimes that the rest of the world doesn’t get nigh-instantaneous page loads, which means I don’t often design with this factor in mind.  However, somebody pointed out that the new sidebar buttons were loading last for them, and as my front page can potentially take 4 seconds to load (longer than a significant portion of visitors stay!) I didn’t want that.

Why is a bit of a long story.  First off, the sidebar is literally the last thing in the HTML body element (with the exception of various bits of Javascript).  This is to push up the main content area in the document, because search engines treat stuff near the top as more important and I want pages to rank for their own content not the content of pages they happen to link to on the sidebar.  This means that, if you parse the webpage starting at the top and going down, you come across those IMG tags pretty much dead last after everything else on the page.

Now, how do browsers download assets (images, Javascript, CSS, and whatnot)?  Basically, there is a FIFO (first-in, first-out) queue maintained per host name.  For example, if I have the web page reference 10 images on www.bingocardcreator.com, then those are listed from first occuring to last occuring in the queue.  The browser then, following HTTP specifications, starts downloading two at a time per queue.  This means that if you have, say, 25 objects to grab from your own domain and 1 offsite Javascript the offsite Javascript will start downloading about as soon as its discovered while the last object from your domain waits in line.

So that is the technical explanation.  What was happening was my front page loads 2 fairly sizeable screenshots totaling 200kb of the total 350kb required to load the page, and as these are both ABOVE the sidebar when you’re reading HTML linearly, they were blocking the download of the buttons.  That is certifiably ungood, particularly as one of the screenshots isn’t even visible when you open the page (it is far below the fold, most people won’t even see it!)

However, since the queue is maintained per hostname, if I could only host on a hostname other than www.bingocardcreator.com, I could have things download in parallel.  Happily, my webserver (Nginx) makes it really simple to set up extra names like, say, images1.kalzumeus.com which are essentially the same as the main domain in every way except they are, well, named differently.  Thus tricking your browser to download from them in parallel.  I set up 4 domains this way and used that to do a bit of manual queue optimization to ensure that the images with the highest payoff (big beautiful buttons!) load fastest. 

And to think I was wondering why so few folks tracked in CrazyEgg were hitting the images compared to before.  Now I know — they were waiting so long they had already found a text link before engaging the image.  The results are visibly different even on my monster Japanese connection.  (Its like the Godzilla of latency… in a good way?)

Rails offers a way to do this automatically, but I wouldn’t recommend it if you have only a few pages to hand optimize.  The reason is simple — randomizing access to domains results in good average case load orders but if you can hand-optimize things you blow it out of the water.  (For example, there is about a 40% chance that the large image would block one of my conversion buttons with random host names.)

In my continuing desire to split test the heck out of everything, and my continuing total inability to graphically design to save my life, I went back to the LogoSamurai folks and got some buttons designed for the website as drop-in replacements of my current ones.  I’m also getting some made by my web designer Gursimran, and will be testing the two against each other (and, of course, against the current set) and keeping the ones which are most successful at driving the conversions. 

Gursimran isn’t done with her set yet, but since the Logo Samurai guys are, I thought I’d give you a sneak peek.

As you can see, these are much more Web 2.0 than most web sites in my niche, and quite different from the previous look&feel for the Bingo Card Creator site.  That’s why we test them, of course — I don’t want my opinion on what my customers want to calcify and blind me to the actual facts of what they act upon.  Gursimran has a very different visual style so I’m intensely interested in seeing how things work out.  (Sidenote: I love the pencil!)

Cost for these, incidentally: $60, $15 per button.  I had to convince Gursimran to take money for the buttons, as she wanted to just throw them in with the price I already paid for the web design.  As great a deal that is, I would feel really terrible imposing on her like that — professionals should not be afraid to charge money for services, in my opinion.

Plus, if the winning set of button increases my trial conversion rate by 1% and/or my purchase conversion rate by 1%, that would work out to be quite a bit of money.  I also like keeping all of my freelancers happy with me, as I feel it will tend to get me work done in a timely and high quality manner, and thus prevent me from having to go play Russian Roulette with the freelancer sites to find a replacement source of talent.