Kalzumeus Software

“Hmm, I wonder why I’ve gone three days without a sale.  I don’t suppose it could be that my trial version got corrupted again.  *click*  Well, that answers that question.”  Time to have a quick chat with GoDaddy and figure out what the heck is going on on their side, because I’m positive that that file was good an hour after I uploaded it.

Update: See Nick Hebb in the comments for something I didn’t think to check for.  This won’t work.  Ahh well.  I’ll leave it up for posterity.

I guess I shouldn’t be mentioning this but, hey, I’m a giving type of guy. One of the best decisions I made during the creation of Bingo Card Creator was cutting my web site designer out of the budget and going with a design by Sang Nguyen which I found on Open Source Web Design. This is my advice to anyone who is wondering how to cut a major expense item out of their budget while not compromising the customer experience much — I am positively thrilled both with my design and with the amount of work I didn’t have to do to get it running.

There has been a discussion on the Business of Software boards recently about whether paying a web designer is a good idea. Cards on the table time: I think not, unless you are getting a site which has functionality in addition to eye candy, an under-market rate, or a web design which positively sings to you. Here is my trump card for the superiority of oswd over a designer: the designer won’t give you a PR8 link to your site for nothing. I just discovered oswd will do that. Granted, its not a maximally effective PR8 link — it will eventually scroll from the front page of links (faster as more people come to know about it — at the moment it appears to be close to the best kept SEO secret on the Internet ;) ), so it won’t be at one click of depth from the front page (!) for forever. But hey, which part of “free PR8 link” are you complaining about?

… it WILL be your trial executable, without fail.  Murphy’s law.  One freaking byte on disk got corrupted at some point within the last week and I just found out about it today, by someone who was kind enough to write in and say “Heya, this won’t install, says the file is corrupted.  And I downloaded it three times”).  One quick download later and I was able to reproduce the error, which is funny because I haven’t uploaded anything to that directory on the server in literally weeks.  Did a quick hex diff and, boom, one byte out of place, which called NSIS to say “Uh oh, checksums don’t match, I’m out of here”.  *sigh*  Well, better to know about it and be able to fix it than to not know about it, right?

I never touched Unix in my life before going to college, but it was the development environment of choice for most of the upper-level coursework at school (lower level was Windows + Emacs… not a combination I would recommend given the excellent Java IDEs which exist right now, incidentally), and so I’ve gotten very used to having my good friends ls, grep, wc, and gawk available on my machine.  I use Cygwin to have them available on my Windows box, because while Linux is a lovely operating system in many ways I rather enjoy having iTunes and games available.

Anyhow, I had a task yesterday: I have a local copy of my website.  All of the pages have a very similar structure because the navigational elements are embedded in static HTML.  However, this didn’t use an official templating feature because NVU doesn’t support any, which means updating the navigational elements means updating approximately twenty pages by hand.  There was one particular snippet of HTML that I wanted to change to another, globally.

Normally, I would do this in a gawk script (you can do just about anything in a gawk script), but it gets a little messy:

BEGIN {

startTag = “<a href=\”guarantee.htm\”>Money-back Guarantee”

replacementTag = “<a href=\”guarantee.htm\”><img src=\”images/money-back-guarantee.htm\” alt=\”Money-back Guarantee\” \\>”

}

($0 !~ startTag { print $0}

($0 ~ startTag) {

gsub(startTag, replacementTag, $0);

print $0;

} 

Anyhow, that is the way to do it in gawk.  Define your regular expression to search for, print out every line that doesn’t match it and do some surgery on the ones that do.  Note that you’ll have to run this on every file in your web directory, with the easiest way probably being a bash script, and output the results.  However, if you output to the file directly it will generally end up empty for reasons that I have never bothered really understanding, so you probably want to have another quick shell script to output it to a temp file and then copy over the original.  Something like, oh,

for file in *.htm do

  gawk -f myscript $file > temp.htm

  mv temp.htm $file

done

But that is two scripts, and two scripts is too complicated!  Enter sed, a handy little stream processing utility.  Sed is perfect when you just need to do a quick substitution for a regular express:

sed -e “/myregularexpression/myreplacement/”

There, you’re done.  Or in my case, you have to finangle the regular expression into a string, which involves quite a bit of escape characters, but it still only took about 30 seconds.  This saves about half of your typing time versus doing it in gawk, and you can’t accidentally bork the matching logic (something which is easy to do in gawk — forget the ! before the ~, or execute the (string ~ regex) block before the (string !~ regex) block, and you’re screwed.

Of course, you could always write it in Perl.  That would probably take a single line and look like your cat had just tap danced across your keyboard.  But if you want a simple, intuitive way to do fairly powerful data processing, you’ll see sed is your man…  err, I mean, you should see man sed.

… perhaps I haven’t overdefined my target market.  On the one hand, one of the nice things about making Bingo Card Creator pretty much built around the assumption that my user is a teacher is that its very easy to consider possible features and marketing campaigns in that light.  I can make assumptions, like assuming that she (I can assume she is a she!  Sort of a rarity in software development, I would think) is not as comfortable using her computer as I am using mine.  I know how she spends her day (she visits Bingo Card Creator during planning periods, lunch, or when lesson planning at home at about 9:00 pm), I know what her key problems are, I know that she absolutely melts when I say my favorite character in Harry Potter is Hermione (totally true, incidentally), and I more or less speak her language (I know what a manipulative is, for example, to pull a random word out of my search queries for this week).  All in all, I’m in a good place to be with regards to my target market.

But my target market isn’t the whole of people who could use Bingo Cards.  I sell a few copies (impossible to know how many, of course — I can only guess based on customer information provided, email addresses, support requests, that sort of thing) to folks who are not teachers but nonetheless click through 100% teacher-oriented advertising copy to buy anyhow.  I have had repeated purchases from folks organizing corporate events, where my icebreaker bingo idea is a natural fit.  And today I got, lets see if I can anonymize this a little… OK, pretend there is a National Association of Skateboard Punks.   There isn’t, to my knowledge (apologizes to the NASP if there is), but a customer about as non-elementary-schoolteacher as NASP wanted Bingo Card Creator for some advertising needs.  Now, I had certainly been of the opinion BCC was a one-use program (it makes bingo cards, and does nothing else, by design), but I know of folks who are currently using it to make worksheets, track patient progress in rehabilitation, make customized placemats (print the card, mount it, laminate — apparently crafty-types use computers now), and now advertise SkatePunkz2k6, the annual get together for the discerning skate punk.

I am considering, after my project calms down at work and my life returns to some semblance of normal, retooling the website a bit to accomodate these multiple user groups.  Minimally, I would like to have one “neutral” presentation which doesn’t assume teacher-hood for the program, and probably one pitch for corporate-types planning for the annual retreat (Synergize your best-of-breed team-building paradigm with Bingo Card Creator and achieve six sigmas of internal customer empowerment!).   Hopefully it will lead to a bit of natural market growth without losing my main focus on the educational market.

(Well, I suppose that technically all of my sales are international, since I’ve never sold a copy in Japan.  Be that as it may, the business is headquartered in the US even if I’m not, or at least thats my story and I’m sticking to it when it comes to taxes and banking matters.)

Bingo is popular throughout the English speaking world (and in some points beyond, too), so its not too shocking that I have a few customers from outside the US.  Probably, oh, off the top of my head 10% or so.  I was unsatisfied with my previous level of service for them, chiefly because I had the prices listed only in dollars and this both confused the Australians (hey, I could just have easily have been an Aussie uISV, and there is nothing in my website to tell you differently — doh) and costed all of my customers currency exchange fees when dealing with Paypal.  I hate currency exchange fees with a burning passion in my soul, having been burned by them plenty (Bank of America will happily ding you for 3% a transaction).  So I decided to price my product natively.  There are a couple of issues here.

VAT: Ahh, I love the smell of overly-complicated growth-killing Eurocrat mandated regulations in the morning.  VAT, for value added tax, is a freaking gigantic consumption-esque tax you have to worry about if you sell in Europe.  My position on VAT is simple: I’m a small businessman, I have never set foot in Europe, and I do not want to deal with the hassle of collecting this tax.  I give my customers one method (using eSellerate), which has them pay the tax (something like 20% of the purchase price!) and one method (Paypal) which doesn’t.  They can worry about their own local laws.  Incidentally, with respect to my local laws, I’m scrupulous about not actively selling in Japan (because I don’t have the legal structure required to remit Japanese sales taxes yet — thats 5%, incidentally), and I will pay sales tax on all of my orders in Illinois.  I don’t know if Illinois will be very happy that I don’t ask for people’s address unless they ask for a CD, but I’m highly unlikely to pass the minimum dollar amount for tax reporting anyhow.

What to charge folks: I got out Google’s currency converter, plugged in $24.95 in the various currencies, rounded to a convinient number in whatever direction was required, and then chopped off 5/100 of the local currency.  My guiding principle was to be mostly fair without sacrificing the convinience of round numbers, which as I recall results in the Brits paying a wee bit more and the Aussies paying a wee bit less relative to the USD than other customers.  However, I’m pretty sure all of my international customers benefit in that I absorb their currency conversion costs through Paypal rather than them absorbing it through their banks.  (This is a matter for me of a few dimes worth of profit per sale, less than nothing really, but its a small thing you can do to really endear yourself to a fraction of your customers.  I’m all about that.)

I’m thinking of officially incorporating myself next year, but there are some non-trivial issues with incorporating in Japan (“Proof of 3 million yen worth of business capital…  hmm, well, can I count all the money I spent on college as human capital?  Because thats the only way I’ve had nearly $30,000 at any point in my life.”)

I am not an accomplished Perl hacker and something has been frustrating me at work for the last several weeks.  If you know of an easy way to solve this, I will be forever in your debt:

We are using Perl to run a HTTP server (HTTP::Daemon, incidentally, works like a charm) to receive information from a rich client application.  The client has to upload text files via HTTP post.  Everything is A-OK on the client’s side of things, but I have no clue how to go about extracting the uploaded text file from the HTTP request on the server side of things.  My experiments have been a bunch of failures, I can’t seem to find any CPAN Module which will do this cleanly, and there are other deliverables up in the air which are getting held up by the server being only 98% complete (that last pesky 2% is critical to testing the client application, sadly).  Does anyone have any suggestions?  Perl’s documentation is, well…  up to its usual standard of lucidity.

Steph of LandLordMax fame recently posted about how to deal with email spam.  This just happens to be the focus of my day job (we’re busy exploiting the synergies of a new paradigm of… actually, no, I don’t work in Dilbert’s office).  I use spam filtering religiously on all my email accounts (4 at last count) and it saves me roughly 2,000 spam a day (my business email address is forwarded to by my college email address, which was widely circulated back when I was young and stupid).

Here is everything you need to know about spam prevention in two sentences sentence: If you are comfortable administering your own email server (and already do), use SpamAssassin.  If you are not, use Popfile.  There, you’re done and didn’t even have to pay me my company’s consulting rate.

SpamAssassin takes a multi-tiered approach to dealing with spam: it has rules written by the administrator (or, if you’re a good outsourcer like any uISV should be, you just download them from the Rules du Jour website) which are handwritten regular expressions which look for spam signs, it has RBL checking, and it has a Bayes statistical engine.  Popfile just does Bayes.  You can find descriptions of exactly how Bayes works elsewhere on the Internet (Paul Graham was one of the early pioneers of the idea, incidentally: see his seminal article A Plan for Spam, with the caveat that the field has improved significantly since then and that you should consider this more of a historical piece than a tech blueprint).

Both SpamAssassin and Popfile, when trained properly, will produce well over 99% accuracy (both in not improperly labeling good mail as spam, and in not missing spam).  Exactly how you do the training depends on your installation.  If you are a mail administrator, start reading about SpamAssassin and understand you will be spending much of your free time feeding the monster.  If you aren’t, can I say for the second time, Popfile will save your life.

Popfile works as an email proxy, which means instead of having your email client (Outlook, Thunderbird, whatever) dial up your server when it wants mail, you have it dial up a port on your local computer instead.  There POPfile says “Hold on chap, just one second” and runs to fetch your mail from the actual server.  Popfile then scans the mail, uses statistical witchcraft to determine whether it is spammy or not, and marks it appropriately (via one or both of adding another message header to it or changing the subject line to read “[spam] Do 1+ 4 yur l0v3d 0n3″).  You then either just scan those subjects yourself or, if you’re smart, configure your mail client to automatically move everything tagged as spam into a special folder, which you periodically run through (look for missclassified emails and then purge).

Classifying emails is a bit of a nuisance: you have to visit a special web page (on your local machine), which you can do with a click of an icon in your system tray.  Then you have to locate the proper email (easy with a search-by-subject or sender feature) and click the appropriate classification (Popfile calls it “bucket”) and click reclassify. 

POPfile isn’t just for spam versus ham — you can have as many buckets as you want.  For example, I use Thunderbird at home to download from both my ISP email account and my bingocardcreator.com account(s).  I have POPfile configured to automatically tag any emails which are autogenerated (Amazon sales receipts, bounce mails, subscription confirmations, and the like), from family, regarding Bingo Card Creator, spam, from 2 particular mailing lists, and the like.  After you classify a shockingly small number of emails (generally less than a day’s worth in my experience, your mileage WILL vary), POPfile gets the hang of things very quickly: it “knows” that a genuine message from Paypal is invariably about Bingo Card Creator (despite it not saying “bingo” anywhere) and it “knows” that a fake message from Paypal (which it is very good at sniffing out) is spam.  My accuracy hovers in the 99.5% range.

The main problem with ANY anti-spam system is dropping legitimate emails.  I lost 3 in the last week, although two were identical (my friend hit the send button twice, apparently).  On a scale of criticality from 1-10 the email that was duplicated was like a 9.2, which was unfortunate (luckily I noticed it in a periodic scan of my spam folder).  The other one was a subscription confirmation (no big deal, since I was expecting it a search found it instantly).  For folks you absolutely want to get mails from, you can set up “magnets”, which are basically hand-written rules which override the statistics.  For example, I’ve got a magnet for my mother — anything she sends goes into family.  I’ve had another magnet for any email which quotes one of my own receipts, because that is what most of my customer support emails look like.  I stopped using that magnet because after the first 3 times that sort of mail happened POPfile was on it like white on rice.

OK, enough plugging POPFile (same story as always: opinions are my true feelings and totally my own, and nobody is sliding me money under the table from them, although you can reasonably expect that since my day job is directly related to this I am not an unbiased observer).

 Steph mentioned another method in his post: challenge/response.  Basically, if someone who you don’t know sends you an email, you send them an automated message saying “Hey, prove you’re a human by responding to this CAPCHA and I will deliver your message”.  DO NOT USE THESE.  The potential problems are numerous and they get even more numerous the more people who use them — and, since if you’re reading this you are a business, you should know that end-users HATE THEM with a burning passion.