I’ve been around long enough–and seen enough disastrous computer crashes–that I have a pretty careful backup strategy. I’ve been the nagging voice of nagginess when my friends have lost data due to hardware failures.

“You should ALWAYS make backups of your data. You never know when you computer will take a permanent siesta.”

I commit all of my source code to a subversion repository on my local machine, and I regularly take snapshots of that development machine, which I keep on an external hard drive. In case of a failure I’d suffer two or three days of lost data. A week, tops.

Lately, my computer’s primary hard drive has been making funny noises, like a tiny helicopter flying into a narrow fjord, deep within my laptop. So I’ve been extra diligent about backups.

On Friday, it finally died.

During the boot-up process, it went into a ten-minute chkdisk loop before ultimately succumbing to a horrible BSOD.

After that… nada.

No problem. I bought a new drive and prepared to transfer everything over. After installing the operating system onto the new drive, I connected the backup drive (in its own external USB enclosure) and listened, horrified, as it made a few pathetic clicks and whirring noises.

Dead.

Huh. That’s weird.

This drive had been chugging along without complaint just a few hours beforehand, and suddenly, it’s a complete brick. The operating system doesn’t recognize it at all. It may as well be a can of sardines.

I fiddled with the connections, finally taking apart the enclosure, swapping the cables. Still nothing. The drive is definitely kaput.

Keep in mind, this is not just my source code. This is my entire life. Software projects, art projects, letters, stories, email archives, proposals, invoices, tax documents, from the last seven or eight years.

I felt like my house had burned down.

Panicked now, I connected the old laptop disk, to see what I could salvage.

As it turns out, there was still a lot there.

Even though the operating system wouldn’t boot, and some entire directory hierarchies had vanished in a puff of corruption, I was able to salvage nearly all of my personal documents (in fact, as far as I can tell, all of my photos and documents survived).

But the source code is riddled with holes. Entire projects are just completely gone. (And about a third of my MP3 collection. The horror!)

The application analytics project–which I’ve been working on since June–is particularly decimated. From a total of probably 30,000 lines of code, I’d say about 25% has vanished. The entire server project is nowhere to be found. (The entire embeddable library project and about 90% of the reporting client GUI survived.)

I was able to recover an old, crusty snapshot of the server code (from September) by manually slogging around in the raw subversion data files.

Also completely gone: a set of ActionScript support libraries, with data structures and algorithms for the charts & graphs in the reporting client. Luckily, at the end of December, I started the process of moving my source code–starting with this particular library–to a repository on a remotely hosted (and independently managed) server.

Thank God for small miracles, right? At least if I could recover that code, I’d save myself the trouble of reimplementing about three or four thousand lines of code.

So I tried doing a check-out, but guess what!

There’s nothing there.

Browsing remotely with Tortoise SVN, the server happily responds that the repository is empty. And the web SVN interface says that I don’t have access to the repository. Which is odd… since it also lists a record of my most recent check-ins.

Wouldn’t it be funny if that repository had somehow evaporated?

Is someone playing a joke on me? Am I being punk’d?

I’m going to send my (two!!) failed hard drives to a data recovery company tomorrow. Maybe they can do better than me at recovering some of this lost files. Yikes.

Anybody want to make a small wager that the data recovery facility will burn to the ground the moment my FedEx package arrives?

The wooden doors of the saloon have long since fallen off their hinges.

A sun-bleached steer skull gleams under the hot desert sky.

And tumbleweed creeps across the road, bumping against the old hitching post.

It’s been a ghost town around here lately, since I posted my last message in June, and the lost signs of life have not gone unnoticed a few of you out there.

Three or four times a month, I get an email from someone curious about my Desktop Analytics project, who maybe saw the GUI mockup, and wondered whether I was still around and still actively working on an implementation.

As it turns out, I’m still here. And I’m still working hard.

In fact, the ongoing emails from interested bystanders have really kept me motivated and have reaffirmed my confidence in the strength of the concept. As my way of saying thank you to the people who have sent me their questions, I’ll take a little time tonight to give you a long-overdue status report. Then, sometime within the next few days, I’ll post a sneak peak into the architecture I’m developing and some of the cool features that I hope to unleash on my users early next year.

I’m thinking…mid to late Q1…

First, a quick refresher, for those of you who might not have been around when I originally floated this idea.

I’m building an application analytics platform, analogous in many ways to Google Analytics or Omniture. But those solutions focus on analyzing web traffic to answer questions like “which search engine keywords provide me with the most traffic?” and “which of my content resources have the highest click-conversion ratio?” Those types of metrics were invented my the advertising industry for messuring consumer responses to ad campaigns. Long before Google emerged from the labs at Stanford, ad executives asked Nielsen analysts for their conversion numbers.

The web analytics software on the market today still reflects that mentality: the internet is a medium for delivering content. Some of that content attracts a lot of eyeballs. Some of those eyeballs tell their mouse-fingers to clicky on an ad. And some of ad-clickers buy hilarious cat calendars from Amazon.com.

Web analytics is all about maximizing throughput of the content/eyeball/mouse-finger/cat-calendar pipeline.

Developers of applications, rather than publishers of web content or cat calendars, get short shrift in the whole equation. Sure, we can get detailed statistics about which types of web surfers are more likely to download our products. But when the last byte finishes downloading, our visibility into that user’s behavior evaporates.

I want an analytics package that tells me how users actually interact with my software.

How many people are using that new spellcheck feature I spent so much time developing? How often do my users fire up the application? Every day? Once a week? Do they use the software all day long, or just for a few minutes in the morning? How many menu clicks does it take to find certain features? Are people getting lost in my user interface? Do they invoke the HELP system? Do the “Premium Edition” users actually use any of the premium features, or do their behavior patterns look just like the “Free Edition” users? How many people are still using a legacy version, and which features should I improve, to motivate an upgrade decision among those users?

Answering questions like that will help me develop better software.

And I like to think that “better software” is almost as important as “good keywords” in driving new revenue.

Anyhoo…I hope I’ve gotten you a little sweaty and breathless…

Now: status report. Tomorrow: architecture details.

The software comes in three pieces: an embeddable library, a server, and a reporting client.

The embeddable library is a tiny piece of code that developers will link into their own “host” applications. The code is written in a snazzy little programming language called D, which compiles to native machine code, providing C linkage on Windows and on x86 Linux and OSX. No matter what language or platform you’ve used to write your application, as long as it provides some mechanism for linking with C libraries, you’ll be able to embed the analytics library in your code.

As of today, the Windows version of the embeddable library is about 95% complete (the OSX and Linux versions are about 75% done).

The server is implemented in Java, so you can run it pretty much anywhere (I personally recommend a “Mr Coffee” deployment environment). The server is about 85% finished, and can handle about 100 requests per second on my dinky little 2-year old laptop. It doesn’t use Tomcat or JBoss, and it doesn’t depend on a gazillion application frameworks. It’s a standalone HTTP server, talking to a MySQL backend, and you can get it up and running within minutes (hint: no XML configuration).

Finally, there’s the reporting client, with all the sexy charts and graphs. I’m only about 10% done here, after making a bold decision to switch my development environment. I started out working in C# with WPF/XAML, but I recently switched to Flash/Flex/AIR (for reasons that I’ll describe in more detail later) and now I’m playing catch-up.

That’s all for now :o)

I’ll be back soon with more details. But right now, I just wanted to check in with you guys, let you know that the project is still rolling forward, and that I’m anticipating a Q109 release.