Business Intelligence for Desktop Software
First of all, let’s start by assuming that you love hilarious tee-shirts.
Something, perhaps, to express your knowledge of contemporary video-game pop culture, while simultaneously making an ironic commentary about being a complete social misfit. Not just a run-of-the-mill computer nerd. But maybe a former a member of an actual high school band. Brass section.
Maybe something like this…
Oh yeah. That’s the stuff.
To buy that shirt, you’ll need to surf on over to the Torso Pants website, where you’ll find a truckload of other pithy outerwear. No matter what ironic outings you’ve planned with your friends this summer, you’ll find a breezy 100% cotton tee with the perfect one-liner somewhere on TorsoPants.com.
While you’re there, open up the HTML source in your favorite text editor and scroll waaaaay down to the bottom. There, you’ll notice a SCRIPT tag, linking to code from a google.com domain. And if you’ve been involved in web development – in even the slightest capacity – over the last two or three years, you’ll immediately recognize that ubiquitous little snippet of JavaScript as belonging to the embeddable Google Analytics library.
Anyone with a GA account can paste those two lines of JavaScript into her website’s design template and immediately start collecting a broad array of metrics about the site’s users, their browser environments, and how they interact with the website. Most importantly, the marketing department can see where their incoming links are coming from, which types of users tend to buy (or not buy) which types of products, and which advertising campaigns are most efficient at converting which eyeballs into dollar bills.
The Torso Pants people can actually quantify the hilariousity of their t-shirts by comparing the popularity of the product pages, even among people who don’t buy! For my money, Tuba Hero is at least 35% more hilarious than Homeschool Valedictorian.
Right about now, given the vast scope of my readership, I assume the guys from the TorsoPants marketing department are looking at a blip on those charts, wondering what the hell is going on. The abnormal rate of traffic from this page will surely show up on their dashboard, and they’ll probably click on over here to investigate. (Hi guys!! I’m sure you don’t mind my borrowing one of your images, cuz of all the free advertising I’m sending over? Right?)
I picked the Torso Pants website for my little story to illustrate the fact that pretty much every company (small or large) doing business on the internet is currently using business intelligence, usually in the form of Google Analytics, to optimize every aspect of their online presence. Torso Pants is just one example out of millions of other small businesses with the same snippet of JavaScript on their pages. The few companies not using GA have — almost without fail — implemented the more sophisticated (and enterprisey (and therefore more complex (and significantly more expensive))) solutions from Omniture or WebTrends.
But the idea is the same.
From the placement of links to the wording of ad copy, every element of the interaction design can be studied and optimized using a web analytics platform, performing A/B testing about the effectiveness of different ads, different landing pages, different offers, different prices, and so on and so forth.
In fact, the existence of these analytic tools on the web provides one of the most substantial positive arguments for developing a new software application as a webapp, rather than on a desktop platform. Even if the web development process is more cumbersome. Even if it means additional IT staff to deploy and manage the online app. And even if the browser environment degrades the user experience.
Releasing your product on the web means you can easily incorporate a sophisticated statistics package, by just pasting two lines of code into a single template file.
It’s hard to argue with that.
But what if there was an analytics package for desktop applications? What if the integration process was as simple as linking to a library and pasting two lines of code into your own project? How would that change the equation? And why is the competitive landscape so crowded in the web analytics marketplace be so barren in the desktop analytics space?
I’ve spent the last few months thinking about just such a platform, wondering what kinds of things you could learn about your users. How could you use that intelligence to design and build better software? Here are a few of the ideas I came up with:
- First of all, how many people are actually using your application? You might know how many people downloaded it, but do you know how often they launch it? Do they use it every day? Maybe only once a week, or once a month? Is it used mostly Monday through Friday, during the day? Or primarily on evenings and weekends, implying the difference between corporate purchasing and leisure and hobby users? After using the application for a week or two, do most people uninstall it, or do they start using it even more?
- Do your users have shiny new, high-powered machines running Vista with 4 gigs of RAM, or are they clunking along in Windows 98 on a Pentium II? Do they have Java? What version? Do they use multiple high-resolution monitors, or do they run mostly on laptops? You probably have more downloads from Windows users than from Mac users, but which population of downloaders is more likely to become a group of loyal users?
- How long does it take your trial users to finally buy a license? If you offer a 30-day trial, do people use the software during the full trial period, or do they launch your software only once? Maybe those people who uninstall at the end of the trial would buy your software if the price was lower. Users who abandon the software long before the end of the trial probably used the software for one task and satisfied their need. Maybe you should offer a shorter trial. Or maybe you should offer a money-back guarantee and get rid of the trial altogether.
- How many people are using pirated serial numbers? Do some of your licensed users install on both a desktop and laptop computer? Those users provide an excellent way of determining which features are more well-suited to each platform, since you can observe the same person’s behavior in two different contexts.
- Maybe you offer a “Pro” or “Enterprise” edition, with a few extra features for a few extra dollars. Do people actually use those premium features? How often? Which features are the most valuable? If people don’t actually use those features, you might need to redesign the distribution of features among your product editions, to take best advantage of your pricing segmentation opportunities.
- Do people get lost in the user interface? How often do they browse through the main menu before clicking on a menu item (indicating they’re having a hard time finding features)? When invoking commands, do people prefer keyboard shortcuts, context menus, toolbar buttons, or the main menu? Maybe you just implemented the vaunted “Office Ribbon” interface. Do people switch back and forth between ribbon panels without choosing an option (again, because they can’t find the right button)?
- When you release a new version of your software, how long does it take your users to upgrade? Do some of your high-volume users resist upgrades, even across major versions? Maybe the features in the new version didn’t capture the interest of those long-time users. That’s a big problem, because in most businesses, your existing customer base is the richest source of new sales.
- Do your users actually read the Help docs? Is it because they understand your software really well? Or is it because they don’t know how to find the Help system? Do they read the docs immediately after installation (maybe because of a confusing interface) or only after a few weeks or months of usage (maybe because the interface is easy to use at first, but the advanced features are hard to understand)? When people do read the docs, what topics tend to be most popular?
- What if you could run a snippet of profiling code on every user computer, to get a baseline performance benchmark for all of your users? What if you could get stack-traces for exceptions thrown on remote machines? What if you could see which devices, platforms, and configurations contribute to the most errors? Should you spend a lot of time fixing those errors, or should you just tell people that Windows 95 is not supported?
Those are the types of questions I hope to answer with my software. And so that’s the focus of my project for the month of June.
Of course, the product I’m describing here is more than a 30-day project. But I don’t intend to stop at the end of the month. In this article, I’m defining the long term vision, and over the course of this month (and, realistically, probably part of the next month) to restrict the scope to something a little more bite-sized for version 1.0.
I hope that sounds as exciting to you guys as it does to me. I’m completely psyched for this project.
At the moment, I’m about 60% finished designing the database schema. I’ve written a few bits and pieces of code (Java and .NET) for the embeddable libraries. And I’ve spent a lot of time (altogether, about a week) working on the user interface design sketches.
While the primary goal of the project is to provide a truckload of business intelligence about desktop software deployments, I also want to provide the most gorgeous, detailed charts ever seen in a BI platform. I’ve been reading a lot of Edward Tufte lately, and I’ve been absolutely slaving over the GUI. So far, I’m really happy with how it’s going.
But more about that later.
I’ll be back within the next day or two with more details. And pictures!
In the mean time, let me know if this is the type of thing you’d be interested in. Based on the bullet points above, are those the kinds of stats that would help you in the development and marketing of your software? What other kinds of stats would you like to collect? (Incidentally, my personal checklist is much longer than what I included above; I trimmed it substantially for publication.)
If this sounds like it’d provide value to your company, let me know some details about your development platform and process. (Either comment below or send an email to benji@benjismith.net.)
What platform do you develop on? Java? .Net? Mac? Would you rather run the server component yourself, or would you prefer a hosted solution (knowing that the cost of the standalone version would probably be about the same as a year of the hosted subscription version)? If you’d rather host it yourself, what does your server environment look like? Windows or Linux? Do you run your own servers? Are you on a VPS? Do you (god forbid) use a shared-hosting provider? Depending on the size of your user population, it might be necessary to get dedicated hardware for a server like this. Would you be okay with that?
I’ll concentrate on producing embeddable libraries and server components that are compatible with the deployment requirements of the biggest segment of prospective customers. So if you’re even remotely interested, let me know!
Holy crap! I’m currently in the process of designing and building a product in 30 days. A whole bunch of other software developers are doing the same thing, at the same time, during June 2008. Everybody’s blogging about the process, and all of our RSS feeds are being aggregated into a single incredible feed. You can follow along here.
June 3rd, 2008 at 3:02 pm
I think this is a really sharp idea. The big battle for acceptance, though, will be over user privacy. When I use a piece of packaged software, I have the implicit expectation that data will not leave my computer without my permission — that my software won’t be spying on me. Even for crash reporting, which is finally becoming ubiquitous, the user is prompted for permission, and I would bet a sizable percentage say no. This privacy expectation is greatly damped with a hosted service.
Good luck! I’ll be watching to see how it goes…
June 3rd, 2008 at 5:40 pm
Benji,
Greetings from a fellow 30 dayer.
I enjoyed your post, especially the advantages of gathering BI from a Web app and your goal of Tufte-influenced design. Have you researched what, if anything, is already available in the market and how proactive/successful the big players have been with in-house solutions?
I am a big Tufte fan also but then so is Google (eg Analytics) and the rest of the world. Even Microsoft is acutely aware of Tufte’s influence though they are probably loathe to admit it.
There are many aspects of my calendar application (http://calendar.schemax.com) that implemented ideas from Tufte’s books and discussion forum (small multiples, chart junk, sparklines etc). In fact, the whole Schemax Calendar interface was designed to remove the heavy grid that plagues the Outlook calendar interface even in Outlook 2007.
One area of BI that calls out for attention is the time scale used in reporting applications. I have tried to create a dynamic time scale that adjusts as the user zooms in or out of the calendar (my report type). I’d be interested in your thoughts on this.
June 4th, 2008 at 2:55 am
I agree with Jared as it was my immediate concern also. This type of ET (phone home) is the sort of stuff M$ gets bagged about continually. There are probably ways around this from a sales perspective by getting consumers to agree with the concept (by discounts etc).
However, that doesn’t stop this being a good tool for deploying in closed shops such as companies that do their dev in house. I reckon usability of an an application that employees are forced to use would allow feedback on software enhancement.
Lastly, nice idea.
June 4th, 2008 at 3:48 am
I agree with the comments above.
The idea looks great at the first glance, but how it was discussed previously, you have to convince me that your application isn’t spying what I’m really doing…
Some applications already send information through the internet. Think on Microsoft Office. After you install it, it will ask you if you want to send anonymous information to Microsoft.
I’m not a suspicious person, but as Jered Floyd said “I have the implicit expectation that data will not leave my computer without my permission”.
Cheers
June 4th, 2008 at 3:50 pm
Seems like the embeddable library ought to be written in C/C++ (.dll on Windows, .so on Mac) with a “native” API for those languages. That would allow it the access it seems it would need to the underlying OS APIs to gather the data you’ve described; and a lot of desktop apps are written in those languages still. Java JNI and C#/.net libraries could sit on top of that base library for integration into desktop apps written for those platforms.
My $.02 …
June 5th, 2008 at 4:24 pm
Hey, everybody, thanks for your comments!
Lots of concerns about privacy. I totally agree. If there’s one achilles heel in this idea, it’s either directly or indirectly a privacy concern.
I’m working on a new blog post all about privacy, but it’ll be another day or two before I post it (it’s hard keeping up with blogging while also trying to write a truckload of new code).
June 5th, 2008 at 4:25 pm
@Simon Shutter Says:
“Have you researched what, if anything, is already available in the market and how proactive/successful the big players have been with in-house solutions?”
There seems to be only one competing product on the market:
http://www.preemptive.com/runtime-intelligence-services.html
http://www.preemptive.com/images/documentation/runtime%20intelligence.pdf
It was launched just a few months ago, and so far the public details are very very sparse. The PDF has a few tiny screenshots that seem to confirm that the software is basically a “web analytics for desktop apps” platform, but there’s very little discussion of the actual set of features. There’s no list of the actual runtime behavior tracked by the platform.
I couldn’t find any other direct competitors.
As for in-house solutions… I’ve been trying to track down all the common desktop applications that collect anonymous usage statistics, but can’t find a comprehensive source of data. In the last few weeks, I’ve noticed WinAmp, Visual Studio, and Google Toolbar all asking for permission to collect usage statistics. The Mozilla products (FF, TB, etc) will ask permission to send crash reports. And, of course, I suspect *any* application that connects to the internet as part of its core functionality (RSS readers, media players, etc) phones home to report behavioral activity at some level.
A future article will talk more about the competetive landscape, both in the expectations set by in-house products (permission for collecting WinAmp stats, for example) as well as the competetive landscape of full-blown analytic platforms for the desktop (the preemptive “Runtime Intelligence Services” product).
@Simon Shutter Says:
“I am a big Tufte fan also but then so is Google (eg Analytics) and the rest of the world. Even Microsoft is acutely aware of Tufte’s influence though they are probably loathe to admit it.”
Right now, my biggest Tufte influences are:
* Eliminate “computer administrative debris” — I’m trying to design my GUI entirely without menus, toolbars, status bars, modal dialogues, wizars. The charts are the central element of the GUI, and I want user to be able to interact directly with those charts, rather than indirectly, through a cluttery mess of GUI widgets.
* Show data at “image resulution” rather than “cartoon resolution” — To me, the google analytics charts are extremely cartoony, with only a few dozen data points displayed at once. Tufte talks about showing thousands of data points all at once, using data layouts optimized for harnessing the image-processing computing power of the human visual cortex. People’s eyes can spot trends when you give them enough data, and when you eliminate enough of the surrounding clutter.
I’m planning a big GUI-related aricle sometime soon. (I’ve already produced a detailed concept drawing, and written some prototype GUI code; I just haven’t written an article yet.)
Oh, and Simon, I just watched your Schemax Calendar demo movie, and I think we’re on exactly the same page. I really like the GUI concepts you use, and I think you’ve done a great job in getting a lot of detailed data onto a single screen in a format that’s easily processed at-a-glance by your users. Nice work.
@Simon Shutter Says:
“One area of BI that calls out for attention is the time scale used in reporting applications. I have tried to create a dynamic time scale that adjusts as the user zooms in or out of the calendar (my report type). I’d be interested in your thoughts on this.”
Yep. I have some design sketches of the same concept.
June 5th, 2008 at 4:57 pm
@Donnie Hale Says:
“Seems like the embeddable library ought to be written in C/C++ (.dll on Windows, .so on Mac) with a “native” API for those languages.”
I totally agree.
At the very least, the environment-detection stuff will demand a native-code implementation, to ensure consistent results.
Technically, I could use various tricks to get environment details from within Java or .Net, but it’s hard to get consistent results (for example, the values from Java’s System.getenv() or System.getProperties() method calls are sometimes different than the values in .NET’s Environment.OSVersion property).
For a single, canonical source of those values, I think it’d be best to either get all values from the registry (or, more likely, through win32 API calls).
So I’ll almost certainly build an environment-detection DLL and then build Java and .Net libraries on top of that DLL.
June 5th, 2008 at 10:18 pm
Thanks for the feedback on Schemax Calendar. A lot of the SVG/javascript that drives the GUI was actually adapted from a GIS implementation (see carto.net).
RE: Google Analytics charts - If only Google Analytics had an API….
June 16th, 2008 at 3:58 am
I have thought about instrumenting my desktop app. But mainly I am just interested in how many time each function is used. That should be relatively simple, but tedious, to do myself. I’m not convince a third party solution would make it any simpler or less tedious.
I am using C++/Qt on Windows and Mac.
June 16th, 2008 at 10:23 pm
* Privacy is a big concern, but perhaps it would be less of an issue for the customers of uISVs relative to large, faceless corporations.
From personal experience, I know I never allowed any desktop app to collect my app usage data. For example, Winamp used to harass a lot about that. I think they finally took it out or turned it off. For some reason, when this happens on the web, it’s not a big deal. I have no clue why that is so.
* All the hype and hysteria is around webapps lately. Not that noone is developing desktop apps, but since the focus is on webapps, people are focused there mostly. Perhaps that’s the reason why there aren’t many other alternatives combined with the privacy issues.
* Isn’t GA more about figuring out where your traffic is coming from and what your customers are paying attention to in the context of (textual) content? A desktop app doesn’t have incoming traffic or any significant content to monitor. It might be nice to know which features are being used the most and the least, but perhaps it is not that big a deal. OTOH, it might be useful to know what kind of resources the PC where the app is installed has.
June 17th, 2008 at 11:39 am
@anon:
Privacy is a big concern, but perhaps it would be less of an issue for the customers of uISVs relative to large, faceless corporations.
On the web, I think the main reason privacy is a dwindling concern is that users know they’re trading their privacy for more advanced functionality (by browsing with cookies/javascript turned on), and they’ve collectively agreed that it’s a good trade. Developers of desktop applications who want to collect anonymous statistical data will need to offer a similar tangible benefit.
@anon:
For some reason, when this happens on the web, it’s not a big deal. I have no clue why that is so.
I think it’s because there’s no way on the web to enable rich interactivity without simultaneously enabling the collection of anonymous statistical data.
Desktop applications are traditionally disconnected from the internet. But as more and more RIA-style apps appear on the desktop, whose entire functionality depends on web connectivity, users will become less fidgety about anonymous usage stats.
@anon:
Isn’t GA more about figuring out where your traffic is coming from and what your customers are paying attention to in the context of (textual) content?
People use GA differently, depending on their web presence. E-commerce sites are interested in knowing which product pages get the most traffic. AdSense advertisers are most interested in knowing the effectiveness of their campaigns. I personally use GA primarily to measure the growth of my blog audience and to see which articles are most popular (Why I Hate Frameworks: 25,324 page views, compared with 12,388 for Interviewing With Google).
I think people will use this platform for a variety of different business intelligence purposes. Check out the GUI Mockup to see the types of reports I’ll be providing.
June 17th, 2008 at 8:15 pm
What is the business model? How do you plan to charge for this, especially the piece that goes into the desktop client?
June 18th, 2008 at 10:18 am
I have two business models in mind:
1. Hosted, where developers pay for the service each month.
2. Packaged, where developers have to host their own server.
My ideas about pricing are still very nebulous, but I imagine the price of packaged version would be roughly equivalent to a year of hosting. And I’ll probably also charge a fixed monthly fee, plus a sliding-scale fee for usage volume.
But that’s all very fuzzy…
I don’t plan on charging separately for the different bits and pieces. At its very minimum, this software has three separate components: the embeddable library, a server, and a GUI client for viewing reports. You don’t have to buy those separately. You buy the system, and you get all the parts.
June 18th, 2008 at 10:21 am
Oh… I forgot to also mention…
The software architecture is very likely to have more than three parts, especially in subsequent versions.
I can imagine a scenario where the user wants the embedded library to report to a URL on their own domain, but wants the reporting server (and the actual of the data) to be hosted. In that case, the architecture would also include a “server side buffer”, probably a PHP script that would collect data from clients and then periodically send batches to the main server.
June 18th, 2008 at 9:56 pm
Cool.
It will be interesting to watch how this project unfolds…