Back in the late nineties, I had money in the stock market.

Big surprise. Everyone had money in the market.

I subscribed to a daily feed of end-of-data pricing data (open, high, low, close, and volume for about 9,000 securities), and every evening, I’d look through about a thousand charts looking for very a few specific patterns. I kept a list of favorable charts each day, and I’d buy a few hundred shares of stock in one (maybe two) of the securities that really impressed me. For each position, I’d set an upper and lower limit. If the stock price crossed either of these thresholds, I’d sell, usually within three or four days.

I had a pretty good system going, and it was yielding double-digit percentage returns every month.

When the market took its downturn in the spring of 2000, I lost most of my gains. But I learned a lot in the process.

One of the important concepts I learned is that the relevant patterns can be difficult to find. Just looking at a sequence of closing prices won’t necessarily tell you much about the character of a stock. But, if you construct a five-day moving average and a ten-day moving average of the closing price and plot them together on the same graph, patterns that weren’t obvious before can start to jump from the page.

Besides just moving averages, securities can be evaluated by a veritable panoply of different formulae: standard deviations, stochastics, bollinger bands, velocity, MACD, relative strength, and on and on and on. By examining these different kinds of charts, a technical analyst can sometimes identify opportunities to buy or sell stock.

Technical Charting Meets Web Analytics

Fast forward a few years.

In early 2004, I started hosting a website. It was a discussion forum with myself and a few of my friends.

And then their friends.

And then more and more people until we had over 1100 members on four different continents, with between 30,000 and 50,000 page views per day.

Pretty early on, I started keeping a database log of every page view and every new user-contributed article. I cobbled together a few dozen SQL statements and regularly ran a batch process to query my statistics and produce a series of charts. Immediately, I was struck by the similarity between a chart of website usage data and those old stock market price charts.

Here’s a chart from my website during that time period, representing about six months of data:

The blue line represents the number of new messages submitted to the site by users each day. The magenta line is a seven-day moving average of the blue line. I used a seven-day period for the moving average as a way of smoothing the curve, to normalize out any variations in the data caused by different day-of-week.

Just like the stock market charts, the moving average smooths the curve, showing patterns that aren’t as obvious in the un-smoothed line. In this particular chart, I was surprised to discover an almost perfect 60-day repeating usage cycle. As a user of the website as well as the webmaster, I knew that the community activity level was cyclic, but I didn’t realize just how cyclic it was until I stared at the chart.

The Market for Web Analytics

At about the same time, one of my close friends got a job at Omniture, as the director of their Professional Services department. I was already familiar with Omniture as one of the big boys in the web analytics world, but I didn’t know much about their business. Omniture’s offices are in Orem (a suburb of Salt Lake City, where I live), and I started seeing billboards for job openings on the freeway. So I investigated a little further and discovered that their revenues had been skyrocketing continuously for more than five consecutive years.

Wow.

From a technical perspective, Omniture’s front-end is extremely simple: a few hundred lines of extremely-obfuscated JavaScript, using an Image object to send user data back to a server-side script. Easy-peasy, lemon squeezy.

From a business perspective, Omniture serves enormous companies like Amazon.com and General Motors. Reading through their product literature, it looked like Omniture really only targeted major corporations, with a monolithic web presence.

And, of course, the Omniture product suite carried a very hefty price tag. So hefty, in fact, that you couldn’t even get any details about pricing without contacting an Omniture sales rep for more details.

Consultingware.

As I explored the competitive landscape a little further, it became clearer and clearer that the web analytics marketplace consisted of two different types of products:

1. Enterprise Analytic Solutions — Like Omniture, these products were almost always subscription-based, hosted services. And the cost of subscribing, I discovered, could add up to hundreds of dollars per month, even for a site like mine with less than 100,000 page views. Not exactly a friendly pricing model for the small-time webmaster.
2. Log-file Analyzers — On the opposite end of the spectrum, there were a few inexpensive products for analyzing Apache log files. These products were very different from the Enterprise Analytic Solutions in that they focused on technical details rather than on business intelligence.

I decided to address this under-served market, and I started developing WebDelve.

Developing WebDelve

WebDelve would be sold as a shrinkwrap product, not a hosted solution. And since webmasters would install it on their own servers, the JavaScript client would be able to use the superior XmlHttpRequest communication model (rather than sending data through an image bug). The reporting application would be a desktop rich client application rather than a web app. The user interface would look a lot like a stock-market charting program, cross-pollinated with Microsoft Outlook.

I also envisioned a mechanism whereby a centralized WebDelve database would keep track of the global traffic volume on the internet. Expanding on the stock market metaphor, this would be like the S&P 500 index, moving up and down with the daily fluctuations internet traffic. Individual WebDelve users would be able to compare their own traffic volumes against the traffic of the internet at large at any point in time. For example, if the traffic on my website dipped 3% on a holiday, I might be disappointed. But if I knew that the overall traffic on the internet dipped by 5% on that day, then I’d know my own traffic was exceeding expectations.

I spent a few months developing the JavaScript front-end of the application, as well as a skeletal server-side architecture.

And then, in March 2005, Google bought Urchin Software and released Google Analytics to the world.

Their product was aimed squarely at the same small-business market that I was targeting. But theirs was free.

I quickly lost interest, and started developing other software.

Re-Examining the Marketplace

I’ve been thinking a lot about that software lately.

Even a year after the release of GA, Google is unable to meet the demand for its product. GA is a hosted product, and it takes a lot of server resources to respond to all those hundreds-of-millions of JavaScript pings each day. In addition, the reporting infrastructure needs to be able to execute fast analytic queries against the data. I don’t know anything about their server architecture, but there’s gotta be a great deal of overhead just in managing such a large volume of data. Developing a web analytics product for a single website would be much more performant, even with more complex reports.

Also, there are a lot of businesses lately that don’t like the idea of storing their business-critical data on anyone else’s servers, even if they do trust Google. These kinds of companies would be more likely to purchase a shrinkwrap product than to use an ASP solution like Google’s or Omniture’s.

In addition, every web analytics product on the market uses a web interface for reporting. And personally, I think I could build a better UI in a rich client.

Finally, just because Google has the majority of the small-business market doesn’t mean there’s not room for another competitor.

A product like WebDelve would probably be priced between $50 and $200. Assuming a $100 price tag, I’d need to sell about 50 licenses per week in order to reach an annual revenue target of $250,000.

Since I relate to software developers and small-business owners, and since I’ve been a webmaster before, I think marketing would be pretty straightforward.

On the other hand, webmasters (particularly small-business PHP types) are more likely to expect free, open-source products. And they might balk at a price tag exceeding $100.

Pros:

1. There is a strong marketplace for web analytics, and the unmet demand is particularly strong among small-business owners.
2. The strongest competitor in this market is Google Analytics, and they have been unable to meet demand for their hosted solution. The majority of their potential users are currently stuck in a waiting list. Many have been waiting for more than a year to be approved for the beta program.
3. With my background in technical charting, combined with my background as a webmaster, I think I could develop a very compelling product.
4. A small business product can generally have higher license fees than a consumer product, so I’d have to sell fewer licenses of a product like this than I would have to sell if I was developing a consumer product.
5. I’ve already written a client-side JavaScript analytics component and a small library of web analytics database queries. Of course, I’d still have a lot of new code to develop, but at least I wouldn’t be starting from scratch.

Cons:

1. My primary competitor would be Google, a company with billions of dollars and thousands of developers.
2. The Urchin Software (which underlies Google Analytics) has been under development since 1997, so they’ve got a big head start in developing features and learning about their customers.
3. Besides Google (the primary, name-brand competitor), there are dozens of smaller, non-name-brand competitors in the marketplace. I’ll need to learn about their products if I want to compete with them effectively.

…

This is the first of 30 business ideas that I’ll be writing about over the next 30 days. Some of these ideas are new. Many of them have been rattling around in my head for several years. One of them will become a product over the next six months, and the foundation of my new software business.

This entry was posted on Saturday, June 10th, 2006 at 10:59 pm and is filed under biz. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.