I’m a few days late to dedicate today’s software idea to Independence Day, but in my opinion, it would still be a fitting tribute.

I’ve always been intensely interested in American politics. I read the news every day. I listen to NPR. I’ve even written letters to the editor of my newspaper and to my congressman.

But the government is so big, it’s difficult to keep track of what’s going on, and I think the public is very poorly informed.

For example, how much do you know about the senators from your state? How often do they vote according to party lines? How often do they sponsor their own legislation? What committees are they on? When they write legislation, does it usually pass? Do their bills attract a lot of unrelated amendments?

Those are the kinds of things that I’d like to know about my own representatives. But, for some reason, it’s very difficult to track down those kinds of statistics. Usually, data like that is only published by academics at universities. And even then, the data you can find is usually several years out of date.

I’d like to draw on my experience in the fields of computational linguistics and statistical analysis to develop a website with comprehensive information about what’s going on in our government, automatically updated and published in real time, and delivered in a format much like Google News.

Very few people (including most members of congress) have the time to read all of the legislation introduced into the congressional record. Most bills are hundreds and hundreds of pages long. Nobody wants to read that.

But a computer program could easily parse the text (which is available in the public domain) and identify some of the interesting tidbits. For example, in a piece of proposed legislation about gun control, it would be very interesting to identify amendments dealing primarily with road construction.

It would also be interesting to know whether a Supreme Court opinion written yesterday contained paraphrased passages from the bible, or from the transcripts of a United Nations session. Beyond the day-to-day events, I’d love to see a scatter-plot word-frequency chart comparing the distributions of word and phrases used during the official speeches of all American presidents. I’d also love to see which words and phrases were omitted by a particular news organization (like the Wall Street Journal) when reporting the same stories as other newspapers (like the New York Times).

I’d love to compare the text of a bill written last year (which failed) to the new version of the bill that passed this morning. How is the new version of the legislation different than the old version?

Which government officials are most likely to use canned phrases (like “death tax”, “cut and run”, “culture of corruption”, etc)? And when a politician uses those kinds of phrases, is he more likely to be on the winning side of a controversial bill than someone who refrains from using those kinds of catch phrases?

And although I consider myself to be a pretty liberal Democrat, I wouldn’t rig the system in any way. I’d just set up the algorithms and let it run. If the results weren’t pretty (for my own team), then they’d be published anyhow. I’d hope to attract people from both sides of the political spectrum to participate in the website, with forums and blogs to supplement the linguistics and statistical analysis (which would be central).

Market Analysis

I think a website like this would be very popular. With the possible exception of IT subject matter (“Python sux!! Ruby rulez!!”), politics is the most blogged-about topic on the internet. People get addicted. Easily.

Of all the ideas I’ve written about so far (and all of the ones I’ve conceived but haven’t yet written about), this is the only one where I’d feel totally comfortable relying entirely on advertising revenue to pay the bills and build a comfortable profit margin. Assuming a pay-per-click revenue model, and an average revenue stream of five cents per click, I’d need to generate about 100,000 clicks per week. And, assuming a click-through rate of 2%, I’d need total traffic of about (ignoring the weekends) a million page views per day to reach my target of $250,000 in annual revenue.

That’s a lot of page views, but it’s definitely doable for a popular site. Also, five cents is on the low-end for per-click revenue.

Technical Considerations

I’d only need to use a handful of NLP algorithms to provide a mountain of really useful information. Off the top of my head, I’d include:

• Statistically improbably phrases (aggregated at the sentence, paragraph, and document levels) for identifying thematic elements.
• Karp-Rabin analysis (using raw text, as well as morphologically normalized text), for detecting quoting and paraphrasing of documents.
• Agglomerative clustering (probably using k-means algorithm), for identifying congressional voting blocks, as well as individual senators who vote within (or deviate from) those blocks.
• A simple back-propagation ANN, to perform dimensionality-reduction and to generate self-organizing maps. These would be especially handy for creating visually-interesting representation of data from n-dimensional feature vectors.
• A general-purpose pivoting mechanism, so that different types of analysis could operate on many different kinds of variables within the system.

Beyond those algorithms, I’d also need to write a few parsers, to consume documents, annotate them with metadata, and generate feature-vectors. The feature vectors would have to be pretty general-purpose, so that they could be fed into the different analysis algorithms.

I’d also need a few spiders, for collecting new documents from known sources (house, senate, white house, supreme court, FindLaw.com, etc) and for feeding those documents into the appropriate parsers.

Pros:

• Although it’d definitely be a tough six-months worth of work, it’s in a field of my expertise, and I’d enjoy it very much.
• Financially, the numbers are pretty straightforward. I’d rely on ads (as well as some promotional products like t-shirts, bumper stickers, etc), but I don’t think the single revenue stream would be problematic.
• In addition to making good money doing interesting work, I’d also feel like I was doing something important. Contributing something to society. Not only would it be fun, it’s be a civics lesson!! That’d be a definite perk for me.

Cons:

• There just so much data out there. It would be difficult to gather enough of it to make a really compelling, and totally comprehensive, website. Without creating the impression that the data is comprehensive, I think it would lose a lot of value.

…

This is the fifteenth of 30 business ideas that I’ll be writing about over the course of 30 days. Some of them are only intended to generate enormous sums of money for their creator, while others might actually do something for the good of society (though the enormous sums of money would still be nice). One of these ideas will become a product over the next six months, and the foundation of my new software business.

This entry was posted on Thursday, July 6th, 2006 at 11:51 pm and is filed under biz. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.