Idea #6: AI-Coder.com
Do you ever get frustrated working on CRUD applications all the time?
Have you been mired in mind-numbing HTML/CSS/JavaScript work for too long?
What ever happened to the interesting “science-y” aspects of computer science?
How can computer science be so interesting, and yet software development is often so boring?
Hundreds of thousands of computer lovers go to school every year to study computer science and when they graduate from school are suddenly thrust into the world of software engineering. Like it or not, these are not the same thing.
Computer science is a field of scientific inquiry into the capabilities of computer technology. Computer science follows the scientific method, proposing hypotheses and conducting experiments. Some hypotheses will be upheld, while others are disproven. Consequently, it’s perfectly reasonable to expect that many computer science projects will fail. Risk is an inherent part of any branch of science.
Software engineering, on the other hand, is the disciplined process of constructing software systems, primarily by wiring together a set of pre-existing software components. Much like mechanical engineering relies on the principles of physics, software engineering builds upon the principles of computer science. Since software engineering aims to produce a high-quality product at a minimal cost, risks must be avoided as much as possible.
Of course, software engineering is satisfying in and of itself. Building a finished product is very rewarding, especially if that product is deployed to a large population of users, or if the product contributes something valuable to the world.
Sometimes, though, computer science is more fun.
The internet is flooded with online communities focused on software engineering, with particularly vibrant communities focused on software engineering for the Java and .NET platforms. But there are very few online communities dedicated to computer science.
And I’d like to change that, with the introduction of the AI-Coder.com community.
The centerpiece of the AI-Coder.com website would be a set of ongoing AI programming contests, structured like this:
- A central server would provide an API for playing certain turn-based games (chess, go, othello, monopoly, texas holdem, etc). The server would keep track of the game-state for all participating players and would handle things like scoring, turn-management, end-of-game detection, etc.
- Using the server’s plug-in architecture, members of the community would write their own AI code, exercising their skills in developing clever problem-solving algorithms. The server would dynamically load libraries for participating players, conducting an ongoing series of round-the-clock tournaments and matches.
- The server would maintain a score for all active players. After every match, the loser’s score would decrease and the winner’s score would increase, proportional to the differential of the two players’ starting scores. This ranking system would probably be very similar to that of professional chess players.
Personally, I think it’d be a lot of fun. And it would help keep my computer-science knowledge sharp.
I’ve already done a significant amount of work on this project. Rather than starting out developing a generic, one-size-fits-none game engine, I chose a specific game (Texas Holdem) and implemented a complete game engine, specific to the rules of that game. The current version of the software (around 14,000 lines of code) keeps track of the current table state (and the state of all ten players at the table), determining the winner of each round. I also wrote a several sample AI implementations (a few of which play pretty decent poker by creating probabilistic models representing each opponent), so that I could explore the capabilities of the engine.
Each AI agent implements the Strategy pattern, and the server itself uses the Command pattern to capture the results of each agent’s strategy into a sequence of Commands. Once each Command is validated as a legal move, the command-handling logic instigates a transformation on the central table-state. After the table-state is updated (and the end-of-game conditions are checked), the server delegates control to the next agent’s Strategy class.
It’s pretty cool.
A few things still need to be done, though, to facilitate the AI-Coder.com architecture:
- Implement a few other sample games (chess, go) and then refactor all the commonalities between those games into a single API.
- Refactor the AI logic-loading so that it can dynamically load agents (from their own self-contained JAR files) without restarting the server. Right now, all dependencies on AI classes are hard-coded, and that’s clearly not the right solution. I’ll have to write a custom classloader, so that I can pull JAR file paths from a database and then dynamically locate and load those JAR files into a new tournament instance.
- I need to tighten security. Currently, the agent-code is unrestricted. But in an actual production environment, the agent code should not be allowed to access the network, read or write files, connect to a database, or handle any other external resources. The agent code should execute within a very limited sandbox.
Competitors
The only company even remotely in the same competitive space is TopCoder. But their programming contests are very simple search/sort problems that can be solved within ten minutes. TopCoder problems are focused on algorithms and data structures, rather than on solving higher-level machine-learning types of problems.
Thankfully, TopCoder boasts tens of thousands of members in their community. The size of their community is a testament to the notion that computer programmers are willing to spend large chunks of their spare time writing code for fun (as if we didn’t all already know that).
Market Analysis
I’ve talked to a lot of my colleagues about this idea, and I usually get a bit of a lukewarm initial reaction. At first people are skeptical of the idea, but then they usually come back to me later, excited about the intellectual challenge of writing a simple AI for chess or go or holdem.
The fascinating thing about a project like this is that the agents won’t ever compete against human players. Only against other machines. So the challenges would be very different (your opponents have the same degree of brute-force computational power as you have, so you can only win by using better strategy).
At any rate, even if only a tiny minority of software developers would be interested in participating in a community like this, there are tens of millions of software developers out there, and I think I could stir up enough interest to make some decent money from a community like this.
Assuming that I charged $7 per month for a membership, I’d need a community with about 2975 paid subscribers in order to generate annual revenue of $250,000. That’s definitely doable.
Pros:
- I have already written a significant amount of code, as a proof-of-concept of this idea. I have a pretty clear idea of the architecture of the system as a whole.
- I’m already intimately familiar with the problem domain (and the people constituting the target demographic), so I think marketing would be pretty simple. Among other things, I’d contact computer science professors at all major US universities, recommending the AI-Coder platform as an arena for student projects.
- This is a project I’d happily participate in, if it already existed. As such, I would be very passionate about developing the software and the supporting community & infrastructure.
- With a community like this, it would be easy to derive multiple revenue streams: subscriptions, ad-clicks, and corporate sponsorships are the three most obvious.
Cons:
- AI code is very computationally intensive. And AI code written by inexperienced coders is even worse. Before being added to any tournament rotation, all member code should be run through a quick sanity test to make sure that it doesn’t overwhelm the system. Players who play too slowly should be automatically disqualified. Also, for certain types of games (gambling games come to mind), the game logic can include small penalties (increased antes) for players that use a disproportionate amount of CPU time.
- Even if all uploaded binaries are vetted against some speed threshold, I’d still need to acquire some pretty heavy-duty hardware. Assuming 2975 users, and one game-server running new games continuously, each user would get only 3 minutes and 23 seconds of processor time per week. That’s not likely to be enough, so I’d need to run multiple servers. This could get very expensive, very quickly.
- Software developers are notoriously cheap. Many of the people I’ve talked to about this idea have said that they’d gladly participate in a community like this, and they’d probably end up spending dozens of hours honing their algorithms. But most of them said they’d be unwilling to pay for a membership, no matter how inexpensive. Most of them even balked at a $5 monthly charge. Are there enough non-tightwad AI geeks out there to make this a profitable venture?
…
This is the sixth of 30 business ideas that I’ll be writing about over the course of 30 days. Some of the ideas are useful products for actual real people. Some of them are just entertaining distractions for people who already spend too much time sitting in front of a computer screen every day. One of them will become a product over the next six months, and the foundation of my new software business.
June 21st, 2006 at 2:03 am
Although I’ll love some kind of “official APIs” to compete against other ai programmers … I’ll not pay for it a cent, as you said I can spend hundreds of hours … but I’ll not pay any monthly fee …
But *may be* I could pay 50$ as an inscription fee to some conquest with a 1000$ price…. mmm … yes I vote for some kind of economical priced competition: gold, silver and pronce … payed with the inscription fees.
(opss…. I’ll forgot to say that the language barrier would trim your market, I would prefer that woud be based in .net framework (J#, C#, VB, …), please not flame war here ;-))
June 21st, 2006 at 4:58 am
It’s the best idea out of published six! You’re right that people will not be willing to pay even $5 for access. If I were you I’d try to earn money on ads, because you’ll have the possibility to run targeted ads. In addition, you can accept donations, it may be a source of income of a project.
If it were free and the interfaces were reasonably simple (so that I don’t have to spend a week to understand how to write an AI bot), I’d participate but I wouldn’t pay for it.
June 21st, 2006 at 6:06 am
It is a very interesting and exciting idea, but the subscription model won’t work. Also, I think you underestimate the security problems — the kind of thing that could keep cutting you off at the knees. Very unlikely to get a significant revenue stream ever, but something everyone would love you for if you could pull off providing it for free with corporate sponsorship and unobtrusive ads.
June 21st, 2006 at 9:13 am
Back in school, we had a fun end-of-quarter assignment where we wrote ‘robot control code’ for simulated robot combat. We used a software application called P-Robots (pascal) to do it.
I’ve often thought it would be cool to do on a larger scale and make the combat more visually appealing (rather than ascii). Apparently this exists for a lot of languages, here is a C version: http://www.nyx.net/~tpoindex/crob.html
I really like your idea, merged with the robot combat. You could have leagues and prizes – to which people pay entry fees, rather than monthly subs.
June 21st, 2006 at 9:24 am
I also think subscription won’t work, however similar model to that of real life poker tournaments will probably work just fine. That is all participants put down some money and then play several rounds. Whoever wins takes everything minus your fee.
June 21st, 2006 at 9:52 am
Ever hear of Terrarium?
http://www.windowsforms.net/Terrarium
June 21st, 2006 at 12:03 pm
Huh. I’m fascinated.
People regularly pay $1 to download four minutes worth of music, or $5 for a month subscription to Yahoo! Music, or $15 for a month of WoW.
And yet, the reluctance to pay even a tiny monthy fee for a service like the one I’ve described is remarkably pervasive. Maybe you guys could explain why you’d be willing to spend large chunks of your free time developing the AI code, but you wouldn’t be willing to spend $5?
That just blows me away.
Do you think there might be some way to overcome that reluctance?
Sure, it might be possible to get corporate sponsorships, and certain companies might want to recruit top-performing competitors. So, there’s definitely money to be made from that kind of business model. But those kinds of revenue wouldn’t start to flow until the community was already well-established. It would cost a lot for me to develop the community to that point, and I’m not sure it’d be worth it for me.
What do you think about a “rake”, like they implement at the online poker tables? For every game, a tiny amount of money would be deducted from your user accont? (Maybe five cents for a game of chess against another AI opponent?)
June 21st, 2006 at 12:04 pm
Actually, Dave, I had never heard of Terrarium. Looks pretty cool.
June 21st, 2006 at 1:48 pm
What about using a web service? So each player just implements the interfaces for the web service. When it’s their turn, you call their webservice by sending the game state, and they respond with their play.
Pros:
- Each player can choose their own framework or language.
- Ease security issues since no user code is running on your servers.
- Ease CPU usage requirements
Cons:
- Participants would have to have access to their own web server. (Although this could be offered as a paid service.)
- No way to guarantee that their plays aren’t rigged in some way.
To gain a user base, you might have free tournaments from time to time.
Make the service something participants love talking about. Make it easy for the geeks to brag on their blog. Create some buzz and keep it going.
As far as Terrarium, it wasn’t all that approachable. I downloaded the starter kits and such several years ago, but it wasn’t super easy to get the HelloWorld() equivalent running.
June 21st, 2006 at 1:49 pm
I cant’ answer why, I don’t even understand how can I have a behavour so irrational, but it’s what I feel. Maybe becaus I’m used to pay “to get” and not “to give” … and tha is the feeling that I’ve when programming bots. But the “rake” idea likes me very much, the posibility to be rewarded with money it looks very atractive, although I would prefer a bigger prize to the winner of a league.
June 21st, 2006 at 3:10 pm
Evgeny Shadchnev says:
“If I were you I’d try to earn money on ads, because you’ll have the possibility to run targeted ads.”
I would definitely run targeted ads, but I’d anticipate a few problems with relying on ads:
1) I don’t like being limited to a single revenue stream, especially if that revenue stream is so closely tied to my Google ranking (as is often the case with AdSense).
2) Tech-savvy people (and especially computer programmers) are probably the least-likely group of people in the entire world to click on ads. If I relied on ad revenue, I’d starve.
3) A closed community of users is unlikely to generate much in the way of click volume. Generally, the closed user community published content (in the AI-Coder forums, of course) and then all of the click volume is generated by a population of transient readers. I’d need to place a lot of focus on luring in the transient users. It’s possible, but I don’t know if I’d want it to be the focus of my attention.
June 21st, 2006 at 4:58 pm
I think this is a really neat idea — it’s something that doesn’t really exist yet; you’ve already got a working proof-of-concept; and it sounds like it would be a fun challenge.
But I don’t think 3000 paid subscribers sounds all that likely. Remember, it’s not a good idea to use “top-down” logic for your revenue projections. (”If we only service 5% of the people in China, we’ll earn a trillion dollars!”) How are you going to get your first paying client? And your 10th? And your 100th?
I think you could make a lot more money from strategic partnerships, like TopCoder does. For example, Sun sponsors TopCoder contests to get people to program in Java. Google sponsors contests to find new recruits for employment. My wild guess is that these sponsorships could net you $5000-10000 each. Probably more for a Google one, as it would be very high profile; less for some others. With this kind of approach, it seems like you could develop a sales and marketing strategy that’s more likely to succeed. It would also help grow your community for the unsponsored contests.
One benefit of the sponsorship approach is that there’s no network effect involved. You don’t have to have a big community to run a Sun contest, just the technical infrastructure to facilitate their contest.
Related to that: if possible, don’t tie your infrastructure to one programming language. You’ll never get a Microsoft sponsorship if your infrastructure requires agents to be developed in Java. And for an open contest, I bet a lot of developers would be more interested if they could do a language shootout (pissing contest), to do to show that their Lisp/Ruby/Python agents are better than my Java/VB/Fortran agents.
June 21st, 2006 at 7:57 pm
I think you are focusing too much on “receiving”. Your business, whatever it ends up being, should focus on “giving”. There is a lot to give. Believe me! You already know your interests and talents. Don’t try to focus on “I” and “receiving”. Look around you for opportunities of “giving” your talents and interests away (and I don’t mean for free). Have a cure for cancer? Give that. Have a cure for baldness? Give that. Get the point? Look for what people want. Not what you want people to want, or what you think people want. If you do this, you’ll have no shortage of clients.
Good luck!
June 22nd, 2006 at 12:32 am
I can see your point, Ryan, buy I’m reluctant to build a business plan where all of my revenue is conditional upon receiving enormous sums of money, in one fell swoop, from Google or Sun or Microsoft.
To me, it’s reminiscent of the whole Web 2.0 business model:
1) Build an online community with no conceivable source of revenue.
2) ???
3) Sell to Yahoo! for $20 million.
Sure, this is a little different, but I want my business plan to stand on its own, without assuming the appearance of some corporate deus ex machina to save me from my own lack of business planning.
Plus, I think the same chicken-vs-egg problem exists with the corporate sponsorship revenue model. Before I could get any significant sponsorships, I’d have to build a thriving community. And building a thriving community could cost a lot of money.
With the subscription model, at least I could scale the community up slowly, building the infrastructure using revenues collected from subscription fees. At some point, when the community reached a certain size and momentum, the inevitable corporate sponsorships would just add gravy to an already healthy stream of revenue.
June 22nd, 2006 at 6:15 am
I’d worry about critical mass if you try to build the community up slowly, although that might not be a problem for this particular group – but in some ways you’re building a social networking site, and they have different behaviors at different scales.
You’ll also have a problem with competition from academics. The IEEE digital library seems to be down right now so I can’t fact-check myself, but the June issue of Computer is about educating games programmers, and if I recall correctly there’s a two- or three-page discussion of an academic group that’s already in the late stages of putting together a system like you describe, has funding, and has run multiple successful competitions with their prototypes.
June 23rd, 2006 at 9:55 am
Sounds a lot like Robocode (and all of it’s predecessors) – http://robocode.sourceforge.net/ – I like the idea, but along with others in this comment thread, I wouldn’t pay for it. Not as it stands anyway.
I have been toying with something similar but to get people to contribute navigation systems and compete against each other to reach destinations etc, and the main way I could see people paying for something like this would be to offer free general entrance to your system, but in order to enter their code into sponsored events (prize money etc), they have to be part of your subscription plan. This way you get the money from the coders and exposure for your advertisers / sponsors.
Other than that I like it, mainly because I *completely* agree with you re: comp sci -> software engineering mind numbing transition. I finished my comp sci degree and for some reason thought webdev would be a good direction to head in. I realised recently that that step was the first in slowly destroying the mind that was interested in computer science in the first place and now I’m trying hard to find computer related interests that engage me again – AI, GA, Computational Art etc. Time will tell! Maybe this could be the next big market – targetting frustrated compsci grads..
Good luck!
June 23rd, 2006 at 11:40 am
I think you should do this as a hobby, as opposed to a business idea. You seem passionate about it, and it seems to have generated the most comments of all your ideas. That means there is a 20% chance it will be a hit and make a lot of money, but a 100% chance you’ll enjoy it. Every other idea has just as slim of a chance of making money, but would you REALLY enjoy making retail analytics software two years or twenty years from now?
August 2nd, 2006 at 8:42 am
I must be missing something but I just don’t believe this idea may be profitable for a software startup. It may be interesting to “big fish” like Microsoft of Sun, but startup?
August 12th, 2006 at 12:12 pm
Since you’ve now decided this is the chosen project. I’ll bump my idea again for feedback.
What about offloading some of the AI computation onto participant machines? I imagine a large number of participants would have access to a public IP for their own use. You could define a web service that participants could implement. When it’s their turn, you call their webservice by sending the game state, and they respond with their play.
This could also play into the segmented pricing model. Those who hosted their own AI code would consume less of your resources and would be elligible for reduced fees.