Was Steve Yegge Right?

I spent the first three years of my software career as basically just a guy that wrote code.  I came into work, I had some project I was supposed to work on, and so I wrote some code for it.  Usually this code worked and people were happy and sometimes it even made the company money, so I considered myself a good programmer.  And then I found Steve Yegge’s essays which made me reconsider all of that, and to this point just reading those essays has probably been the most formative experience in my career so far.

Steve was the first person I knew of that spoke about software engineering as a discipline that demanded self-improvement.  He wrote about dozens of subjects, but the one huge message I got across from reading all of them was, “there is more to this craft than meeting requirements at your day job and learning the syntax of the single programming language you use there.”  Some of it was a little intimidating, especially when Steve gave examples of suboptimal behavior that I was directly exhibiting (or when he was bashing on Perl), but as long as you focused on Steve’s overall point and didn’t just argue his potentially imperfect examples in your head, you would realize he always had something great to say.

So I’ve made it a habit to reread his essays from time to time, and the other day I checked out ‘Ten Predictions.’  It’s on his old Amazon blog (as opposed to his personal blog, which sadly he doesn’t update as often as he used to), and I remember first reading this in 2007 and thinking, “hmm, I wonder if he’ll be right.”  Well when I reread it, I saw a lot of references to things happening or not happening in 2010 and 2011, and since that’s, well, now, I don’t have to wonder if he’s going to be right — we can actually tell if he’s right or not.

So, let’s see what Steve had to say.  Keep in mind the original essay was written in early 2004 (I believe).

Prediction #1:  XML databases will surpass relational databases in popularity by 2011.
Reason for prediction: Nobody likes to do O/R mapping; everyone just wants a solution.

XML databases like BaseX never caught on, so while Steve was wrong about that, he was right that people did want solutions to problems that didn’t fit nicely into a relational database.  The past five years have seen an explosion of “NoSQL” solutions for structured and unstructured data, named such because they don’t require fixed table schemas and easily scale horizontally.

Will these solutions eventually become more popular than relational databases?  Well, I think that question is misleading because I don’t think relational and non-relational databases fight over ‘market share.’  We’ve seen a gradual evolution of both sides where relational database solutions are becoming more horizontally scalable and getting better at storing unstructured data, and NoSQL solutions are getting better at providing the same kind of query languages and developer tools available to relational databases.  So the idea of one surpassing the other in popularity is kind of a false dichotomy, as their designations will only get less and less binary as time goes on.

Prediction #2:  Someone will make a lot of money by hosting open-source web applications.
Reason for prediction: most small development shops would pay a monthly fee not to have to administer the apps themselves.

Steve worked at Amazon at the time he wrote this, and I’m wondering if this thought triggered Amazon’s EC2/S3 offerings.  “We’re moving to the cloud” is the latest buzzphrase, at least if you’d believe the overeager actors in terrible Microsoft commercials. Startups can launch their web applications in an incredibly fast and convenient manner and are gladly paying the nominal costs that Amazon charges.  Certain companies of a certain size ultimately have to manage their own hardware themselves (Oversee is one of them, incidentally), but for the most part offerings like Amazon’s EC2/S3 are a great option for companies to take.

Prediction #3:   Multi-threaded programming will fall out of favor by 2012.
Reason for prediction: Cost/benefit ratio isn’t high enough.

Probably Steve’s worst prediction. Unless multi-threaded programming ends up accidentally posting a picture of its privates on Twitter in the next five months, there’s no indication it’s going to be gone in 2012.

Prediction #4:  Java’s “market share” on the JVM will drop below 50% by 2010.

Almost every metric I’ve seen for “programming language” popularity is flawed (search results, github commits, job offerings, etc) so I won’t quote any, but my own anecdotal sense is just not seeing as many Java shops around as I used to.  At a previous employer in 2006 we decided to rewrite almost all our application code to Java for no apparent reason whatsoever, and I cant imagine any web companies making the same decision now.

Prediction #5:   Lisp will be in the top 10 most popular programming languages by 2010.
Reason for prediction: Lisp is a keeper. And it’s getting a lot of good press these days.

Again, I don’t know what it means to be “top 10,” but I do know that this would caused anyone to snicker until Clojure emerged recently.  Clojure’s emergence, as a Lisp dialect that runs on the JVM, pretty much means Steve nailed Prediction #4 and #5.

Prediction #6:   A new internet community-hangout will appear. One that you and I will frequent.
Reason for prediction: People like to socialize.

This was already obvious to anyone on the internet in 2004, when Friendster, MySpace, and Facebook were emerging.  So Steve was right, but I’m sure a lot of people could have made this prediction back then. In fact a lot of people are claiming they made this prediction back then, and feel that the mere prediction should mean they get more than $65 million in compensation, but that’s another story. Or movie.

Prediction #7:   The mobile/wireless/handheld market is still at least 5 years out.
Reason for prediction: My spider-sense isn’t tingling yet.

Steve goes into the various wireless offerings at the time, complaining that they’re all too expensive, not powerful enough, have crappy usability, crappy battery life, etc.  I remember in 2007 all the engineers got a Palm Treo at the company I was at, and I was so frustrated with it that I begged my company to just let me expense my personal cell phone at the time, the LG Chocolate.

And then the iPhone hit later that year, and the App Store opened in 2008, and basically everything changed.  And then Apple made everyone step up their game, just like Michael Jordan made everyone in the NBA step up their game, because otherwise everyone was just going to sit around and let Apple/MJ eat their lunch for 15 years.  Mobile is now a legitimate space where hardware and software companies can reap huge revenues.  Steve was right in that it was about 5 years until smartphones became a totally viable consumer good with mainstream penetration, but it wasn’t any longer than that.

Prediction #8:   Someday I will voluntarily pay Google for one of their services.
Reason for prediction: They’re putting technological heroin in their services, and someday they’re going to start charging.

Google’s continued to offer compelling products, but their model has always been one based on data and monetiizing on that data.  The mere presence of users (and thus collecting data on those users) will indirectly make them money, so pretty much everything they do is still free in terms of dollars.  Free in terms of privacy?  That’s another question…

Prediction #9:   Apple’s laptop sales will exceed those of HP/Compaq, IBM, Dell and Gateway combined by 2010.
Reason for prediction: Macs rule. Windows laptops are as exciting as a shiny disco ball in the ceiling.

I tried to actually Google the market share for Macbooks, but it’s probably impossible to find an absolute number.  Pretty much everyone is in agreement that Mac sales have increased dramatically since 2004.  Relative to PC usage, especially when you consider various sub-markets (laptops over $1000, laptops under $500, etc), the numbers are a lot iCloudier.

I do know this — I know a lot more people using Macs than I did back in 2004.  By probably any metric, Mac marketshare has increased. More than Windows though in overall mainstream penetration? Probably not anytime soon.

Prediction #10:   In five years’ time, most programmers will still be average.
Reason for prediction: That’s just how the math works out.
Steve’s point here wasn’t that “most programmers will be average” (since that’s a tautology, obviously) but that the average programmer will pretty much be the same guy in 2004 as they are in 2011.  There won’t be any sort of paradigm shift in things like programming languages or developer tools to turn the average programmer in 2011 into an order of magnitude more productive than he was in 2004.  And I think Steve’s point here was the one I felt he was always trying to make — no matter how the future turns out and whatever new concepts, applications, technologies, are introduced, we’re the most responsible for our own improvement.