From the meetup announcement:

The Brain Emulating Cognition and Control Architecture (BECCA) is a software framework for feature creation, reinforcement learning and action selection, designed for especially robotics, but is also more broadly applicable. It’s currently implemented in MATLAB, with a port to Python underway, and has demonstrated some very basic ability to allow simple robotics hardware to learn basic behaviors given a defined goal or task.

This presentation will include some demos of capabilities, a review of the published literature, and an introduction to hacking the code base and creating your own BECCA Agents and Tasks.

Oversee is hosting a meetup, Introduction to Mobile Development for Web, IOS, Android Native Apps using PhoneGap/HTML5, here in our offices on Wednesday evening, March 28. Jorge Garifuna will teach us how to design, build, test, and deploy one app across a wide range of iOS and Android devices using the PhoneGap HTML5 app platform. The meetup is open to the public. We’re providing snacks, beer, and soft drinks, as well as free parking to the first 45 attendees. RSVP on the meetup site to attend.

PhoneGap is very cool technology. PhoneGap’s parent company, Nitobi just got acquired by Adobe. The software, however, has gone into incubation as an Apache Software Foundation open-source project, named Cordova. It’s a JavaScript library providing a common API to different mobile platforms, along with tools to aid in packaging and distributing the apps you build using it.

It should be very interesting for anyone interested in mobile application development.

• EduBasic (on a DEC PDP-8e): I could write programs, and they’d do stuff!
• PDP-8e machine language: taught me how computers really worked. I still remember a few of the opcodes, decades later.
• C: Introduced me to pointers in a high-level language, and K&R introduced me to a clean elegant style of both coding and technical writing. I still use C today.
• APL: I used this for a summer job, on an IBM 5100 Portable Computer. It introduced me to the idea of operating in parallel on whole vectors at a time, as well as teaching me both the appeal and the danger of crafting one-liners.
• SPRITE: I can’t find any web references for this, it was the proprietary systems-programming language for the Burroughs (later Unisys) Medium Systems mainframes. It introduced me to the idea of separating interface declarations from implementations (C could do this with header files, but I hadn’t fully grasped the concept until I saw SPRITE).
• ALGOL: This was Burroughs Large Systems ALGOL, which was an “extended subset” of ALGOL-60 that had been hacked into something rather different. With it I wrote a program that ran on the most expensive computers I’ve ever worked with.
• Pascal: I used several variations, starting with USCD Pascal, and later Turbo Pascal. It’s type safe, and the Turbo variant compiled really quickly.
• Modula-2: Drove home the concept of modularity, and had the best post-mortem dump analyzer I’d seen up to that point (or for a long time after!).
• COBOL: I only ever wrote one test case in COBOL, but I discovered that it had a very rich (if wordy) language for describing data structures.
• C++: Showed that multiple inheritance was possible, and why it probably wasn’t worth the effort. Later, taught me about generic programming. I still admire the Standard Template Library.
• Visual Basic: Awesome for quickly prototyping Windows user interfaces. I prototyped a tool in VB in a less than a week that ended up getting used for a couple years in production.
• Perl: I started with version 4, which was a cool little language for hacking away at text processing tasks. Then I did web stuff in version 5, which unleashed the power of CPAN. Now I’ve discovered Moose, which almost makes Perl a whole new language.
• Java: Showed what C++ could have been, and how much easier automatic garbage collection makes programming in the large.
• Javascript: While I had futzed with it a little, it was Douglas Crockford’s writing that got me to take the language seriously. It’s a cool instance-based OO language, that’s got incredible market penetration.

I’m leaving out various languages I’ve played with but never did anything serious with, including SAM76, SNOBOL, Forth, Smalltalk, Lisp, Simula-67, Erlang, and so on. I learned a lot from many of them, too.

That actually wasn’t as hard as I’d expected, finding one thing nice about even the languages I liked the least.

(Image from osocio.org‘s story on Improv Everywhere‘s Say Something Nice mission.

Just a quick note to point out (boast?) that Oversee.net has signed up to co-sponsor SCALE 10x. We’re hoping to talk to lots of interesting Linux folks, not least because we’re currently looking to hire several software developers and sysadmins.

One inter-language discrepancy Tantek found has resulted in the silliest bug-report debate I’ve seen in a while. It’s remarkable how upset people seem to get when arguing over whether the “protocol” part of the URL should include the colon or not!

When Google+ came out, I was very skeptical.  After the debacle that was Google Buzz, I was pretty sure any product that Google put out in the social space was just doomed to failure.   Google gets search and they get advertising but social is something that can’t easily be described by an algorithm.  But being the tech nerd that I am, I signed up for it as soon as I was able to get an invite so I could evaluate it myself.

So what does Google know about social?  It seems they know quite a bit!  One of the things I can really appreciate is someone learning from their mistake.   It’s rare in life when you get something right the first time.  Is Google+ perfect? No way.  But it combines certain features and improves on others to make a pretty decent offering.  Should Twitter and Facebook be running scared?  Twitter maybe.  Facebook, probably not right now.  One thing is for certain; competition is good for consumers.  Up until now, there was not any competition for the two social Juggernauts.  Google+ changes that and I think that can only be good for innovation in this space.  So what do I like about Google+?

Best of Twitter and Facebook -  I really like that I can follow other people.  If it weren’t for this feature, Google+ would be a ghost town since most of my friends aren’t on it.  Outside of the people I know, there are actually a lot of people who contribute interesting content.  My favorites are probably +Trey Ratcliff and +Kevin Rose.  I like that I can follow them like I would on Twitter but comment and message each of the post like Facebook.

Circles are a first class citizen – I don’t want to share everything with everyone.  Circles represent real life.  I have a “friends” circle and a “work” circle.  They usually don’t mix.   The ironic thing?  Having the ability to segment my users this way makes me more likely to share even if I don’t use it.  Most of the posts I make now are public.  But I post a lot more on Google+ now than I did on Facebook.  Because I know I have the safety of segmenting my posts, it means I don’t usually think twice about what I might or might not put on Google+.  I can’t say the same thing about Facebook.

Now I know what many of you will say.  You can accomplish the same functionality on Facebook using Groups.  But the problem is that this is an “add on” to Facebook.  It was not the paradigm from the very beginning and therefore it seems bolted on.  Like I said in my last post, easy is not an add on.  Facebook Groups is not intuitive to use and because it was bolted on later, it seems too complicated to use now.  That isn’t to say that Circles was easy at first, but now that the model is established, I think people will find it much better going forward.

Post can be public - While I really like the concept of circles, I like it even more that my posts can be public.  It should in theory allow people to reach a bigger and wider audience and that is valuable.  I love sharing my knowledge and experience.  It is the main reason I started and contribute to this blog.  The ability of me to have fine control over what is very limited sharing to very broad sharing is VERY attractive to me and I’m sure to others with public profiles.

Limits suck –  If I want to write an essay, I should be allowed to.  Twitter has a needless limit of 140 characters.  I get why this made it unique at the beginning.  I even think terseness is valuable.  But Twitter never really worked for me because it ended up just being links to other content.  Facebook has a limit to their posts length, and you actually hit it pretty easily if you are as verbose as I am.  If the average user can hit your limit, then you probably need to rethink your limit.

Photos sharing is superior to most everything else out there – Facebook became a giant because it allowed people to do something they really wanted to do, share photos easily with their friends.  Google+ took this concept and then put it on steroids.  Twitter didn’t even really have photos till recently and the photosharing capabilities of Google are just better than Facebook.    Let me list all the ways this is true.

1. Big beautiful pictures in the stream.  Pictures are not supplemental to the post.  They dominate the post and that is good.  When I stroll through my stream, I notice the pictures.
2. Photo Galleries in Google+ are gorgeous.  Just check out another one of my favorites, Thomas Hawk’s Gallery and you will know what I’m talking about.  Love the infinite scroll.
3. You can edit the photos after you upload them.  Google+ allows you to color correct or add filters to the photos after you have uploaded it.  This is actually a pretty awesome feature, and I can see it being a big differentiator for Google as more and more people discover it.
4. The Google+ lightbox where they place the photos is just better.  The comments make more sense to be on the side and having the slider on the bottom makes scrolling much much easier.

You can edit posts after the fact – Seriously.  Who the heck gets everything right the first time (see above comment)?  I can’t tell you the number of times I’ve screwed something up only to want to fix it and be unable to.  That’s just dumb.  The only way to clarify something on Facebook is to update it through comments and that is just lame.  If I wrote it, let me fix it.

 You know Google will integrate Google+ into its search results –  In fact, they already have.   I can guarantee you, now that Google has a reasonable foothold in the social space, they aren’t going to let go of it very easily.  That means, if you care about SEO, you better figure out what this means to you.

Now, am I really that bullish on Google+ replacing Facebook?  No, not really.  Facebook has a huge advantage of numbers.  Everybody you know is on Facebook.  Even the people you may know on Google+ are probably not all that active.   Like I said earlier, Google+ is kind of a ghost town except for some of the more prolific posters.  I have a number of friends who have signed up for Google+ but most are not using it.  I’m actually pretty sure I’m dominating most of their streams.  The network effect is working very strongly in Facebook’s favor.  But keep one thing in mind.   At one point, MySpace had the same edge on Facebook.  At one point Microsoft had an insurmountable lead on Apple.  However, if you put a better product in front of people, over time, you can erode any sort of network effect.  If Google+ can reach critical mass, they have a chance.  And that will be good for everybody whether they are on Facebook or they are on Google+.

By the way, if you want to follow me on Google+ you can.   +Terrence Lui.

Update:  One other thing I wanted to add that seems very promising is the unusual level of engagement that Google+ seems to be generating.  I posted two identical updates to my streams on Facebook and Google+.  Despite knowing many more people on Facebook’s platform, I got over 5x the number of comments on Google+.  Other users of Google+ have noted the same thing.  I don’t know what is behind this phenomenon but this behavior could prove to be a gold mine for marketers.

I am a busy guy.  This means I’m not the type of person who likes to spend lots of time eating lunch.  The quicker and easier it is for me to grab lunch the happier I am.  For lunch today,  I decided to get a sandwich.  I discovered that the sandwich shop downstairs allows me to call in my order ahead of time so I can just walk up to the cashier and pay.  There is a special, shorter line for call-in orders.  Further, if paying by credit card, it is one simple swipe and less than seven seconds later you have your receipt and you are out the door.     It is a true model of efficiency.

I am much more likely to be a customer of this establishment in the future because they have made it exceedingly simple for me to get what I want.  I didn’t waste time in line behind people who don’t know what they want nor did I have to wait for something as simple as handing over my money.  This simple yet important principle seems to be lost on so many of us in technology.  As engineers, we are used to dealing with the complex.  We don’t mind jumping through hoops to configure web servers, navigate deep directory structures, or manage the dozens of windows on our desktop.   We seem to forget that not everyone has the same love affair we have with computers.

We tend to think of usability as an afterthought.  We figure that if we got you your sandwich that should be good enough.  Are you still hungry?  No?  Great, requirements met.  It did not matter how we got you there it only mattered that you got there.  This started to change in large part thanks to Apple and Steve Jobs.  The introduction of the iPod and then the iPhone changed the way people talked about creating products.  When I worked at Microsoft, I cannot tell you the number of times the words “iPod” and “iPhone” were said in discussions about design.  It became a running joke in my team and others that no matter what we talked about, we eventually had to tie it back to how great the iPod experience was.   Before this, engineers used to talk about cramming more and more features into a product and did not worry about people were going to access those features.

Ease of use should be a basic product principle, not an afterthought bolted on after all the functionality is complete.  If you are working on a feature, and there is not a clear and intuitive way for your users to access it, you should honestly question whether your user will derive any value from it at all.  Even worse, having even one hard to get at feature can remove the whole usefulness of all the other hard work you put into the product.  To use my analogy further, imagine what I would have thought if I got all the way to the payment portion of my sandwich transaction only to realize I had to wait five more minutes to pay.   The “payment” feature needed to work in concert with “ordering”, “taste”, and the rest to ensure a great overall experience.

We are living in an ever more complex world.  While it is very hard to mask this complexity from your user, it will be an ever increasing differentiator between those who succeed and those who do not.  Not treating ease of use as a first class citizen when it comes to design, requirements, and planning is just handing over victory to your competitors.

Steve was the first person I knew of that spoke about software engineering as a discipline that demanded self-improvement.  He wrote about dozens of subjects, but the one huge message I got across from reading all of them was, “there is more to this craft than meeting requirements at your day job and learning the syntax of the single programming language you use there.”  Some of it was a little intimidating, especially when Steve gave examples of suboptimal behavior that I was directly exhibiting (or when he was bashing on Perl), but as long as you focused on Steve’s overall point and didn’t just argue his potentially imperfect examples in your head, you would realize he always had something great to say.

So I’ve made it a habit to reread his essays from time to time, and the other day I checked out ‘Ten Predictions.’  It’s on his old Amazon blog (as opposed to his personal blog, which sadly he doesn’t update as often as he used to), and I remember first reading this in 2007 and thinking, “hmm, I wonder if he’ll be right.”  Well when I reread it, I saw a lot of references to things happening or not happening in 2010 and 2011, and since that’s, well, now, I don’t have to wonder if he’s going to be right — we can actually tell if he’s right or not.

So, let’s see what Steve had to say.  Keep in mind the original essay was written in early 2004 (I believe).

Prediction #1:  XML databases will surpass relational databases in popularity by 2011.
Reason for prediction: Nobody likes to do O/R mapping; everyone just wants a solution.

XML databases like BaseX never caught on, so while Steve was wrong about that, he was right that people did want solutions to problems that didn’t fit nicely into a relational database.  The past five years have seen an explosion of “NoSQL” solutions for structured and unstructured data, named such because they don’t require fixed table schemas and easily scale horizontally.

Will these solutions eventually become more popular than relational databases?  Well, I think that question is misleading because I don’t think relational and non-relational databases fight over ‘market share.’  We’ve seen a gradual evolution of both sides where relational database solutions are becoming more horizontally scalable and getting better at storing unstructured data, and NoSQL solutions are getting better at providing the same kind of query languages and developer tools available to relational databases.  So the idea of one surpassing the other in popularity is kind of a false dichotomy, as their designations will only get less and less binary as time goes on.

Prediction #2:  Someone will make a lot of money by hosting open-source web applications.
Reason for prediction: most small development shops would pay a monthly fee not to have to administer the apps themselves.

Steve worked at Amazon at the time he wrote this, and I’m wondering if this thought triggered Amazon’s EC2/S3 offerings.  “We’re moving to the cloud” is the latest buzzphrase, at least if you’d believe the overeager actors in terrible Microsoft commercials. Startups can launch their web applications in an incredibly fast and convenient manner and are gladly paying the nominal costs that Amazon charges.  Certain companies of a certain size ultimately have to manage their own hardware themselves (Oversee is one of them, incidentally), but for the most part offerings like Amazon’s EC2/S3 are a great option for companies to take.

Prediction #3:   Multi-threaded programming will fall out of favor by 2012.
Reason for prediction: Cost/benefit ratio isn’t high enough.

Probably Steve’s worst prediction. Unless multi-threaded programming ends up accidentally posting a picture of its privates on Twitter in the next five months, there’s no indication it’s going to be gone in 2012.

Prediction #4:  Java’s “market share” on the JVM will drop below 50% by 2010.

Almost every metric I’ve seen for “programming language” popularity is flawed (search results, github commits, job offerings, etc) so I won’t quote any, but my own anecdotal sense is just not seeing as many Java shops around as I used to.  At a previous employer in 2006 we decided to rewrite almost all our application code to Java for no apparent reason whatsoever, and I cant imagine any web companies making the same decision now.

Prediction #5:   Lisp will be in the top 10 most popular programming languages by 2010.
Reason for prediction: Lisp is a keeper. And it’s getting a lot of good press these days.

Again, I don’t know what it means to be “top 10,” but I do know that this would caused anyone to snicker until Clojure emerged recently.  Clojure’s emergence, as a Lisp dialect that runs on the JVM, pretty much means Steve nailed Prediction #4 and #5.

Prediction #6:   A new internet community-hangout will appear. One that you and I will frequent.
Reason for prediction: People like to socialize.

This was already obvious to anyone on the internet in 2004, when Friendster, MySpace, and Facebook were emerging.  So Steve was right, but I’m sure a lot of people could have made this prediction back then. In fact a lot of people are claiming they made this prediction back then, and feel that the mere prediction should mean they get more than $65 million in compensation, but that’s another story. Or movie.

Prediction #7:   The mobile/wireless/handheld market is still at least 5 years out.
Reason for prediction: My spider-sense isn’t tingling yet.

Steve goes into the various wireless offerings at the time, complaining that they’re all too expensive, not powerful enough, have crappy usability, crappy battery life, etc.  I remember in 2007 all the engineers got a Palm Treo at the company I was at, and I was so frustrated with it that I begged my company to just let me expense my personal cell phone at the time, the LG Chocolate.

And then the iPhone hit later that year, and the App Store opened in 2008, and basically everything changed.  And then Apple made everyone step up their game, just like Michael Jordan made everyone in the NBA step up their game, because otherwise everyone was just going to sit around and let Apple/MJ eat their lunch for 15 years.  Mobile is now a legitimate space where hardware and software companies can reap huge revenues.  Steve was right in that it was about 5 years until smartphones became a totally viable consumer good with mainstream penetration, but it wasn’t any longer than that.

Prediction #8:   Someday I will voluntarily pay Google for one of their services.
Reason for prediction: They’re putting technological heroin in their services, and someday they’re going to start charging.

Google’s continued to offer compelling products, but their model has always been one based on data and monetiizing on that data.  The mere presence of users (and thus collecting data on those users) will indirectly make them money, so pretty much everything they do is still free in terms of dollars.  Free in terms of privacy?  That’s another question…

Prediction #9:   Apple’s laptop sales will exceed those of HP/Compaq, IBM, Dell and Gateway combined by 2010.
Reason for prediction: Macs rule. Windows laptops are as exciting as a shiny disco ball in the ceiling.

I tried to actually Google the market share for Macbooks, but it’s probably impossible to find an absolute number.  Pretty much everyone is in agreement that Mac sales have increased dramatically since 2004.  Relative to PC usage, especially when you consider various sub-markets (laptops over $1000, laptops under $500, etc), the numbers are a lot iCloudier.

I do know this — I know a lot more people using Macs than I did back in 2004.  By probably any metric, Mac marketshare has increased. More than Windows though in overall mainstream penetration? Probably not anytime soon.

Prediction #10:   In five years’ time, most programmers will still be average.
Reason for prediction: That’s just how the math works out.
Steve’s point here wasn’t that “most programmers will be average” (since that’s a tautology, obviously) but that the average programmer will pretty much be the same guy in 2004 as they are in 2011.  There won’t be any sort of paradigm shift in things like programming languages or developer tools to turn the average programmer in 2011 into an order of magnitude more productive than he was in 2004.  And I think Steve’s point here was the one I felt he was always trying to make — no matter how the future turns out and whatever new concepts, applications, technologies, are introduced, we’re the most responsible for our own improvement.

To serve ads on millions of pages per day, we used to use a fairly traditional LAMP stack. We used Apache with mod_perl to run a fairly complicated tangle of Perl code which queried several back-end MySQL databases and other back-end servers, in order to choose what to put on each page. One of the biggest problems we had was that we wanted to be able to easily add new (possibly buggy) features for A/B testing, knowing that many of those features would fail and be discarded. The old way we did this was to have a separate cluster of servers running the test version of the code. The trouble was that the test cluster was never 100% identical to the main production cluster, and the redirect needed to send a sample of requests to the test cluster introduced a delay, so we had problems with test results not being reproducible when we later launched the same feature in the main production cluster. We wanted to be able to isolate test features, while still running them in the same production environment that handled the majority of our traffic. In order to do this, while maintaining or improving our response time, we re-structured our serving system into a tiered, distributed system. There’s a front-end that just understands HTTP requests, but is fairly agnostic about what to do with them. After parsing the requests, it sends them on to a server which implements the main business logic, including the ability to run new features in a sandbox, such that their failure won’t affect the rest of the system. This business logic server queries a variety of back-end data sources, and sends the resulting information to an HTML layer, which does the presentation formating of the information in HTML/CSS/JavaScript, and passes the resulting page back to the HTTP server.

We needed an efficient, but extensible, protocol for these servers to communicate with each other. At previous jobs I’d worked on other projects where we rolled our own binary protocol for similar distributed systems, and I’d seen the headaches involved in upgrading the protocol version, even just to change the format of one field. We looked around, and found several fairly similar libraries for extensible binary protocols. The main open source competitors at that time (mid-2008) were Protocol Buffers and Thrift. Protocol Buffers (or “protobufs”) had just been released by Google, after several years of internal use. Thrift had been created by some ex-Google interns working at Facebook, and so the two are fairly similar. Both produced very compact on-the-wire encodings, though protobufs, with its variable-length numeric fields, had a slight edge. Both were designed to allow easy extensibility; new optional fields could be added without breaking existing code (which would just ignore the new fields).

We chose protobufs for several additional reasons, not that overwhelming individually, but combined they made protobufs a better choice for us. First, the documentation of protobufs seemed more complete and thorough. Second, their Perl support, while not part of the core project, was better suited to our plans to write non-blocking event-driven clients and servers. Thrift’s Perl support was very complete on the client side, but there was no server-side implementation, and the client code did its own blocking network I/O, which would be hard to integrate with an event-driven framework. There are actually at least 3 Perl bindings for protobufs. protobuf-perl was the first pure-Perl implementation. We initially tried protobuf-perl, but it had some serious performance issues, partly due to teething issues with Moose, which was still relatively new and unoptimized at that point. At that point, protobuf-perlxs was released, which puts a Perl wrapper around the C++ code generated by Google’s stock protocol buffers compiler. It has the disadvantage that it requires compiling and linking to C++ code, which makes it slightly harder to port and install on unusual architectures, but its speed advantage over pure-Perl implementations more than makes up for that. Google::ProtocolBuffers is a newer pure-Perl implementation that includes its own parser for the .proto interface description file format, and is the only implementation in CPAN; it was released after we’d finished our evaluation of the other two implementations. It supports creating message definitions at run time (as opposed to build time), which could provide some interesting flexibility, but for our purposes protobuf-perlxs’s speed is more important.

We wanted our distributed server software to be as efficient as possible. At the time, the old system was using Apache in a pre-forking multi-process mode. In this mode, Apache forks dozens of child processes when it starts up, and hands requests to those children to be processed. At any given time, a child process is handling only one request; therefor, the server as a whole can only handle as many simultaneous requests as there are child processes. Processes are relatively expensive, you probably don’t want the OS to have to time slice among many thousands of them. Event-driven programming is an alternative model; it gives greater efficiency, but more care must be taken to avoid calling code that blocks, typically to do I/O. In an event-driven model (see the Reactor pattern for more details), the framework controls the main loop, and all application programming is done in callback functions. Python’s Twisted framework, Ruby’s EventMachine, and JavaScript’s Node.js are popular event-driven frameworks in other languages. Given that we were designing a new architecture, working with several pieces of new technology, and had a team already familiar with Perl, we decided to stay with that proven, robust, flexible language.

When it comes to just about any need, the Comprehensive Perl Archive Network (CPAN) provides an embarrassment of riches. There are several popular event-driven frameworks for Perl. The largest and most mature is POE; a more recent competitor is AnyEvent. The main author of POE, Rocco Caputo, has been working on a cleaner re-implementation of POE using Moose, called Reflex. There are also some smaller event-driven libraries that aren’t as all-encompassing: Event::Lib, Event, EV, and IO::Async. For our purposes, at the time we made the decision, POE provided the most useful components for things we needed, such as asynchronous DNS queries, fully-featured HTTP request processing (including keepalive, HTTPS, etc.), and the ability to easily support our own protocol.

Today, as a result, we have a large distributed system handling a huge number of requests across an easily-scalable cluster of machines, it’s relatively easy to write new servers using our POE-based framework. We can also add new fields to the messages we send between the tiers, without having to worry about compatibility issues when old and new versions of the code talk to each other.

It also means you’re bored. It’s not a case of possibly being bored, or eventually becoming bored. Once you are are no longer a problem solver, that means you’re bored, and if you’re bored at your current company for long enough, eventually you’ll find a new company to work for.

Companies talk a lot about retention, but rather than wring their hands about salaries and titles, they’d do well to look at their engineers and ask a simple question: “Who is bored, and what can we do about that?”