Bob On Development

I read a post this morning about the “lack of craftsmanship” in software development. The basic argument in a nutshell is that the “lack of commitment to quality” is pushing the whole industry towards the “commoditization” of software development. In fact, the author feels that “the days of in-house software development as we know it are in danger of extinction.”

Despite the fact that I agree with the author that “commitment to quality” is generally weak and contributes to the problem, I believe that larger forces are the primary driver behind this trend. It wasn’t lack of craftsmanship, after all, that resulted in the success of Henry Ford’s mass production methods. It was the demand for quantity at the expense of fit and finish. And let us not forget that the mass-produced Model T was not better or even easier to use than many of the hand-crafted cars that preceded it. That didn’t stop it from being a success.

Closer to home, the success of DOS comes to mind. It was an adequate product at the right time and price point. I’m not pretending that this fact is appealing to me as a craftsman, but I have to acknowledge that is was a success nonetheless.

History is full of examples of new products, produced originally by craftsmen, and eventually bent to the will of mass production. It’s a basic, natural business and technology cycle issue any time you’re dealing with something that there is a quantity demand for.

We’re in a similar situation in the software industry. Business and government need software of all kinds more than ever, but we can’t produce and maintain / extend “good enough” software in the quantities and within the time frames that the world needs (or thinks it needs, which in practice amounts to the same thing).

The only thing that has saved custom software development at all is that software does not, and never will, lend itself to commoditization efforts to the extent that mass produced physical goods will. However, it lends itself to more commoditization than most developers give it credit for.

Consider humble VisiCalc, the first spreadsheet program. It was a metaphor that took the world by storm, simply by allowing a small subset of bean counting and statistical tasks to be performed by non-programmers. I submit to you that Excel 2007 has just taken spreadsheets to the next level, by the simple expedient of removing Excel’s old 65,535 row limit. Several versions back, Excel acquired some basic database functionality, resulting in much mirth amongst database developers as accountants and casual developers tried to make Excel serve as a crude “database”. The exponential growth of typical table sizes so that a table of over 64K rows is no longer considered huge or even hefty, has pretty much knocked Excel out of the competition for awhile. But suddenly, it’s back in the running again, with bigger capacity, better DB features, and with user-friendly features like pivot tables in place of SQL GROUP BY queries. And with multiple sheets, users are no longer limited to single tables, so we’re not just talking about a simple list manager substitute here. Add in the ability to read and write to real databases with greater and greater ease, and you have modern Excel “spreadsheets” doing more useful work than the custom applications of twenty (or in some cases, even ten) years ago.

Now of course the scope and reach of modern database applications has also grown, and it’s still way beyond what even the newly beefed-up Excel can cope with. But the point is, at the low end — where it doesn’t need to scale and where standard capabilities are “good enough” — there will always be a place where casual and non-programmers can get by without us. And what constitutes “low end” is constantly growing. Eventually, “low end” will probably encompass all the elementary things that a small to medium-sized business needs for it survival, and the projects that require actual skilled developers will probably be more and more esoteric.

I see no point in bemoaning this. It’s just the usual march of progress. It’s not up to reality to conform to us. We have to conform to reality.

The only question is in how we chose to place our bets. What will we educate ourselves about? What kinds of projects will be seek out? And with how much urgency?

One thing’s for sure. The world is moving too fast to imagine that the craft of development will look anything like it does today, some twenty or even ten years hence. It was possible for a developer to learn COBOL in 1965 and stick with it until retirement, provided you didn’t care about the bleeding edge or the top dollars throughout your career. But I very much doubt that C# or VB.NET or even the darling of the moment, Ruby, will be employing anyone at all come 2040.

That’s going to burn a lot of people in this business, because for all our embracing of computer technology ahead of the rest of society, we developers have a strong conservative streak in us. We become emotionally attached to certain platforms and even languages, and fight to the death to keep them alive. A lot of us find something that appeals to us aesthetically and want to stick with it because it becomes our identity. That’s not going to work any more.