Bob On Development

Let me say right up front that we’re not going to clear up the age-old argument between developers and bean-counters in this space. It’s been tried, over and over, since the beginning of the computer age.

Instead, I’ll just say that I think that, in the same way light has properties of both waves and particles, software development is both art and science. Indeed, in my view it’s a little more art than science. And I think that, just as there are very few true athiests in the world, there are very few people who would truly maintain that there is no art to development, and even fewer who would maintain there is no science to it.

Now that we have that out of the way I want to address the folly of going to the extreme of removing the craftsmanship from development and treating it as science.

Leaving aside my deeply-held suspicion that science isn’t going to save us anyway … that the image of dispassionate, relentlessly objective Men in White Lab Coats is a myth … let’s just say that there are powerful forces that want to make development into something that can be quantified, bottled, automated, commoditized, offshored, and in general made Predictable. These forces are well understood.

Historically every New Thing starts out as an arcane craft and is made one at a time, painstakingly. At some point, when its worth is proven, that product becomes standardized and commoditized so that it can be mass-produced, mass-affordable and more reliable (or at least Good Enough).

It is perfectly understandable that the world of business would hope that this paradigm could be applied to development. Especially since, tantalizingly, it appears that in some ways, it can be applied to development. For example, we can codify “best practices” and “naming standards”, and perform “static analysis”. We can use team development environments that enforce Corporate Standards — which can be either good or Stoopid, but let’s just assume they’re all good.

Even while conceding those points, though, there is a significant part of any non-trivial software development project that requires, not cold hard science, but passionate craftsmanship. And this is not just isolated to, say, the analysis phase of a project. It’s a thread that runs all through every part of the development life cycle.

Software develpment requires precious intellectual resources to be done right. Every day on a project, dozens of decisions must be made as to how to spend and apply those scarce resources. One must not only navigate the technical decisions, but the often conflicting and irrational political, regulatory, and marketing considerations which the project must effectively engage in order to be useful to anyone. And all of these factors are in constant and often chaotic change.

It’s tempting for non-developers and new developers to focus on the relatively easy stuff … the bare, obvious technical considerations, and assume that standard reductionist scientific thinking will eventually bring about predictable, reproducible results. But the truth is that organic, integrative, holistic thinking is needed, too. And that is the work of the mature craftsman.

This is happening in other fields of scientific endeavor as well, but it’s just easier to overlook. Historically, science’s hope was that by providing increasingly detailed understanding of natural processes we would come to fully understand them. But the fatal mistake was assuming that the universe is not infinitely intericate and complex, and that the same rules apply to the tiny picture (the sub-atomic level) as to the big picture (the cosmolocial level) and to everyday life at human scale. Science is increasingly running into trouble here. We have not found a hard limit to the size of the universe in either direction; we keep discovering annoying new dimensions we didn’t know existed; we keep encountering new rules that put previously accepted scientific fact to the lie.

It’s no different in the world of software development. You’ll never remove the need for the true craftsman. It takes someone who can deal with both the slippery, subjective, gut-level and human factors; it’s not just the technicians twiddling dials. And even the technicians should use Common Sense ™, which is not something that grows on trees.

The core skill of the software developer is correctly identifying and solving problems. Everything else is just execution. When software development projects are so often misbegotten and misdirected, we should not redouble our efforts to execute better. We should redouble our efforts to correctly understand the problem and the best points of leverage for addressing the problem.

Alas, accurate perception and accurate judgments about what to do with those perceptions, are not terribly amenable to Standard Processes. Knowledge and wisdom are two different things.

Update: Go here for more on this topic including links to related discussion.