Once in awhile, for entertainment, I visit online freelance software development exchanges. It’s a fascinating study of the endless human capacity for witless self-deception. I actually saw a posting today from a guy who wants someone to clone an entire accounting system. Other than the name of the system to be cloned and an unexplained desire to have the work done in PHP, that’s the entire specification. And the budget? Two hundred dollars. I kid you not.
Apparently this guy thinks there are people who can knock out an entire accounting system in two hours. Okay, maybe he’s thinking twenty hours with third world labor. But still.
Apparently his reasoning process goes something like this: (1) I can buy Quicken for $79, therefore (2) it will cost a LOT more than $79 to write my own clone of Quicken — maybe as much as three times as much, but (3) I’ll cut it down to an even $200 as a starting point for negotiations.
I recall a couple of years ago getting a contract pitched to me where the objective was essentially to clone the whole Google search engine. The guy seriously believed that I could lead a team of a half dozen others and pull off in six months what Google, a company with thousands of employees, has spent a decade building. We would all work for fifty grand a year but in six months we’d go public and we’d all be millionaires. Hooray for us!
This guy got my award for Most Improbable Personal Delusion of the Year, until a scant week later I saw a very similar posting to a freelance site with a budget of $500!
There are literally hundreds of such postings floating around the Internet at any moment in time. It makes me wonder if these boards exist for no purpose other than to fleece the simple minded. What kind of contractor would respond to such a post? It must be the sort who will take a one-third deposit up front and then disappear.
What disturbs me most about all this is that while it represents an extreme, it seems to reflect that the commoditization of the craft of software development has reached some kind of fever pitch. I turn down many projects that come my way, for the simple reason that the customer has totally unrealistic expectations. Thankfully no one has yet asked me to write an accounting system or a major public web site or a spacecraft control system in one afternoon for under a hundred dollars — yet — but sometimes it seems like things are heading that way.
What this tells me is that the average person no longer values software. We are used to freeware, shareware, open source, and sub-$100 list prices for commercial software. The gargantuan size of the software market hides the gargantuan effort and expense that went into developing all those general-purpose software products.
Additionally, software development remains a painful and challenging process with many pitfalls for the unwary, and it just doesn’t deliver the effortless and instantaneous results we’ve come to demand of every aspect of life. People will jump through rings of fire and eat little pieces of glass before they will make allowances in their business plans for the fact that the mission-critical software they need will take six months and a couple hundred thousand to put together. “Screw it … we’ll use Excel!” they say.
So … what’s the deal? Is software a commodity now? Can we really put together complex systems in minutes from off-the-shelf components with no planning or testing?
According to a recent profile in Technology Review, no less a person than uber-programmer Charles Simonyi (the guy who was once the driving force behind Microsoft Word) plans to give the public what they are clamoring for. He is on a multi-year quest to create something he calls “intentional software”.
The over-simplified summary of what Simonyi wants to create is: “an expert system that will allow non-programmers to put together software specifications.” Then, by pushing a button, all the code will be generated to produce complex applications that fulfill those specifications.
I’ll concede that I probably don’t appear to be the best candidate to be impartial about the feasibility of Charles’ dream. Still, I doff me hat to ‘im, and wish ‘im well, I do. Because if he actually pulls it off, we will be living in a world of luxury and abundance and riding the wave of another “dot boom” and I will find a way to prosper from it, as will you, gentle reader.
However, my personal prediction is that what will come out of it, is something akin to Charles’ original brainchild, the graphical / WYSIWYG word processor. By which I mean that it will be something we will all take for granted and wonder how we could possibly live without, but it will also fall short in ways that will annoy the living bejesus out of us. (In the above referenced article there is a priceless vignette where Charles’ PowerPoint presentation hangs and the much-hated Clippy the paper clip pops up to offer useless advice. Charles confesses ruefully to the interviewer that even he has no idea how to disable the blasted thing).
Why do I think Charles will fall short of his lofty goal?
One of the reasons is well presented in a sidebar to that article in Technology Review, and that is generically known as the “leaky abstraction problem”.
At a more fundamental level, Charles’ earnest vision depends on reductionism, the idea that every problem can be solved if you can break it down far enough. This is an understandable stance for a software developer, as much of what we do involves breaking large problems down into smaller sub-problems and then putting it all back together. But it has its limitations. When developing software, you are ultimately solving problems that involve “wetware” (people) and that is inherently subjective and messy and chaotic. At some point that interferes with doing science and you have to make very subjective judgments to find a way forward. No development methodology or tool will ever fully automate those judgments.
Now I’m going to say something provocative and easily misunderstood: most of the world’s business is conducted by small to medium-sized companies. Most custom development needed by such companies are relatively modest compared to an enterprise-scale “Manhattan project”. In almost every case, by the time you’ve adequately spec’d most of these projects, you have finished implementing them. It is literally a case of “we’ll know when we are done”. I know this will make the Agile folks, the bean counters, and others apoplectic, but it’s the truth. Anyone who tries to paper it over is just patronizing you.
Software development is an iterative process of discovery. It’s done when it’s done. If you have a big enough project you can do proof-of concept “mini” implementations of various things, run simulations, conduct focus groups, have committee meetings and fill out ISO forms to your heart’s content. And then maybe, assuming all the stakeholders have been honest and diligent (a huge assumption), you will be able to come up with a reasonably close cost estimate / budget.
But in a smaller setting the grim reality is, you’ve got a champion with a fairly vague idea of what they want — more of a vision than a spec — which is communicated in glowing terms to someone like me, who does their best to ask pertinent questions and classify the project as a one, three, six or twelve month project times X number of developers. As the project rolls on the champion has a thousand suggestions for changes, the actual users inform you that for various reasons certain aspects of the vision are completely unrealistic, certain others decide they want the project to fail and employ passive-aggressive tactics to bring that about … and in the end, you have something that is not really what anyone expected up front, in terms of its final form or its cost, other than in very general terms, such as, for instance, “an online credit bureau for trucking companies”. Which comes to think of it, sounds suspiciously like the tiny postings on those freelance boards.
What the online freelance exchanges represent is the earnest desire of many to be able to express a complex system in one sentence, pay a tiny, nay, microscopic fixed amount of money and have it appear magically in a few days or even hours (another common subtext in these postings is “I’ve known about this for two months but just remembered that the deadline is tomorrow”).
Listen to me carefully: IT ISN’T. GOING. TO HAPPEN.
What Charles Simonyi’s more refined vision represents is the understanding that you can’t express complex systems in one sentence. But he still labors under the belief that with the right tools you could express it in terms that people who know nothing about software architecture could comprehend, in ways that will reproducibly result in relatively fast, easy implementations that are also accurate. This, I don’t think is going to happen either, at least not in my lifetime. I strongly suspect it’s a reductionist fantasy.
Check back in fifteen or twenty years and we’ll see!
Update: For more on this popular topic, please refer to the follow up post.
{ 2 trackbacks }
{ 7 comments… read them below or add one }
People with an eye for quality will pay a premium for the right kind of service – I myself have often paid MORE for a service, just cause I want quality service.
Why else do people still buy Maseratis? :)
@Listerate,
Quality as a unique selling point is difficult in software.
How are most average joe non-technical clients to understand that the code I give them is beautifully written, documented and unit tested, rather than some slapped-together rubbish written for $7/hr? All they have is my word, unless they’re going to hire someone else to inspect the code.
Nice writeup.
Greetings from Brazil!
You described my life as a scenarist. Some people have a Title, and the Title really says it all, and they know who’s going to play in it, and then they have meetings about how much this will cost, how many days they need for filming all this, and another meeting to change the title because they tested the title and it’s not reaching the targeted audience, and they say Kill your beauties, we take the second title. Then finally they agree on changing the title to the first title again because they already put it in the program scheme.
Then they phone me and say, look can you write a few lines, things the actors will say, you know, like you always do?
What ‘s it about ?
I’ll tell you the title, and you’ll say That’s it, you know, the title say’s it all, you just fill in the blanks here and there. And by the way, it’s been turned into a series because we needed a series really urgent, and this was the one everyone felt good about it. Can we meet in about three days, because the actors are really excited about this, and it’s the only day we could bring them aal together.
Bob,
Excellent post. I’ve been a “provider” on Rent-a-Coder for over 7 years. It should actually be called “Rent-A-3rd world programmer for $2/day who didn’t pay anything for the software he’s using to build your “product” even though it’s not open source”.
Maybe that sounds a little “axe to grind”-ish, but frankly when I first signed up, I was genuinely intrigued that there might be a “quick and easy” way to make some extra $$$ via freelance software work. My Brother-in-law, in the wisdom his extra decade in the business over me has given him (me: 19 years in the business, him, he’s older than dirt, the man chews lisp compilers for lunch) said to me “It’s an online sweatshop, you won’t make money there”.
And, he was right.
The most ridiculous example of asking for cloning a highly mature application was someone on GetACoder who wanted a complete identical clean-room clone of Windows XP, with full binary compatibility and everything. He wanted it in a year. There were bids from people talking about how experienced they were for ASP.NET, making bids in the thousands of dollars. It was posted to reddit: http://reddit.com/info/100qw/comments . The comments from the reddit users are kind of interesting; the original bid on the coding site seems to have disappeared.
The only way to make any money on RentACoder is to live in China or India, and even then it’s worse than the Western notion of living paycheck to paycheck.
One possible strategy is to write a code generation system in a more expressive language (more expressive than PHP, that is — Lisp, Scala, F#, Scheme, OCaml, whatever) that can quickly handle CRUD apps where you can bang out something in 30 minutes. There’s a fellow by the name of Alex Peake (waybettersoftware.com) who has done this successfully for quite a few years. He gave a demo at the International Lisp Conference a few years ago where with a few hundred lines of declarative application metadata he would generate about 25,000 lines of C#, which would take about a minute to generate and build. Using expressive languages for source-to-source transform with the target being a popular but crappy language like PHP is IMO an underexploited niche that could be useful for learning more about the domain model. I mean, if you spend 30 minutes on a quick throwaway app where the actual generative software artifact is 150 lines of DRY XML, 100 of which you borrowed from a previous throwaway app, it’s no big deal.
The problem is these bids never ask for just a simple database frontend, they want the fucking moon and stars for less than the cost of a Big Mac, and they literally want it yesterday. What a race-to-the-bottom (in terms of wages and software quality) crapfest these code sweatshop sites are.
I know a guy that made money from Rentacoder. He took those “made me the moon for $200” requests and completed them in a couple of hours. How? He knows how to understand the requests properly, not literally. He uses Delphi, C# and VB, and uses very high-level open source components. He has a repository of code and documentation to cover inmediately common-tasks. He lives in Spain, that’s cheaper to live than USA or UK, but much more expensive than India or the east of Europe. I confess that I couldn’t have done the same work, but I’ve seen him in action, and the result is happy customers.
I don’t understand why you think that software cost can’t be measured or foreseen. I work for a consultancy firm. We do it all the time and it works.
Bob replies: Hi, Rico, and thanks for commenting. I am a consultancy, and I also do it all the time and it works — for a given value of “works”, anyway. The problem is not so much estimating time as what the estimate is based upon. I tend to work with small companies with very specialized problem domains and often they have difficulty articulating what they want. Sometimes various stakeholders disagree on requirements and these disagreements are not always exposed right away. I also encounter a fair amount of scope creep in the form of statements of fact that turn out to be fancy, not to mention the constant churn of changing business requirements. Given that, I generally steer away from clients who want flat or capped pricing, because such clients generally don’t have the commitment to the project to “go the distance” and work through the various adjustments along the way. One of these days I will post more about that. Rest assured I’m not suggesting that you can’t come up with a reasonable estimate at any point in time; but for any non-trivial project it needs to be understood that this is subject to change and refinement because one cannot do 100% of discovery up front in the real world.