Tech 4D

“Candy Mountain! Fill me with sweet sugary goodness” – Charlie the Unicorn

In a previous post, I wrote about what I learned from an undergraduate engineering design competition. In this installment, I explain why I think that most engineers and software developers don’t know how good they are. In a future post I will show why I think most of us are rarely as good as we think and suggest some methods for improving our average.

While the example is drawn from a bunch of mechanical engineering students (myself among them), it applies equally well to any technical creative endeavor; especially to software development.

Why you don’t know how good you are

The superficial results of the coffee can competition were:

• The ramp stopped most entrants: only three or four successfully completed the course
• The difference between the fastest entrant and the next fastest was an order of magnitude: 7 sec, 65 sec, 240 sec

But that only tells part of the story. Table 1 ranks entrants by other key design factors, with 10 being the best and 0 being the worst. I’ve created six design archetypes into which most of the designs fit neatly.

Table 1 Design Factors

Based on these rankings, you can create a weighted average in order to quantify the success of each design. Table 2 shows results weighted towards “test results”. In other words, “time to complete” is more important than other factors such as cost and reliability.

Table 2 - Weighted Average Test Results

If you weigh “Time to Market” (e.g. how many hours it took to build and test) you would get something like Table 3.

Table 3 Weighted Average Time to Market

I have identified a few patterns of failure (sometimes called anti-patterns) in the various entrants:

• Failure to “Paradigm Shift” :Turbine, Flywheel, Rubber Band
• Poor execution : Other electric vehicles
• Gold Plated : Delorean
• Good job! : Buggy
• Great Job! : Oatmeal

My entry was in the Failure to paradigm shift group. I think all of the terms are self-explanatory.

No matter how you weight the engineering factors, the oatmeal box was far superior to other designs. It took less time to build, cost less, had less parts, was more reliable and it was faster. It was obviously a great solution. But what about the other successful entrants - were they good or great? More importantly, when you finish a project, how will you know whether you did a great job?

The saying goes, “Great is the enemy of good enough,” but in this discussion great is exactly equal to good enough. More specifically, the best engineering solution is the simplest one that meets the requirements. So it should be a simple matter of making a few tables (like above) and adding some weighted factors and you will know how you did, right? Unfortunately, there are two major problems with this.

Firstly, if you are doing something new or inventive, you can never know all the requirements because your customers can’t articulate what needs to be invented. Malcom Gladwell does a great job of discussing this type of thought process in his book Blink: The Power of Thinking Without Thinking.

Secondly, in real life we aren’t all given the same set of rules and asked to solve the same problem, so it is often difficult, if not impossible, to find any concrete “competition” with which to compare ourselves. For example, there is no direct comparison for an internal IT project for a customer service application. If it is over budget and one year late, how do you know whether the planners were too optimistic or the implementation was sloppy and inefficient? Conversely, if it is perfectly on time and budget, how do you know that it didn’t cost ten times what it should have cost?

Here are a few more ways you might be blinded from the truth:

• Project Hell. Often things go so badly wrong that there is no sorting out what actually happened. When this happens, it usually makes no sense to try to make sense of the disaster.
• Lax standards. Sometimes people don’t care whether they got the Oatmeal Box or the Delorian, even though they should care.
• Technical Obfuscation. Technical people are people too. As in any population, there are always a few who will hide behind technical jargon and plausible but false explanations.

For all of these reasons, the sad reality is that the vast majority of engineers and software developers will rarely know whether what they created was great, good or bad.

Fortunately, there are ways to deal with this problem, which I will blog about later.

Engineering Goodness, Part 3

Engineering Goodness, Part 1

This entry was posted on Tuesday, June 5th, 2007 at 11:53 am and is filed under technology. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.