Solving the Software Problem

Software failure is fundamentally a human problem, not a technical one. Purely technical solutions fail to effect truly meaningful and lasting change.

Jonathan Swift

A Shot Across the Bow

I aim to solve what I call The Software Problem.

"Which software problem," you ask, "There are so many!" Well that is my whole point: there are so very many problems with software that software in itself has become A Problem which desperately requires a solution.

As I write this, in March of 2010, The Software Problem is a significant burden on society. While I have not yet tallied the figures, I expect that software bugs, security holes and poor user interface design together cost the world economy at least one hundred billion dollars per year - every year.


Human intervention is no longer required to lift your credit card number: organized crime is now able to steal in a completely automated way. Billion-dollar spacecraft are lost because of dumb mistakes any child could have prevented. Things explode. Airplanes fall out of the sky.

People Die.

Completely innocent people have gone to their graves because some software engineer couldn't be bothered to test his code. I expect that many more will join them before The Problem is solved.

What's worse: maybe the engineer did want to test his code. Most coders I know really are conscienscious people. Maybe he was unable to test his code because he didn't have the support of management. Maybe he tried to institute a Formal Software Quality Process at his company, but his pleas fell on deaf ears because the management didn't regard quality assurance as a profit center.

I can hear it now. I have heard it so many times. Just about all of my colleagues in the software engineering profession have heard it as well, repeated over and over and over again for decades throughout the entire world-wide software industry:

I pay you to implement features, not to fix bugs.

At a recent meeting where my coworkers and I were discussing software quality, I said:

It is plainly apparent to me that the problems of software quality are fundamentally human problems. Every software problem could be solved if we all changed the way we think, the way we work, and the way we interact with each other.

But when I try to point this out, all that happens is that I piss everyone off. Nothing ever changes.

I said then and I say to you now that I regard that insight - that software problems are fundamentally human in nature and not in any way technical - as the most important insight of my twenty-two year software engineering career. But as you can see, that insight didn't get me all the way to where I needed to be. How to convince my colleagues to implement the human solutions?

I puzzled over this question for quite a while, and discussed it with many of my coworkers. I am ready to propose a solution: I will give you no choice.

I will solve The Software Problem by so fundamentally altering the economics of software development, as well as the laws that pertain to the software industry, in such a way that if you don't fix your quality problems - and keep them fixed - then...

You will find yourself unemployed.

The Self-Appointed One

Why Me?

No one could dispute the solution's value, but actually obtaining it is clearly a task of epic proportions. What makes me so special, to think that I can solve The Software Problem?

The very existence of certain kinds of problems demands solution: a drowning child, or a critically wounded victim. Just being there demands one swim to the rescue, or render first aid.

Thus my awareness of the seriousness and pervasiveness of The Software Problem demands I solve it.

But many such problems demand solutions. Why this one for me?

Suppose you and I witnessed a drowning near an accident scene. After a quick discussion to establish my strength as a swimmer and your medical expertise, I would dive in while you staunch the bleeding.

My coding is largely self-taught as my degree is in Physics, not Computer Science. Many software jobs that others find easy, I don't know how to even approach. But I have known almost since the beginning that I am one of the very best at one software job; only recently have I come to understand why:


I can find and fix the bugs that no one else can. While I have met others who match my insight into the nature of software faults, they are few and very far between. Thus, if I am to solve a problem at all, I can make the greatest contribution by Fixing All The World's Bugs.

Just don't ask me to code up an eCommerce website. I would not have a clue.

Chosen Battles

Recall my statement:

Software failure is fundamentally a human problem, not a technical one. Purely technical solutions fail to effect truly meaningful and lasting change.

That I have such a big mouth is not any kind of grandiosity, but my foolish way of biting off far more than I could ever chew: I aim to solve not just one Software Problem, but three. Each is a fundamentally human rather than technical problem. Technical "solutions" exist that are helpful and productive, but that ultimately fail to yield real satisfaction. I aim to solve all three through work with people rather than any kind of machine or tool.

All are very real Software Problems, all with very real Bugs. Some of the Coding Errors in each kind of Problem can and often do take the lives of innocents. Just one kind of Software runs on electronic computers; the other two run on computers made entirely of meat. We have:

I have such insight into the nature of software faults because I struggled with all three problems long before I wrote my very first line of FORTRAN back in 1976 at the tender age of twelve. No, my diagnostic training began one passionate night in the Spring of 1963 when my father's seed found purchase in my mother's field. As far as I am able to tell, G-d Almighty Himself sequenced my very DNA as a tool for debugging in much the same way as Seymour Cray designed Emitter-Coupled Logic as a tool for cracking Soviet codes and designing Hydrogen Bombs.

No... not grandiose at all... Not by any means.


While all three Problems are quite different, there are certain common elements such that solving the three together will be much easier than solving each of them separately.

My clever plan has an easy part and a hard part. While far from complete, you're looking at the easy part: my web site's contribution to the Solutions of all three Problems is to educate you all about each kind of Software Problem, to convince each and every one of you that Solving these Problems is worth your own personal time and effort, and for my colleagues, the practitioners of each Software craft, my site aims to teach you to implement the actual solutions.

The hard part is not up to me, but you: armed with the education and the conviction I have given each of you, your job will be to compel those who publish computer software to adopt the very highest standards of quality. Most will do that by making wiser software purchasing decisions.

Your job will be to compel the delusional to find within themselves the courage to face the reality they fear so desperately.

Your job will be to compel those who would conquer, those who would terrorize, those who would send their Nation's bravest into Harm's Way for whatever reason, first to lay down their arms, then to beat the lot of them into plowshares.

Those of you who are government officials will use the force of law or of governmental regulation to compel each of these things, in such a way as to put immediate ends the more egregious abuses.

Crazy? Yes, it sounds that way. I assert it's not crazy at all. It's the only rational choice we have given the Problems we all face.

Unreasonable? Without a doubt! If you don't know me personally, just ask anyone who does: I have no problem whatsoever with being unreasonable.

I have not yet begun to be unreasonable!

I am certain that each type of Problem - the Problem of Software Reliability, Security and Ease of Use, the Problem of Madness, and the Problem of Humanity's Endless, Murderous Wars are each the result of conceptual misunderstandings. Bugs not in our code, but in our minds. Each can be fixed, and fixed forever, by each and every one of us applying the process of debugging not to our code, not to anyone else, but to our own selves.

I am writing down absolutely everything I know about debugging not just Software, but the Human Mind as well as Human Society. Far more difficult will be for me to somehow commit to The Written Word the method or methods by which you folks may, when called upon to do so, not merely perform the kind of work that I do, but experience what it is like for me when I do that kind of work.

Far more than any diagnostic tool, debugger or technique, my unique insight into the nature of software faults is my own very special sort of Madness. I really did get my start as a Debugger at the instant of my own conception. I am quite proud to proclaim that I have been utterly and irretrievably Batshit Insane since long before the day I was born. It is a symptom - an adaptive, or positive symptom - of that very Insanity that makes me such a good Debugger.

Your own personal Bag of Debugging Tricks requires but one Trick: find some way to adopt my own personal point of view and even the most intractibly complex bugs will drop like... dare I say it? Flies.

The Five Virtual Machines

To solve each of the three Software Problems, we must first understand the structure and function of, then learn to operate each of the Five Virtual Machines.

Two of the Five Virtual Machines are known to every software engineer; the Third is familiar as well, but naming this Third Machine as one of the Five Virtual ones is, as far as I can tell, my own invention of a little over twenty years ago. While unfamiliar to my fellow coders, I expect I can make a case for its inclusion in our set.

The Fourth and Fifth Virtual Machines are familiar to everyone - not just software engineers - but at the time of this writing I am certain that I am the first to describe these two as any kind of Machine. I expect my colleagues to dispute their mechanical nature, but experts familiar with their operation, while new to my terminology will quite likely agree that my terminology is appropriate.

There are other kinds of Virtual Machines, far more than just these Five. But these Five should provide all the Machinery we require to solve each of my three Software Problems.

What is a Virtual Machine exactly? How is it different from a Real Machine? How is it the same?

Virtual Machines are like Real Machines in that both are constructed of components that interact with each other in well-defined ways. Useful machines - neither Virtual nor Real - are not built by haphazardly sticking metal pieces together; there is some logic to the choice of each component, just as there is logic to the connections between components that interact.

Virtual Machines are idealizations, representations of physical machines that suffer few if any of their limitations. There are limits to the accuracy with which pieces of metal may be crafted; every mechanical engineer must deal with such limitations in every design they create. Metal gears that mesh firmly and smoothly when first assembled will later wear, thus developing slop and backlash. Friction where parts rub together generates heat, which may cause the parts to expand and jam the works up completely.

Electronic circuits are Real Machines in this sense, just made of different parts. Electrical Engineers must concern themselves with heat dissapation, power consumption, parasitic capacitance and magnetic induction, package size and robustness in the face of heat and vibration.

Virtual Machine designers deal with few such considerations. In a very real way, we can design our Virtual Machines as if the Laws of Physics simply did not exist. The limited accuracy of his machine tools prevented Charles Babbage from ever getting all the mechanical flaws out of the Analytical Engine. It would be a lot of work to implement, but the Analytical Engine's mechanical design could be completely implemented in software, and so made to run not just flawlessly, but forever, without ever wearing out.

How do we make Virtual Machines? They are often machines-within-machines; that is, a Virtual Machine is constructed within a Real machine in such a way that the Virtual Machine suffers few if any of the host Real Machine's limitations.

Real Machines are made of nuts, bolts, gears, levers, capacitors, transistors, wires, inductors. The components of Virtual Machines are composed of data structures; the interaction between the data structures is implemented in software code.

Thus we see why Virtual Machines need not obey the Laws of Physics: they are composed purely of Information, completely without any manner of corporeal form! Gravity doesn't pull Information downward when you drop it. Information has no inertia to resist when you push it. Information doesn't wear out, doesn't vibrate, doesn't overheat and never needs lubrication!


There's a problem...

In reality...

None of that is true!

I lied! Straight up: I just told you a bald-faced lie!

While it is easy and convenient to forget that Information obeys the Laws of Physics, in reality, it always does. Every bit and every byte and every opcode in every manner of computer program that has ever existed or ever will exist obeys each and every Law of Physics just as slavishly as the Apple that bopped Sir Isaac Newton's head back in the day.

Thus we have the first Trick that I will share with you from my Debugger's Bag of Tricks:

Every Bug, whether it be Software, Hardware, Mental or Social, has a completely rational explanation, that can be expressed purely within the framework of the Physical Laws. There are no exceptions to this rule whatsoever.

I grant you: actually producing this rational explanation may be far beyond the reach of any Human intelligence. Yet, just knowing that such an explanation exists out there, somewhere, may really be all we need.

More Coming Real Soon Now... Stay Tuned...