Blog

/ˈblɒɡ/

Lack of Falsifiability in Software Engineering

06.06.2024.

Since the first half of the 20th century the ideas of Karl Popper have fundementally changed how we think about the natural sciences. His main idea in the area of Philisophy of Science was an answer to the so called "Demarcation Problem". That is to say, how do we distinguish between things that are science—and should with that carry all the good things we impart to scientific knowledge—and things that are not. Namely, the idea of falsifiability.

Falsifiabilty is a deductive principle, that according to Popper, is a necessary—albeit not sufficient— condition to determine if a proposition can be considered scientific. Inasmuch as a proposition has the logical property that it can be proven false, can this idea be thought of as falsifiable. On the other hand statements that by their mere construction cannot ever be proven false are unfalsifiable, and pejoratively designated as pseudo-science. There is much pseudo-science in Software Engineering.

Perhaps an example is necessary to crystallize this concept. A canonical one is Russell's teapot:

If I were to suggest that between the Earth and Mars there is a china teapot revolving about the sun in an elliptical orbit, nobody would be able to disprove my assertion provided I were careful to add that the teapot is too small to be revealed even by our most powerful telescopes. But if I were to go on to say that, since my assertion cannot be disproved, it is an intolerable presumption on the part of human reason to doubt it, I should rightly be thought to be talking nonsense. If, however, the existence of such a teapot were affirmed in ancient books, taught as the sacred truth every Sunday, and instilled into the minds of children at school, hesitation to believe in its existence would become a mark of eccentricity and entitle the doubter to the attentions of the psychiatrist in an enlightened age or of the Inquisitor in an earlier time.

Another example from pop culture is as follows: If I encounter a rock on the street, and claim that it repels tigers. I could sit there indefinitely waiting for a tiger to come attack me, continiously collecting positive evidence of the proposal. And after a few years of having no feline encouters, I could conclude that it is indeed, an effective tiger repellent. This is would be a positivist and inductivist approach to knowledge. On the other hand somebody aware of falsifiability would take the rock to the nearest Zoo and enter a tiger's cage. The immediate negative evidence would very effectively disprove the proposal.

The first approach strongly resembles an area of Academia currently facing a replication crisis. Whereas the other resembles an area of Academia that has constantly provided excellent results, shedding light on many of the big unknowns of the universe. It is my opinion through observation, that modern Software Engineering looks more like the first approach than the latter. Obsessed with unmeasurable metrics, unfalsifiable propostions, and subjective values, we are drowning in a sea of ideology, non-reality-based aesthetics, overly complicated systems, and terribly, terribly slow software.

The "Clean code" ideals are a succint offender. Appealing mostly to vague, unverifiable, and unfalsifiable measures of how good code looks, or how easy it is to read and understand. It forgets the physical foundations of our computing systems. Preferring subjective appreciations of "cleanliness", supposed to make the practice of software programming more appealing to the engineer, as opposed to trying to provide the best possible technology to the users.

Without a serious set of reality-based values in our programming culture, derived from the relative youth of our field, and its explosive growth, and seemingly endless room for cultural, and industrial application. It's as if we are a thoughtless herd of grazing animals, moving in seemingly random directions, feeding on hardly distinguishable bits of dopamine-inducing grass, and the next big JavaScript framework. Rather than being worried about the performance and correctness of their products, most people seem concerned about what text editor they write with, or which color theme some Twitch streamer is using.

This problem seems consistent with a closely related phenomenon also plaguing our industry. A distinct lack of care for the non-accountable negative externalities of our programs, that cost humanity an uncountable amount of time waiting for loading bars, and for endless lines of code to finish meaningless work, that do nothing other than shuffle papers around the desk, and that no user wants. Or that also cost our species, hundreds of Tera Watthours in unnecessary CPU execution, for similar reasons.

If we were all more aware of falsifiability when thinking about software, when thinking about what software to use, and what software to write, we could all come up with much better guiding vectors, that would result in empirically measurable, and better systems. Maybe then we could correct the current course that software seems to be on. It appears as though software speed has a derivative more negative than hardware speed positive. And this will only get worse, if we keep forgetting that programmer comfort is unfalsifiable, and program speed falsifiable.