 |
| Still from Adam Curtis' 2016 documentary Hypernormalisation |
Software is all about abstraction. We take an existing or desired process, model it and codify that model into a set of instructions and data that computers can read and follow.
We come up with different processes, methodologies, notional lifecycles, workflows, test strategies, measurements and quality controls to steer our immensely difficult projects to a satisfactory conclusion. For that we get teams of people with different roles and skills to work together, manage the creation and deployment of code with its own complexities and dependencies and implement what (we believe) users want is incredibly hard.
The processes we use to create software are also abstractions. The problems we face are, how much do the abstract requirements, process and methodologies we implement really reflect a desired real world outcome? In the midst of a software project, how do we maintain the link between abstraction and reality? What happens when a software team loses the ability to know about the difference between simply fulfilling the process and getting to a real world product people actually want? More dauntingly, at what point will the team stop caring?
Everything Was Forever Until It Was No More
 |
| A stamp featuring Pimenov's "Wedding on a Tomorrow Street", 1973 |
Anthropology and Sociology provide some interesting allegories that within limits help us to look at the questions above. The Russian academic Alexei Yurchak in his 2006 book "Everything Was Forever Until It Was No More: The Last Soviet Generation" discussed the paradoxical nature of life in the 70s and 80s Soviet Union. People were realising the disconnect between the ideological and propagandistic announcements of a Soviet state that saw itself as successful and immutable ("the end of history" as Marx envisaged) and the increasingly obvious fact that its economy was stagnant, its institutions were failing, events such as the War in Afghanistan were hurting its collective psyche and their quality of life was getting worse by the day. However in the absence of an alternative, whilst they understood something wasn't quite right they remained dissonant and carried on with the pretense that nothing was wrong, deluding themselves into a self-fulfilling prophecy. All the while the national discourse was piling more upon itself and becoming even more repetitive, in unwavering copying and reinforcement of earlier dogma. When the Soviet Union did collapse so quickly and dramatically, its disintegration was shocking to most Soviet citizens who had seen it as eternal. Yurchak coined this condition "Hypernormalisation".
The disconnect between the official narrative and the reality of life in the final decades of the Soviet Union gave rise to artistic protests of a satirical nature, with films such as Andrei Tarkovsky's "Stalker" describing a world that was outwardly similar to ours and could fulfill all our dreams but at a deeper level profoundly different and nightmarish.
Yurchak's book had a profound influence on the English BBC journalist and filmmaker Adam Curtis. His impressive 2016 documentary "HyperNormalisation" expressed the belief that this had been extended to the whole world. In his view, from the 1970s onwards our leaders and media figures had grown tired of struggling to articulate and tackle the complex problems of the real world, resigned themselves to managing the current order and predicting risks and created a "fake world" which trivialised long-existent complex social, economic and political issues into half-baked narratives of "Good vs Evil" with misplaced or contrived enemies. Instead of challenging themselves and the narratives told to them, people turned to virtual worlds in cyberspace that were designed to be free but had powerful hierarchies of their own - harvesting the information and activities of billions of people and and filtering the media people saw to reinforce instead of challenging their opinions.
Consequently people either grew to be dissonant of the "fake" perception-managed world and the real world, accepted the narrative as fact and opted into solipsism and self-focused activities or used the internet to create huge protests and revolutions. Those who retreated became susceptible to agents who were skilled in maintaining confusion and instability - the film uses the examples of then-candidate Donald Trump and Vladimir Putin's strategist Vladislav Surkov - turned politics and news into bewildering theatre such that facts and truth no longer became politically relevant and rumours and chaos reigned. This left them much more open towards the status quo (as in Putin's Russia) or demagoguery (as in the 2016 US Elections).
For those who revolted, in the absence of a compelling alternative direction of society these revolutions failed tragically and spectacularly, such as the protests in Egypt that were hijacked by the Muslim Brotherhood, eventually to be deposed by the military, or the Syrian Civil War that led to the rise of ISIS.
The Road to Disneyland. Simulation and Simulacra.
 |
| Pluto, Dale and Daffy Duck on a firetruck, Disneyland Anaheim, 2010 |
Both of the narratives above borrow from postmodernism, particularly semiotics (the study of communication with and assigning meanings to symbols and signs). The French sociologist Jean Baudrillard in his controversial 1981 treatise "Simulacra and Simulation" described the process where human culture progressively replaces our direct experience of reality with a series of signs and symbols (which he refers to as "simulations") - copies and representations of the real world with a seductive quality.
Over time the simulation is tweaked and copied to the point where it deviates from the original to the point where it is a (in his words) an "evil appearance—it is of the order of maleficence". This continues to the point where any relationship with the original reality is semantic and arbitrary at best.
The end point of this, which he calls a "simulacrum", is a simulation or copy that has been abstracted to the point where it has no relationship to an original at all - only to other signs that may themselves be simulacra. Our life experience of the simulacrum, not being based on any grounded physical object, is not even acknowledged or cared to be based on reality - yet it becomes a “truth" in its own right. This artificial existence is known as "Hyperreality".
The most common example stated, referred to by Baudrillard and Italian philosopher Umberto Eco, is Disneyland. The facades of Main Street with its allusion to an idealised past create a facade that appeals to our imaginations and desires. The Big Thunder Mountain ride is an exciting allusion to old mining rail tracks in a cowboy setting. Nevertheless, Main Street as a real street never existed and mining rail tracks of bygone times were not anything remotely like Big Thunder Mountain. Of course, Disneyland is so seductive that none of this matters. We engage with the fantasy world as if it were real. This is the definition of the Hyperreal.
Other stated examples of simulacra include historical fallacies that have been promoted widespread as facts, artificial substitutes for human interaction such as sex dolls and AI chatbots, films with digitally created CGI characters and backgrounds and "structured" reality TV shows.
A Postmodern Perspective on Software Delivery
I came across the work of Baudrillard along with Adam Curtis' "HyperNormalisation" at about the same time in 2016, after about nine years of working as a tester. Whilst drawing conclusions from ideas from sociology and anthropology applied to software teams without thought is a flawed and dangerous activity, I believe that they provide a lens through which our inability to deal with the inherent contradictions behind key concepts in software engineering can result in software teams falling into flawed and misused practices - and why even if they are suspected to be flawed and misused, teams fall into dissonance and fail to act right up to product catastrophe.
For this I offer perspectives on four concepts critical to software engineering and testing - quality, measurement, requirements and methodology/process.
Quality - Relative at Best, Simulacrum at Worst
The usual aim of a software team is to deliver a "quality product", however Quality is a vague and shifting target. Gerald Weinberg, seen as the "Godfather of Agile", wrote in his 2012 blog article "Agile and the Definition of Quality" on the "Relativity of Quality' - "what is adequate quality to one person may be inadequate quality to another." He states the following -
"If you examine various definitions of quality, you will always find this relativity. You may have to examine with care, though, for the relativity is often hidden, or at best, implicit.
Take for example Crosby's definition:
"Quality is meeting requirements."
Unless your requirements come directly from heaven (as some developers seem to think), a more precise statement would be:
"Quality is meeting some person's requirements."
For each different person, the same product will generally have different "quality," "
He list various statements defining quality from the point of view of specific project stakeholders.
"
a. "Zero defects is high quality."
1. to a user such as a surgeon whose work would be disturbed by those defects
2. to a manager who would be criticized for those defects
b. "Lots of features is high quality."
1. to users whose work can use those features–if they know about them
2. to marketers who believe that features sell products
c. "Elegant coding is high quality."
1. to developers who place a high value on the opinions of their peers
2. to professors of computer science who enjoy elegance
"
etc...
His ultimate definition of Quality - that it is "Value to Some Person" - has consequences to agile teams. He states that the definition of quality is "political and emotional" and thus leads to decisions on whose opinions count most. Also these decisions can be less than rational and are hidden from public view.
It is my assertion that in cases where not enough thought has been given to grappling with the contradictions above, a team’s conception of “Quality” has little in relation on the reality of the product (to the extent that the reality can be defined or measured) or the aims of the project. Quality as a symbol or concept becomes nothing more than the biased opinion or dogma of some powerful manager or stakeholder or a vague guess of the desires of some abstract “target market”. However in the perceived absence of anything better, the team acts as if it were the truth. It becomes a simulacrum.
The effect of this is we have started to give up the belief that we can make a “quality" product. James Bach, in his 2009 blog article “Quality is Dead #1", paints a despairing picture -
“A pleasing level of quality for end users has become too hard to achieve while demand for it has simultaneously evaporated and penalties for not achieving are weak…. When I say quality is dead, I don't mean that it’s dying, or that it’s under threat. What I mean is that we have collectively - and rationally - ceased to expect that software normally works well, even under normal conditions. Furthermore there is little any one user can do about it.”
Bach asserts that the result of this disillusion with the possibility of quality is that management have given up and moved towards the lowest common denominator approach - dispensing of good test teams and moving to cheaper and less capable offshoring teams.
“Top management can’t know what they are giving up or what they are getting. They simply want to spend less on testing. When testing becomes just a symbolic ritual, any method of testing will work, as long as it looks impressive to ignorant people and doesn’t cost too much.”
This “going through the motions” while publicly acting as if quality is being improved, as I see it, is a type of mini-hypernormalisation.
The Question of Measurement
Measuring an attribute that does not lend itself to precise and agreed definition and arises from human feelings and politics is evidently problematic and will lead to inconsistent and tenuous results. Rich Rogers, in his 2017 book "Changing Times: Quality for Humans in a Digital Age" writes -
"If stories and feelings tell us more about human responses than cold facts, dates and numbers, this might provide a clue as to why software development teams, and the organisations they work with, sometimes struggle with the question of quality."
"In this field, refuge is often sought in numbers.... By setting numeric quality targets - sometimes called exit criteria - a desired state can be agreed upon: a point at which the product is deemed good enough for whatever follows, including whether it is ready to be released to customers. The numbers act as a comforting security blanket, creating a sense of control. Even if the picture they paint is troubling, at least the provide a means of seeing that picture."
However as Rich Rogers continues to explain, this is an abstraction that while comforting, is limited and prone to mislead. Crude metrics such as counting of passed test cases and defects assume an equivalence between test cases that isn't likely to exist. They also hide the types of investigation and testing outside of these test cases along with defects that were resolved without the need to record.
"Quality is not measurable in the way cost or time might be measured... There is no such unit of measurement for quality."
He makes the case that metrics still have a role to play in discussions about patterns and potential issues. Reports are only useful in triggering those discussions - about what the metrics tell and do not tell.
The Illusion of Requirements
What about requirements? If one takes a list of use cases, desired functionalities and acceptance criteria - and the product contains all of these working to some agreed order - does that mean we have a quality product?
I believe that this is also tenuous at best and a comforting fantasy at worst. Requirements follow the same rules as "quality" defined by Weinberg. They depend on the judgement, political power and knowledge of the person who defined them - as filtered through project stakeholders, business analysts and the printed page.
The person who conceived the requirements, we have to assume, has a correct knowledge of the problem to be solved. Anyone with experience of software development will know of circumstances where that is a bad assumption. They may arise from a high level abstraction of an existing business process, ignorant of the day to day challenges and tacit knowledge of those implementing the process. The person who stated the requirements may be a high level manager without any experience of ever implementing the process, a slave to the reports of subordinates.
The illusion doesn't end there. Various test case management tools allow use cases and acceptance criteria to be linked to one or more test cases, generating requirements traceability. This is also flawed for the same reason that test case metrics are flawed. They make presumptions that quality in a requirement can be neatly expressed by X test cases passed in an associated matrix, or an hour's session based exploratory testing revealing no defects. This is very wrong as any tester with experience will tell you.
Methodologies and Primacy of the Process
In response to the above, there have been created many different approaches to a software development project along with practices within them. Even within Agile we have XP, Kanban, TDD, BDD, Continuous Integration, Scrum, SAFe, DevOps… Even within testing we have competing schools and methodologies. These are all simulations of how teams work in practice - very rarely do teams follow a prescribed methodology to the letter. How one team defines “agile” can have only a tenuous relationship to another even within the same company.
The danger is that the process is deemed to matter at the expense of the outcome, or implemented to provide benefits to the adherents without respect to the project. For new adherents, as Rich Rogers points out -
"The techniques and tools used in carrying out the work can seem exciting, and there is no shortage of new ideas and skills to learn. The desire to be at the forefront of change, or at the very least to demonstrate awareness and familiarity with current methods, can be a powerful motivator in this field. Adoption of new techniques, training in how to use new tools and acquisition of knowledge and skills related to new ideas, or possibly a desire to enhance a resume."
One area where the hype and impetus to learn is already causing an effect is in test automation. The drive to automate more (if not all) tests, and its effects on tester recruitment, causes testers to devote project and personal time to learning automation tools and programming - and to implement them without experience or real forethought. I know from my own experience the waste and effects of poorly implementing an automation strategy based surreptitiously on a desire to learn a new skill in a commercial context.
Another danger is that processes are defined at the company level and not the team level. Because they were believed to work before, we enforce processes and methodologies on other projects in an effort to "standardise". This may make sense at a management, resource allocation and control level, however projects with ill-considered, enforced processes risk great failure.
Leaky Abstractions in a Complex World
If we understand the flaws in defining quality, measurement and requirements, and our methodologies are idealisations, why do so many teams still persist in making critical project decisions based on their flawed understanding of them? Is it purely ignorance or something else?
Rich Rogers makes the point that metrics, however flawed, act as a comforting security blanket, providing some basis for decision making. We may not be measuring the right thing but we are measuring SOMETHING that may or may not be close to quality.
In the same way, requirements may be incomplete, flawed or at the extreme end utterly irrelevant to the problem we wish to solve, however they do exist. One can use them for contracts, resourcing, planning, development, testing and delivery. In agile methodologies we can continually add changes until (we hope) the end product approaches something the user or stakeholders might want.
In any case these, along with working definitions of quality, are all models of the "real" state of a project or product. For better or for worse they are vital in reducing a development project from something almost ineffable to something that can be planned or achieved. I treat them as "simulations" as defined by Baudrillard. Because of this, they are greatly seductive.
What matters is their relationship to the real state of the product and the development process.
Hyperreality and the Failing Team
My conjecture is that in projects that are dysfunctional or fail to provide what we call a quality outcome, these simulations are so far removed as to be devoid of reality. In the same way that Disneyland is a simulacrum, so are these.
However, simulacra are greatly seductive simply because they are easier than embracing the difficult to manage real world they pretend to describe. Hyperreality is not real but its sway on team certainly is. Also, changing approaches and mindsets within teams and at the company level, especially without either the privilege of management support or a clear narrative of a better solution and how to get there, rates from difficult to impossible. The closer to the project deadline, after so much has been invested and with the probably painful consequences of a late or failed delivery, the less the team will be willing to experiment and the more the team is likely to act is if nothing is wrong. Status reports show green, problems are ignored, process and the simulacra are gospel - and then the Soviet Union collapses.
Nevertheless, as with the final decades of the Soviet Union, it is probable that a few team members at some level know that something was wrong with the system even if they could not articulate the problem and don't see an alternative. They may follow "best practice" to the letter, see great test coverage as defined by the flawed metrics above, defects resolved, requirements ticked off but their stakeholders still complain, or their clients find defects that are tangential to or don't make sense based on the requirements they worked with. Everything is said to be running smoothly but everyone and everything is stressed. This is what I regard as HyperNormalisation - hyperreality with a subtle chink in the armour.
In companies that are greatly hierarchical, have established and immutable processes and teams with egos and dominant personalities reliant on the status quo it is likely that the unnerved team members above, especially if not at senior level, will keep quiet. They may decide that "Nobody got fired for just doing their job", or patiently wait for the end of the project. They may ask to be moved to another project or if the stress becomes an issue take sick leave. They may go through the motions of work without committing fully to it. These are a protest of a sort, but all they do is accelerate the eventual collapse and slump in quality.
The unfortunate truth is that without some shock that forces the team to reflect on its assumptions and processes, most teams trapped in a seductive but vicious hyperreality are not likely to be self aware enough to change until the project suffers dramatically or collapses entirely.
Epilogue
In this text I have covered concepts of hypernormalisation, simulacra and simulation and hyperreality and suggested them as analogies to look at four key concepts in software engineering - quality, measurement and requirements and methodology/process - and their impact on software projects. I have asserted that our inability to understand that these are ultimately seductive simulations with varying levels of relationship to the real world, and the difficulties of teams to reconcile these, cause great risk of projects deluding themselves into poor products and failure.
In Part 2, to be completed shortly, I look at applying these concepts to the IT industry and our tech culture as a whole, and look into various ways suggested to prevent or rescue teams from seductive self delusion. In the meantime, I would be grateful for your comments and considerations.
