QA Research and Tackling The Causes of Defects - A CEO's Perspective

I often find it difficult to articulate what testing, bug detection and quality assurance means at the C-suite level. I know for a fact that I am not the only one and the inability to articulate our benefits to the bottom line is a big problem among testers. We know that it reduces cost of rework and increases customer satisfaction (if done properly), however how do you quantify or articulate that to the people at the top? Also, would the CEO of a large company, say a multinational, really understand what tackling bugs and why they happen does not just to the health of products and services but also to the overall health and strategy of the business? Would the CEO understand why it is a priority deserving of major R&D, and be proud to discuss it with investors?

Then I found an example of a CEO who, for want of a better word, "gets it". I watched a recent interview with Richard White, the CEO and founder of the Australian logistics software company WiseTech Global. The part where he talked about quality enhancements (12:00 onwards) as their major R&D factor is impressive.



What made this stand out to me was that when asked about R&D and having the edge on competitors, he spent time talking about the continual work done with his CTO to find out why defects occur and waste occurs in software development. He was proud of their continual work in "squeezing out" the failure points and cleaning up technical debt (something that I have never heard a CEO even mention in an investor interview, and something that is de-prioritized often in favour of working on new features).

To me, the most powerful thing he said was that he saw their investment on the above and the plummeting defect rates as providing a higher "yield" on each dollar spent, using the example of a 32% yearly increase in product updates done with just a marginal increase in staff.

This is a tech company thus software development is the core of their work, however there have been more than a few tech companies that have neglected quality assurance. What this shows is a CEO and technology team who see QA as a major priority to the bottom line and have taken steps to quantify it, allocate R&D resources to drive defect rates down and then be able to present and proudly talk about it on a show aimed at investors.

If a CEO can articulate the value that working on QA and defect resolution and prevention can talk about it so proudly, precisely and articulately, we testers should too.

On “Testing vs Checking” and other Oppositions - Dialectics and Deconstruction

Software testing appears to be riddled with contradictory narratives argued from opposing sides. We argue about what is "testing" vs "checking", "traditional" vs "modern" testing approaches, "scripted" vs "exploratory" and make arbitrary distinctions to argue about "automated" vs "manual" testing. In these cases, the opposition is not just based on semantics and meaning but also on value (i.e. that one is wholly “better” than the other). Despite efforts of testing practitioners to debate these topics online or at conferences it is arguable that the arguments on either side are not significantly more sophisticated than before and no more likely to be resolved by consensus. In the second of my blogs touching on postmodernism and testing, I look at the nature of oppositional debate and touch on why that could be.

I also, through concepts developed by the philosophers Hegel and Derrida, theorize that we can question the oppositions above when we see them in a text by a process of dialectic and  deconstruction to develop more sophisticated, contextual and progressive propositions.

On Dialectics, the study of Discourse

The reasoned discourse between two opposing arguments being debated is known as a dialectic, a philosophical term dating from the classical Greek period of Socrates and Plato. Among the earliest noted examples are the Socratic dialogues, where statements are clarified via questioning and their logical consequences studied until a contradiction arises that refutes these statements. Socrates' aim was to find truth, therefore he had no qualms about changing his arguments to further that aim. It is closely related to rhetoric (the public defending of beliefs and attacking opponents) but seen as a counterpart to it (notably by Aristotle). In the middle ages it was included within the teaching of Logic.

In modern times the dialectic was revived by the great German philosophers JG Fichte and later, GWF Hegel. Based on ideas by Immanuel Kant, Fichte described the dialectic as the triad Thesis-Antithesis-Synthesis (later misattributed to Hegel), with the definitions below -

• Thesis - Some Statement or Proposition

• Antithesis - A statement that negates the Thesis

• Synthesis - A process in which the Thesis and Antithesis are reconciled to form a new proposition.

In Fichte’s definition, the Synthesis could be used as a Thesis in a future dialectic. The continual synthesis of thesis and antithesis in a chain would constitute more sophisticated understanding and progress.

Credit to https://questians.wordpress.com/2010/12/14/a-dialectical-approach-to-the-bible/

Example 1: Hegel's Being and Nothing

Hegel's book "Logic" provides the following example regarding existence.

•Thesis - Existence must be posited as pure Being

•Antithesis - Pure Being, upon examination, is found to be indistinguishable from Nothing

•Synthesis - When it is realized that what is coming into being is, at the same time, also returning to nothing (in life, for example, one's living is also a dying), both Being and Nothing are united as Becoming.

Example 2: Marx’s Theory of History

Marx and Engels took the Hegelian dialectic from a context purely of ideas to a materialist context (that of the real world of production and economics) to denote how they saw change and progress would develop in societies. In Marx’s theory of history, the thesis and antithesis were different classes within society and the synthesis would be societal transformation resulting from inevitable conflict between these classes.

Credit to https://www.pinterest.com.au/pin/346073552589181632/

Each of the above involves some overcoming of the negation (antithesis), resulting in keeping the useful parts of an idea to develop a more sophisticated proposition (which Hegel called Aufhebung or Sublation). Hegel stated that this "negation of the negation" would result in incorporating the opposing idea into itself. In doing so this was the foundation of the Hegelian dialectic, providing a counter-approach to the skeptical Platonic dialectical method of “reducto ad absurdum”, where following a contradiction the thesis would be rejected, leaving us with nothing. Hegel believed that contradictions were a necessary outcome of reasoned argument, thus only a synthetic approach would bring us to a complete truth.

Semiotics - A Dualist Notion of Words and Signs

Later in the 1800s, Ferdinand de Saussure, the so-called Father of Linguistics, proposed a structured interpretation of the use of words, signs and utterances. He called the word or phrase used the "Signifier", and the concept or object referred to by the Signifier the "Signified". His great contribution was to note that the Signifier did not necessarily have to have a connection to the Signified - this is completely arbitrary and the brain acts to combine the signifier word to the signified concept based on some customary or agreed relationship. In doing so, the brain synthesizes and provides meaning to the sign. This is a sort of application of the Hegelian Dialectic. The study of the relationships between and use of signs and words as described above is known as Semiotics. For the purposes of this essay, “word” and “sign” will be treated as synonymous.

How can a word and its meaning have no actual connection? Saussure made the case that words only have meaning relative to other words. Therefore the word "tree" only has meaning when distinguished from the words "bush" or "shrub" - only having use as part of an overarching structure of synonyms and antonyms. Also the relationship between Signifier and Signified changes over time, such that some words become dated as circumstances change. In this respect, Saussure was considered a pioneer in the field of "Structuralism".

Derrida - Spot the Différance

Jacques Derrida

In 1963 the great French Postmodern philosopher Jacques Derrida took de Saussure's ideas to the next level. He stated that signs (including words) can never in themselves denote what they mean. They can only do so by deferring to a potentially endless chain of other words (signifiers) that they differ from. Also our understanding of word meaning changes with each reading and over time with the changing definition of existing and new introduction of new words.

This results in what he called "espacement" or spacing, stating that this differentiation causes binary oppositions and hierarchies.

Derrida coined the term "différance" (sic), a deliberate misspelling to denote not just the difference between words but the concept of hierarchy and deferral (to defer in French is "differer"). It includes the fact that the word can only have meaning within the context of the text ("there is no outside text").
According to Derrida, this chain of signifiers will never get to a point of a complete meaning that transcends context and is understood at all times and by all parties (the so called "transcendental signified", used in philosophy as an ontological argument for the existence of God).

However, he pointed out that the opposition of words and signs with others express not just meaning but values.

"On the one hand, we must traverse a phase of overturning. To do justice to this necessity is to recognize that in a classical philosophical opposition we are not dealing with the peaceful coexistence of a vis-a-vis but rather with a violent hierarchy. One of the two terms governs the other (axiologically, logically etc.) or has the upper hand.

To deconstruct the opposition, first of all, is to overturn the hierarchy at a given moment, To overlook this phase of overturning is to forget the conflictual and subordinating structure of opposition."

Binary Opposition

Many word pairs in binary opposition have not just logical but also value oppositions depending on their use. Examples include "Christian" and "Pagan", "Left" and "Right" (philosophical/political sense), "Speech" and "Writing", "reality" and "fiction" and "reason" and "passion". Different authors and readers at different times will value one over the other. Depending on their understanding by the reader, their definition at the current time and their use in the text, the differences in meaning and value between the two are variable and changing. Derrida called these discrepancies "traces", thus stating that a word not only denotes its own meaning but, by the its partner in binary opposition, what it does not mean (which he referred to as the "absence of the present").

This "deconstruction" was the ongoing practice denoted by Derrida for all texts to be studied for conceptual and temporal oppositions and these broken down to develop new meaning. It formed his life's work and is heavily influential in postmodern philosophy, law, linguistics, LGBT studies, psychoanalysis and literary theory. Note that the structures of word opposition remain and are required for any sort of communication, however the act of deconstruction allows for them to be broken apart and the gaps between the implied sense of each word in opposition to be analysed and understood within the context of the text.

It must be stated that according to Derrida, unlike Hegel's Dialectic, these oppositions (Thesis and Antithesis) are irreducibly complex and unstable. They could be understood and their différance noted and determined, however they could not be synthesised into a whole as Hegel proposed.

"The oppositions simply cannot be suspended once and for all. The hierarchy of dual oppositions always re-establishes itself. Deconstruction only points to the necessity of an unending analysis that can make explicit the decisions and arbitrary violence intrinsic to all texts."

The above ideas are controversial and have been disputed (particularly by John Serle and Alan Sokal) however they can be used as a model to approach the contradictions and arguments stated in my introduction.

Applications to Testing Discourse

By applying the concepts above we can make assertions about the oppositions so heavily debated in testing. The pairs "testing" vs "checking", "scripted" vs "exploratory" and "automated" vs "manual" testing, in the context of how they are usually debated, are treated as binary opposites. As stated in the introduction, arguments around them also imply not just logical or semantic opposition but value opposition (i.e. that one is "better" or more worthy than the other).

An Example - Testing vs Checking

Let us take a look at the first example, “testing” vs “checking”.

Personally I would define “testing” as a systematic exploration and investigation of the application, less algorithmic and more a constant thinking process. I would equally define checking as a more systematic procedure of experimentation and comparison of a scenario outcome with some expected result or test oracle, then moving on. However my definitions are meaningless without appeal to the implied meanings of the other words in my propositions above - and the words they are related to. I also admit that the simple words “testing” and “checking” are probably inadequate in explaining what I had in mind. In taking sides or commenting on a debate of “testing” vs “checking” it may be that I state or imply a hierarchy of value - i.e. that “testing” is better (or worse) than “checking”.

"Testing" defers meaning in its relation to other words such as "exploration", "analysis", "scrutiny", "thinking", "investigation" - which themselves can only be defined by relation to others. Equally the word "checking" also only has meaning in its relationship to other words such as "control", "find out", "learn", “comparison” - each deriving meaning by relation to other words. At some point the "web of language" (another term defined by Derrida) may overlap and both words may derive meaning from at least some other common words, however this does not necessarily mean that they are the same.

They are (contentiously) defined by what they are not. "Testing" is "not" "checking" (same as "Automated" is not "Manual" etc). Their meanings cannot be determined outside of the context of the text they were written in, the time they were written and the fickle views and understandings of the writer and reader. No transcendental meaning exists outside of this.

These represent a dialectic, and from a Hegelian perspective an effort should be made to synthesize them into a whole. However a deconstructionist perspective would dictate that these differences are necessary (at least until some new words are developed with more sophisticated meanings) and irreconcilable and contain an inherent violence of meaning and value. This may explain why after many years we are still having angry debates online and at conferences about these oppositions.

So what can be done? The way to address this is to deconstruct each use of the words in their context and time.

• For example, when we see the word "testing", does this mean a specific incident of testing or the general practice?

• Instead of testing, what other words can be used to describe what the practitioner is doing? Is the word "testing" adequate?

• Who is apparently doing the "testing" - one person, a machine, a group of people, a context driven tester?

• When was the text written and how was testing defined at that time?

• Can a single word apply to all investigative approaches in every company? What does the author think of testing and what other words are used to describe it? What about the reader?

• Are there instances where "testing" and "checking" could mean the same thing? What may these be?

• Regarding the "value" of testing vs checking, who says that one is better than the other? How is it, and under what circumstances can that hierarchy be overturned?

• What "testing" are they referring to, based on the questions above, and in what context? Has one always been better than the other?

What can those who engage in the debates do to clarify the points above to the reader? One way is to publicly state clear and sophisticated definitions of “testing” and “checking” as one sees them. James Bach and Michael Bolton, in their co-written 2013 article “Testing and Checking Refined”, go to great lengths to do just that - along with their definitions of related words that would fall within each one’s “web of language” (such as evaluation, exploration, learning etc.). Also included are narratives of the definitions along with various implications.

“Testing is the process of evaluating a product by learning about it through exploration and experimentation, which includes to some degree: questioning, study, modeling, observation, inference, etc.

(A test is an instance of testing.)

Checking is the process of making evaluations by applying algorithmic decision rules to specific observations of a product.

(A check is an instance of checking.)“

However, while this is better, from Derrida’s perspective the extensive web of language behind these definitions and their related words, in the specific text and time they were written in, will always prevent these definitions from being perfect and unchanging, the basis of a founding philosophy of software testing.

Michael Bolton, in a follow-up piece on his blog, also attempts to take the hierarchy of value out of the debate. He makes it clear that his (and James’) side of the debate is to reconcile checking in a more complex definition of testing that embodies but distinguishes between the human, reflective and thought driven tasks and algorithmic/mechanical and process driven tasks.

“I would like to emphasize our goals here. Our purpose is not to denigrate checking, nor to disparage the use of tools, nor to deplore those people who are asked to do human checking. On the contrary: we’re attempting to deepen our understanding of our craft; to show that checking is deeply embedded in testing; to emphasize that tools and the skilled use of them are essential to our work in many ways; to realize that humans will always inject human elements into the things they do; to realize the value of those human elements and the risks involved in asking humans to behave like machines. We must be clear on the differences between what humans do and how our processes and tools—media, as McLuhan would call them—do. Or more accurately, the differences between what we do and how our tools affect what we do.”

This removes the opposition entirely by making checking a subset of testing - arguably a successful Hegelian, synthetic approach.

Epilogue

The questions posed in the example of the binary opposition “testing” / “checking” can be applied to texts containing the other contentious testing oppositions mentioned.

The process above is necessary, if not to remove or synthesise oppositions out of the picture, to create propositions and assertions with more contextually accurate and sophisticated meanings. With the debates we are currently having, unless we deconstruct the concepts we are using in the texts they are written in we are arguing over shadows. Then can we make headway in some of the tense binary oppositions that have dogged the testing profession for many years.

Useful Links and References

Dialectics https://en.wikipedia.org/wiki/Dialectic

Thesis, Antithesis, Synthesis https://en.wikipedia.org/wiki/Thesis,_antithesis,_synthesis

Hegel’s Dialectic https://plato.stanford.edu/entries/hegel-dialectics/

A Dialectical Approach to the Bible https://questians.wordpress.com/2010/12/14/a-dialectical-approach-to-the-bible/

Jacques Derrida https://plato.stanford.edu/entries/derrida/#ElaBasArgWorHos

Deconstruction and différance http://www.signosemio.com/derrida/deconstruction-and-differance.asp

Binary Opposition https://en.wikipedia.org/wiki/Binary_opposition

James Bach, “We use tools” http://www.satisfice.com/blog/archives/1598

Semiotics in Marketing http://coolerinsights.com/2012/05/the-role-of-semiotics-in-marketing/

Jermais Rossler, “Testing vs Checking, so what?” https://medium.com/@roesslerj/testing-vs-checking-so-what-9eb4c97c166c

James Bach (cowritten with Michael Bolton), “Testing and Checking Refined” http://www.satisfice.com/blog/archives/856

Michael Bolton “On Testing and Checking Refined” http://www.developsense.com/blog/2013/03/testing-and-checking-redefined/

Some Trust in Models! On Simulations, HyperNormalisation and Distorted Reality in Software Teams (Part 1)

Still from Adam Curtis' 2016 documentary Hypernormalisation

Software is all about abstraction. We take an existing or desired process, model it and codify that model into a set of instructions and data that computers can read and follow.

We come up with different processes, methodologies, notional lifecycles, workflows, test strategies, measurements and quality controls to steer our immensely difficult projects to a satisfactory conclusion. For that we get teams of people with different roles and skills to work together, manage the creation and deployment of code with its own complexities and dependencies and implement what (we believe) users want is incredibly hard.

The processes we use to create software are also abstractions. The problems we face are, how much do the abstract requirements, process and methodologies we implement really reflect a desired real world outcome? In the midst of a software project, how do we maintain the link between abstraction and reality? What happens when a software team loses the ability to know about the difference between simply fulfilling the process and getting to a real world product people actually want? More dauntingly, at what point will the team stop caring?

Everything Was Forever Until It Was No More

A stamp featuring Pimenov's "Wedding on a Tomorrow Street", 1973

Anthropology and Sociology provide some interesting allegories that within limits help us to look at the questions above. The Russian academic Alexei Yurchak in his 2006 book "Everything Was Forever Until It Was No More: The Last Soviet Generation" discussed the paradoxical nature of life in the 70s and 80s Soviet Union. People were realising the disconnect between the ideological and propagandistic announcements of a Soviet state that saw itself as successful and immutable ("the end of history" as Marx envisaged) and the increasingly obvious fact that its economy was stagnant, its institutions were failing, events such as the War in Afghanistan were hurting its collective psyche and their quality of life was getting worse by the day. However in the absence of an alternative, whilst they understood something wasn't quite right they remained dissonant and carried on with the pretense that nothing was wrong, deluding themselves into a self-fulfilling prophecy. All the while the national discourse was piling more upon itself and becoming even more repetitive, in unwavering copying and reinforcement of earlier dogma. When the Soviet Union did collapse so quickly and dramatically, its disintegration was shocking to most Soviet citizens who had seen it as eternal. Yurchak coined this condition "Hypernormalisation".

The disconnect between the official narrative and the reality of life in the final decades of the Soviet Union gave rise to artistic protests of a satirical nature, with films such as Andrei Tarkovsky's "Stalker" describing a world that was outwardly similar to ours and could fulfill all our dreams but at a deeper level profoundly different and nightmarish.

Yurchak's book had a profound influence on the English BBC journalist and filmmaker Adam Curtis. His impressive 2016 documentary "HyperNormalisation" expressed the belief that this had been extended to the whole world. In his view, from the 1970s onwards our leaders and media figures had grown tired of struggling to articulate and tackle the complex problems of the real world, resigned themselves to managing the current order and predicting risks and created a "fake world" which trivialised long-existent complex social, economic and political issues into half-baked narratives of "Good vs Evil" with misplaced or contrived enemies. Instead of challenging themselves and the narratives told to them, people turned to virtual worlds in cyberspace that were designed to be free but had powerful hierarchies of their own - harvesting the information and activities of billions of people and and filtering the media people saw to reinforce instead of challenging their opinions.

Consequently people either grew to be dissonant of the "fake" perception-managed world and the real world, accepted the narrative as fact and opted into solipsism and self-focused activities or used the internet to create huge protests and revolutions. Those who retreated became susceptible to agents who were skilled in maintaining confusion and instability - the film uses the examples of then-candidate Donald Trump and Vladimir Putin's strategist Vladislav Surkov - turned politics and news into bewildering theatre such that facts and truth no longer became politically relevant and rumours and chaos reigned. This left them much more open towards the status quo (as in Putin's Russia) or demagoguery (as in the 2016 US Elections).

For those who revolted, in the absence of a compelling alternative direction of society these revolutions failed tragically and spectacularly, such as the protests in Egypt that were hijacked by the Muslim Brotherhood, eventually to be deposed by the military, or the Syrian Civil War that led to the rise of ISIS.

The Road to Disneyland. Simulation and Simulacra.

Pluto, Dale and Daffy Duck on a firetruck, Disneyland Anaheim, 2010

Both of the narratives above borrow from postmodernism, particularly semiotics (the study of communication with and assigning meanings to symbols and signs). The French sociologist Jean Baudrillard in his controversial 1981 treatise "Simulacra and Simulation" described the process where human culture progressively replaces our direct experience of reality with a series of signs and symbols (which he refers to as "simulations") - copies and representations of the real world with a seductive quality.

Over time the simulation is tweaked and copied to the point where it deviates from the original to the point where it is a (in his words) an "evil appearance—it is of the order of maleficence". This continues to the point where any relationship with the original reality is semantic and arbitrary at best.

The end point of this, which he calls a "simulacrum", is a simulation or copy that has been abstracted to the point where it has no relationship to an original at all - only to other signs that may themselves be simulacra. Our life experience of the simulacrum, not being based on any grounded physical object, is not even acknowledged or cared to be based on reality - yet it becomes a “truth" in its own right. This artificial existence is known as "Hyperreality".

The most common example stated, referred to by Baudrillard and Italian philosopher Umberto Eco, is Disneyland. The facades of Main Street with its allusion to an idealised past create a facade that appeals to our imaginations and desires. The Big Thunder Mountain ride is an exciting allusion to old mining rail tracks in a cowboy setting. Nevertheless, Main Street as a real street never existed and mining rail tracks of bygone times were not anything remotely like Big Thunder Mountain. Of course, Disneyland is so seductive that none of this matters. We engage with the fantasy world as if it were real. This is the definition of the Hyperreal.

Other stated examples of simulacra include historical fallacies that have been promoted widespread as facts, artificial substitutes for human interaction such as sex dolls and AI chatbots, films with digitally created CGI characters and backgrounds and "structured" reality TV shows.

A Postmodern Perspective on Software Delivery

I came across the work of Baudrillard along with Adam Curtis' "HyperNormalisation" at about the same time in 2016, after about nine years of working as a tester. Whilst drawing conclusions from ideas from sociology and anthropology applied to software teams without thought is a flawed and dangerous activity, I believe that they provide a lens through which our inability to deal with the inherent contradictions behind key concepts in software engineering can result in software teams falling into flawed and misused practices - and why even if they are suspected to be flawed and misused, teams fall into dissonance and fail to act right up to product catastrophe.

For this I offer perspectives on four concepts critical to software engineering and testing - quality, measurement, requirements and methodology/process.

Quality - Relative at Best, Simulacrum at Worst

The usual aim of a software team is to deliver a "quality product", however Quality is a vague and shifting target. Gerald Weinberg, seen as the "Godfather of Agile", wrote in his 2012 blog article "Agile and the Definition of Quality" on the "Relativity of Quality' - "what is adequate quality to one person may be inadequate quality to another." He states the following -

"If you examine various definitions of quality, you will always find this relativity. You may have to examine with care, though, for the relativity is often hidden, or at best, implicit.

Take for example Crosby's definition:

"Quality is meeting requirements."

Unless your requirements come directly from heaven (as some developers seem to think), a more precise statement would be:

"Quality is meeting some person's requirements."

For each different person, the same product will generally have different "quality," "

He list various statements defining quality from the point of view of specific project stakeholders.

"

a. "Zero defects is high quality."

1. to a user such as a surgeon whose work would be disturbed by those defects

2. to a manager who would be criticized for those defects

b. "Lots of features is high quality."

1. to users whose work can use those features–if they know about them

2. to marketers who believe that features sell products

c. "Elegant coding is high quality."

1. to developers who place a high value on the opinions of their peers

2. to professors of computer science who enjoy elegance

"

etc...

His ultimate definition of Quality - that it is "Value to Some Person" - has consequences to agile teams. He states that the definition of quality is "political and emotional" and thus leads to decisions on whose opinions count most. Also these decisions can be less than rational and are hidden from public view.

It is my assertion that in cases where not enough thought has been given to grappling with the contradictions above, a team’s conception of “Quality” has little in relation on the reality of the product (to the extent that the reality can be defined or measured) or the aims of the project. Quality as a symbol or concept becomes nothing more than the biased opinion or dogma of some powerful manager or stakeholder or a vague guess of the desires of some abstract “target market”. However in the perceived absence of anything better, the team acts as if it were the truth. It becomes a simulacrum.

The effect of this is we have started to give up the belief that we can make a “quality" product. James Bach, in his 2009 blog article “Quality is Dead #1", paints a despairing picture -

“A pleasing level of quality for end users has become too hard to achieve while demand for it has simultaneously evaporated and penalties for not achieving are weak…. When I say quality is dead, I don't mean that it’s dying, or that it’s under threat. What I mean is that we have collectively - and rationally - ceased to expect that software normally works well, even under normal conditions. Furthermore there is little any one user can do about it.”

Bach asserts that the result of this disillusion with the possibility of quality is that management have given up and moved towards the lowest common denominator approach - dispensing of good test teams and moving to cheaper and less capable offshoring teams.

“Top management can’t know what they are giving up or what they are getting. They simply want to spend less on testing. When testing becomes just a symbolic ritual, any method of testing will work, as long as it looks impressive to ignorant people and doesn’t cost too much.”

This “going through the motions” while publicly acting as if quality is being improved, as I see it, is a type of mini-hypernormalisation.

The Question of Measurement

Measuring an attribute that does not lend itself to precise and agreed definition and arises from human feelings and politics is evidently problematic and will lead to inconsistent and tenuous results. Rich Rogers, in his 2017 book "Changing Times: Quality for Humans in a Digital Age" writes -

"If stories and feelings tell us more about human responses than cold facts, dates and numbers, this might provide a clue as to why software development teams, and the organisations they work with, sometimes struggle with the question of quality."

"In this field, refuge is often sought in numbers.... By setting numeric quality targets - sometimes called exit criteria - a desired state can be agreed upon: a point at which the product is deemed good enough for whatever follows, including whether it is ready to be released to customers. The numbers act as a comforting security blanket, creating a sense of control. Even if the picture they paint is troubling, at least the provide a means of seeing that picture."

However as Rich Rogers continues to explain, this is an abstraction that while comforting, is limited and prone to mislead. Crude metrics such as counting of passed test cases and defects assume an equivalence between test cases that isn't likely to exist. They also hide the types of investigation and testing outside of these test cases along with defects that were resolved without the need to record.

"Quality is not measurable in the way cost or time might be measured... There is no such unit of measurement for quality."

He makes the case that metrics still have a role to play in discussions about patterns and potential issues. Reports are only useful in triggering those discussions - about what the metrics tell and do not tell.

The Illusion of Requirements

What about requirements? If one takes a list of use cases, desired functionalities and acceptance criteria - and the product contains all of these working to some agreed order - does that mean we have a quality product?

I believe that this is also tenuous at best and a comforting fantasy at worst. Requirements follow the same rules as "quality" defined by Weinberg. They depend on the judgement, political power and knowledge of the person who defined them - as filtered through project stakeholders, business analysts and the printed page.

The person who conceived the requirements, we have to assume, has a correct knowledge of the problem to be solved. Anyone with experience of software development will know of circumstances where that is a bad assumption. They may arise from a high level abstraction of an existing business process, ignorant of the day to day challenges and tacit knowledge of those implementing the process. The person who stated the requirements may be a high level manager without any experience of ever implementing the process, a slave to the reports of subordinates.

The illusion doesn't end there. Various test case management tools allow use cases and acceptance criteria to be linked to one or more test cases, generating requirements traceability. This is also flawed for the same reason that test case metrics are flawed. They make presumptions that quality in a requirement can be neatly expressed by X test cases passed in an associated matrix, or an hour's session based exploratory testing revealing no defects. This is very wrong as any tester with experience will tell you.

Methodologies and Primacy of the Process

In response to the above, there have been created many different approaches to a software development project along with practices within them. Even within Agile we have XP, Kanban, TDD, BDD, Continuous Integration, Scrum, SAFe, DevOps… Even within testing we have competing schools and methodologies. These are all simulations of how teams work in practice - very rarely do teams follow a prescribed methodology to the letter. How one team defines “agile” can have only a tenuous relationship to another even within the same company.

The danger is that the process is deemed to matter at the expense of the outcome, or implemented to provide benefits to the adherents without respect to the project. For new adherents, as Rich Rogers points out -

"The techniques and tools used in carrying out the work can seem exciting, and there is no shortage of new ideas and skills to learn. The desire to be at the forefront of change, or at the very least to demonstrate awareness and familiarity with current methods, can be a powerful motivator in this field. Adoption of new techniques, training in how to use new tools and acquisition of knowledge and skills related to new ideas, or possibly a desire to enhance a resume."

One area where the hype and impetus to learn is already causing an effect is in test automation. The drive to automate more (if not all) tests, and its effects on tester recruitment, causes testers to devote project and personal time to learning automation tools and programming - and to implement them without experience or real forethought. I know from my own experience the waste and effects of poorly implementing an automation strategy based surreptitiously on a desire to learn a new skill in a commercial context.

Another danger is that processes are defined at the company level and not the team level. Because they were believed to work before, we enforce processes and methodologies on other projects in an effort to "standardise". This may make sense at a management, resource allocation and control level, however projects with ill-considered, enforced processes risk great failure.

Leaky Abstractions in a Complex World

If we understand the flaws in defining quality, measurement and requirements, and our methodologies are idealisations, why do so many teams still persist in making critical project decisions based on their flawed understanding of them? Is it purely ignorance or something else?

Rich Rogers makes the point that metrics, however flawed, act as a comforting security blanket, providing some basis for decision making. We may not be measuring the right thing but we are measuring SOMETHING that may or may not be close to quality.

In the same way, requirements may be incomplete, flawed or at the extreme end utterly irrelevant to the problem we wish to solve, however they do exist. One can use them for contracts, resourcing, planning, development, testing and delivery. In agile methodologies we can continually add changes until (we hope) the end product approaches something the user or stakeholders might want.

In any case these, along with working definitions of quality, are all models of the "real" state of a project or product. For better or for worse they are vital in reducing a development project from something almost ineffable to something that can be planned or achieved. I treat them as "simulations" as defined by Baudrillard. Because of this, they are greatly seductive.

What matters is their relationship to the real state of the product and the development process.

Hyperreality and the Failing Team

My conjecture is that in projects that are dysfunctional or fail to provide what we call a quality outcome, these simulations are so far removed as to be devoid of reality. In the same way that Disneyland is a simulacrum, so are these.

However, simulacra are greatly seductive simply because they are easier than embracing the difficult to manage real world they pretend to describe. Hyperreality is not real but its sway on team certainly is. Also, changing approaches and mindsets within teams and at the company level, especially without either the privilege of management support or a clear narrative of a better solution and how to get there, rates from difficult to impossible. The closer to the project deadline, after so much has been invested and with the probably painful consequences of a late or failed delivery, the less the team will be willing to experiment and the more the team is likely to act is if nothing is wrong. Status reports show green, problems are ignored, process and the simulacra are gospel - and then the Soviet Union collapses.

Nevertheless, as with the final decades of the Soviet Union, it is probable that a few team members at some level know that something was wrong with the system even if they could not articulate the problem and don't see an alternative. They may follow "best practice" to the letter, see great test coverage as defined by the flawed metrics above, defects resolved, requirements ticked off but their stakeholders still complain, or their clients find defects that are tangential to or don't make sense based on the requirements they worked with. Everything is said to be running smoothly but everyone and everything is stressed. This is what I regard as HyperNormalisation - hyperreality with a subtle chink in the armour.

In companies that are greatly hierarchical, have established and immutable processes and teams with egos and dominant personalities reliant on the status quo it is likely that the unnerved team members above, especially if not at senior level, will keep quiet. They may decide that "Nobody got fired for just doing their job", or patiently wait for the end of the project. They may ask to be moved to another project or if the stress becomes an issue take sick leave. They may go through the motions of work without committing fully to it. These are a protest of a sort, but all they do is accelerate the eventual collapse and slump in quality.

The unfortunate truth is that without some shock that forces the team to reflect on its assumptions and processes, most teams trapped in a seductive but vicious hyperreality are not likely to be self aware enough to change until the project suffers dramatically or collapses entirely.

Epilogue

In this text I have covered concepts of hypernormalisation, simulacra and simulation and hyperreality and suggested them as analogies to look at four key concepts in software engineering - quality, measurement and requirements and methodology/process - and their impact on software projects. I have asserted that our inability to understand that these are ultimately seductive simulations with varying levels of relationship to the real world, and the difficulties of teams to reconcile these, cause great risk of projects deluding themselves into poor products and failure.

In Part 2, to be completed shortly, I look at applying these concepts to the IT industry and our tech culture as a whole, and look into various ways suggested to prevent or rescue teams from seductive self delusion. In the meantime, I would be grateful for your comments and considerations.

I'm Paul and I'm a Failure (and it's ok!)

Every day I make one silly mistake. Be that misreading some acceptance criteria or use case, setting something to the wrong value in JIRA so that it doesn't appear in a bug report, forgetting to send a document to someone, missing my train stop on the way home, accidentally pulling out my wife's computer's power cable when disconnecting my own laptop, the large number of times I wrote a tweet that in retrospect I shouldn't have...

My life in IT has had a few pretty startling incidents of backing a losing horse. The time when as test lead I fully and excitedly backed my superior's decision to solve our automation problems by buying a licence for HP QTP for tens of thousands of dollars (buying into the logic that spending lots of money of a tool shows undying commitment) going gung ho at creating automated tests in it (my team of three being the only team to do so). When my superior left and it was sensibly decided by the tech management that we would save our money and move to Selenium, my team lost its entire automation suite overnight. I couldn't argue - it was totally the right decision.

Or the time when due to my lack of experience in development, knowledge of the relative benefits of a breadth-first search and willingness to ask for help the recursive database abstraction layer I wrote took days instead of tens of minutes to run (or more often crashed due to an out of memory error - no mean feat in a garbage collection language like C# !)  and caused severe delays to a product release. The fallout from that shattered my belief in my own programming ability for years, but I got over it slowly.

Or the time when I managed to break a database migration system I was coding a multilingual interface for due to the fact that one of the lines I (clearly in an unthinking fashion) translated into French was actually a SQL statement.

Why do I mention these potentially career-limiting paragraphs in my blog? Because whilst I suffer from self belief issues as much as the next person, I don't feel ashamed of the mistakes I made (and occasionally still do), have got over the need to be a perfectionist and slowly recognised them as opportunities for learning and self reflection. Only the most privileged and least willing to take risk of us sail through life without failing at something.

Yet when I read the blogs of some of the other testers and thought leaders in this field (not that I would ever call myself a thought leader in anything), I am unsettled by the lack of humility or mention of the hard lessons behind the advice they give. Of course we have careers to protect, conferences to speak at and consulting gigs to apply for, however none of us were great IT consultants, devs and testers out of the womb. Not One Of Us. Rarely is a career a perfect and graceful trajectory from school, university via junior to senior. We all have mistakes we have made (some of us may well have been fired from jobs and had to bounce back) and opportunities to learn, so why don't we write about them, or talk about them at conferences, or mention them on Twitter?

Why aren't we more tolerant of the mistakes our colleagues, managers or staff make? Do we think we are above them? I once had a boss whose lack of tolerance of perfect work (or for that matter, temper) was legendary. I once heard him shout "I don't pay you to make mistakes!" Was he devoid of errors himself? Not at all.

Was his team a perfectly oiled machine as a result? Of course not, and most hated him. High performance doesn't come about through bullying and threatening people. He achieved nothing but probably high blood pressure. I have seen other attitudes in teams I have worked with that, whilst they are not quite as aggressive or extreme, were no less haughty.

Let's calm down and appreciate our fallibility. Admit our errors. State what we learned in the context of how we learned them. Let's be more tolerant of the honest failures of those work with. That is the only way we will receive forgiveness for our faults in return.

On Testing Technocrats

The agile manifesto, written in 2001, states as one of its aims -

 "Individuals and interactions over processes and tools"

It is hard to imagine how in an agile team, where stakeholders, product managers, developers, testers, BAs and other groups work so closely together and much knowledge is tacit and undocumented, the above aim could possibly be ignored.

Software is ultimately used by people to implement solutions to problems people suffer from. It is created by people. The use cases and requirements that developers implement and testers test against are created by someone to reflect someone's (or some people's) wishes and needs. Testers test to find issues that we hope will never be discovered by people whose problems the software under test is built to solve.

...Which makes me annoyed about the idea that development/test teams can automate everything. The problems that bad software can cause for people are many and varied and the numbers and kinds of ways that a non-trivial application may fail are far greater, often more subjective than we can possibly plan for in advance. Requirements and specifications have subtle holes and areas of tacit knowledge that risk creating a product that does a wonderful job of something people don't actually want.

I am a big fan of test automation in its rightful place. As a regression tool it frees the tester from hours of tedious and often unfruitful checks to concentrate on those areas that require more exploration, analysis and thinking. It provides stubs and mocks so that the developer and tester can continue their work without the full integrated system being ready or available. It creates reams of test data of whatever attribute is necessary so that we can continue our work without laborious setup. It helps us perform checks with multitudes of simulated users to discern areas of poor performance. For all of the above however it doesn't replace the thinking mind of a competent test analyst. The kind who is adept at putting him or herself in the frame of the user, with all the complexities this involves.

But some of us feel that we can automate all of this away for 100% test coverage by automation, or that some future AI will make all non-automated test approaches redundant (not convinced that this will ever happen). I wonder, is it all about cost-cutting? The bottom line?

Or is there a type of person who thinks that the ambiguous and complex can be completely reduced to a set of checks and algorithms? That the human dimension, the outlook that human beings provide, can be abstracted away by technology? That what people see as quality can be reduced to a set of Yes and No answers. We hit the button, stand back and like the great tech sausage factory, we pump something in, our list of deviations comes out. Rinse and repeat with a few fixes and we get quality at the end...

I call these types the testing technocrats. By reducing our work to purely algorithms and checks to be done by machines - and quashing the thinking factor - they reduce the wishes and concerns of the users of our products to exactly that.

All in the hope of getting releases out faster, saving a bit of money and not to have to deal with the complex findings that a thinking human tester provides. This has been called out as wrong many times but often to deaf ears.

Test automation is an amazing thing that lets us achieve great efficiencies. It is not a replacement for the thinking human element. As testers we should be taking every effort to continue argue against those who believe it is.

On the Need to Test for Beauty and Elegance

I have spent the last month off work, visiting parts of the UK, Paris and Seoul with my wife. During this time I have spent much time engaged in visiting museums and historic sights and strangely enough pondering over my half-formed and admittedly likely flawed ideas regarding what beauty and elegance is.

What IT products would end up in a museum or art gallery in 100 years time? Computers that didn'ty look great but represented an impressive landmark in CS such as Tom Kilburn's Baby? Beautiful and stylish Apple iMacs and iPads? More "functional" but equally noteworthy VIC 20s and Atari STs? What about software? Computer games such as the Tomb Raider and Destiny series are beautiful to look at and very fun to play but the much more mundane Microsoft Office has been more important to people in a wide range of areas. Can one genre be seen as "art" and elevated to the Louvre one day, sharing a wing with the Botticellis and the Venus de Milo, and the other deemed more fitting for industrial museums such as the venerable Museum of Science and Industry in Manchester - sharing a wing with the steam engines and 19th Century cotton weavers? Is this a poor distinction to make and should all software be treated equally?

Sandro Botticelli: Madonna and Child with St. John the Baptist, c. 1470–1475, Louvre (CC: Wikipedia)

Replica of Tom Kilburn's Manchester Small Scale Experimental Machine, nicknamed "Baby". The world's first stored program computer. (CC: Wikipedia)

How many development teams see their end product as aesthetically pleasing and write this into their requirements? Who is the best judge of something so abstract and intangible - a UI expert, key users, the testers or the manager? In the middle and latter stages of a project, when the deadline is upon us and the resource restraints come thick and fast, how much of aesthetic quality sacrificed to reach the goal? How often do testers actually raise an issue that a design or its implementation is ugly or inelegant?

I have worked with some (what I regarded as) damn ugly and inelegant software in some past projects, yet I admit that rarely have I put up my hand and stated that some feature was just not stylish or pretty enough. I have improved on this recently. In my experience the all too common wisdom in teams has been that it was "what was agreed in the spec" or "what the client wants" or "what the designers/devs came up with and it is too late to change it". In terms of aesthetics, I have simply tended to focus on useability and how closely the user interface matches the design. In this respect I should have done better and fought against the perceived wisdom - in some cases nothing more than excuses for inaction.

However useability is not the same as elegance and "matching the design" is not the same as beauty. All software aimed to be used by people will be judged by its aesthetic qualities and elegance. We want software that elates the heart as well as satisfies the need. Testers have a critical role to play in achieving this.

The focus (especially in agile development) on inclusion of features above all else - done in a rushed fashion without care for aesthetics - detracts from the elegance (where simplicity is a large part) and beauty of the eventual product. Even after production deployment, we create environments where the final has new challenges to cope with - be they oversized and dynamic ad content that change the layout of web pages or overly complicated and badly designed later-introduced game levels and characters - that give a poor impression to our users.

This can also be caused by testers being brought in at the development (and later) stages of the project - after requirements and design are "locked down". The current trend towards "shift-left" may resolve this. Another cause for certain projects heavy focus on meeting the letter of requirements over thinking about what is best for the users - which takes leadership from the stakeholders and project management down to  resolve but for which testers must be willing to ask the difficult questions to achieve.

Regarding who is best to judge what is beautiful or elegant or what even these terms mean in IT, I have no definitive answer and put the question out to the community. There are web and UI design standards which can help but they do not solve the problem. At one level, product walkthroughs and the inclusion of management and user views at every iteration can at least provide opinion and consensus (where all are allowed to speak with equal merit), however even in these cases the focus can tend towards what is functionally sufficient and not the aesthetics.

I implore all of us working in development teams to challenge views about design and look and feel as early as possible in the lifecycle. Testers should ask hard questions about the aesthetic qualities of the product, however without acceptance from stakeholders, management, business analysts, designers and developers nothing will change. Clearly there are many examples of stunning and elegant products created that people like as well as use (Apple is a case in point), so good practice does exist. We all need to find and follow it.

On Minor Bugs, or Death by a Thousand Cuts

Defects like other products in IT have a lifecycle. There are variations but in most places I have seen and worked the lifecycle for defects that are fixed, retested and pass retest is largely as below -

Found -> Logged or Reported -> Triaged, Assigned Severity -> Allocated to Developer -> Reported Resolved -> Retested -> (if pass retest) Closed

This is not the only way. Some testing schools, particularly Rapid Software Development recommend talking with the developer to see if a quick fix can be added efficiently without the need to log or report the issue. One with access to source code and sufficient knowledge may have the development knowledge to implement one's own fix. The one problem with this approach is that these incidents would not appear in defect counts in test summary reports.

In a large number of cases the reporting and triage/severity assignment of a defect are done in the same step. That is a problematic approach and assumes that the tester has the background and knowledge to make the same judgement about defect severity as a user or stakeholder. There are numerous times I have misjudged the user impact of some defect and been subsequently corrected.

Typically defects are graded based on an enumerated severity score. One I use is -

Sev 1 : Blocker.

Catastrophic failure of the application or major functionality. No further testing can proceed. Defect must be resolved immediately before delivery.

Sev 2 : Critical.

A failure of a functionality or loss of data which cannot be worked around. Defect must be resolved before delivery.

Sev 3 : Medium

Failure of a functionality for which a workaround exists. Delivery may be done if PM or stakeholders accept the risk.

Sev 4 : Minor, Trivial

Cosmetic defects. Styling issues, spelling mistakes, layout and minor workflow errors etc.

Defects are then prioritised based on available resources, the impact of other work, timelines and fixes applied to Sev 1 and 2 defects first, then 3 and 4. Defects 3 and 4 may be resolved following a production release dependent on the acceptable risk...

..Except that the pressures of timelines, staffing and resourcing and the need to release other features make it very difficult for us to return to work on these lower severity issues. They often get forgotten - that minor useability issue that customers have to live with or that styling defect that makes the application look somewhat less than professional.

This is a dangerous state for the application and the project. While high severity defects mean that the customer cannot do some task or other and thus do typically get prioritised and fixed quickly, the less severe look-feel and useability defects left unresolved are what will annoy your customers and users in the long term, leave a bad impression of the company and at worst will leave your product as shelfware. These are the things that they grumble about in user groups or forums on the web, mention in app and product reviews and tell their friends and colleagues - each a lost avenue that may overwise have lead to a new customer. Once the complaints and dissatisfaction reach a crescendo these initially small issues become major ones and you will have to deal with them in a fast, probably stressful manner.

So how can we manage these? Some ideas I have seen are -

1) Endeavouring to resolve all of them before the first production release. This is likely to be impractical, negates the idea of priorities and severities in the first place and the sudden redevelopment will require more regression testing (automated or not). However it would get the job done.

2) An agile cycle ringfenced to resolve all outstanding non-severe defects. This is a good approach but not perfect. Agile development cycles centre around the release of features and you would lose a cycle of new features in the process. It will thus require buy-in from the stakeholders of the project, who may themselves not be so concerned about minor defects vs key features.

3) A blended approach of fixes drip-fed over time. This may be particularly good where there already exists a continuous integration and deployment process such that the said fixes can be deployed speedily and seamlessly. However this still impacts on resources otherwise likely to be working onthe development of features and there would still need to be a schedule devised such that all the lower severity defects are resolved as soon as possible.

The above was written as a point of discussion and I would love to know your views.

Some Security Podcasts I Like

As a tester and a current student of web security and pentesting I have recently tried to update and improve my knowledge on general security issues. I am a big fan of podcasts, being able to listen to them while I work and travel. Having a set of podcasts on the go is I find an easy win in terms of time and opportunity - and there are some free, good and enlightening ones...

OWASP 24-7

The official podcast of the OWASP security consortium. A podcast released around once a month. Conversations with speakers on big and important areas such as security, devops, continuous integration and project management along with details on individual OWASP projects and the APPSEC conferences. I find the style engaging and informative and almost never dull. Not to be missed.

TWiT Security Now

The Daddy of the genre. IT Guru and broadcaster Leo Laporte and anti-spyware pioneer Steve Gibson spend two hours every week discussing the big IT security issues of the day. Recent topics include the huge Struts 2 and Cloudflare vulnerabilities, the leaking of CIA's Vault 7 and cyber warfare against Poland and the Ukraine. Once again, interesting conversations and information to while away the hours.

Defensive Security Podcast

Security experts Jerry Bell (@maliciouslink) and Andrew Kalat (@lerg) talk about the big security issues of the day and how companies can protect themselves. Covers similar topics to TWiT but in a shorter time and at a somewhat more technical level. A decent and informative podcast, however I would not believe that there is much value in listening to this and TWiT Security Now every week.

Paul's Security Weekly / Hack Naked News

​A large mixture of podcasts and webcasts produced by Paul Asadoorian aimed at the jobbing security tester, with demos, news items, online interviews and advice entertainingly delivered. The Hack Naked News cast comprises short (no more than 10 min) news episodes about the world of hacking and security. Great for those who are busy or reticent about listening to the longer recordings above. Worth checking out..

On Vegan Cooking and What Testers can Learn from it

This admittedly odd idea for a blog came from an irreverent Twitter discussion between testers Jean Ann Harrison (@JA_Harrison), Mike Talks (@TestSheepNZ), Gem Hill (@Gem_Hill) and I that touched on vegan cookbooks and cooking, binding agents, testing conferences and random twitter followers! The idea came up that there could be some concept X that works in vegan cooking and yet could be readily applied to good software testing practice - and that someone should submit a testing conference abstract on this!

I haven't yet accepted the challenge to write a conference abstract, however as a tester, occasional blogger on eccentric testing topics and vegan I did offer to think hard about this and blog some ideas. This is my attempt at a testing article using vegan cooking as a metaphor. This blog post could crash and burn a horrible vegan death, taking my reputation with it, however fortune favours the brave and all that....

I have been a vegetarian since 2006 and a vegan since late 2012. I have never been comfortable with the idea that animals are there for our exploitation and consumption, and took the vegetarian and eventually vegan path for moral reasons. I am disappointed that it hasn't changed my waistline much - a combination of the large number of vegan confectionaries out there and my lack of  willpower - therefore I cannot vouch for it as a weight loss plan!

I spent some time thinking about this tonight while cooking dinner. Note that cooking and software testing are obviously very different activities and therefore the concepts I outline are somewhat nebulous, general and already well known, however that does not detract from their importance. Most cooking concepts listed below apply to some degree to other restricted diets and non-vegan cuisine as well.

My Dinner - Vegetable, Basil and Mock Chicken broth with rice.

1) Constraints Abound. Get Used to Going Off-Script and Exploring.

There is no national cuisine that fully lends itself to veganism (except maybe Vietnamese and some Middle Eastern cuisine). Vegan recipes and cookbooks are typically developed by either taking existing recipes and exploring suitable plant-based replacements for animal-based ingredients or developing from scratch.

This in itself is not perfect. Take baking for example. Baking usually uses egg as a binding agent and milk and butter to provide flavour, structure and texture. There are alternatives to these - binding agents such as ground flaxseed and mashed banana and nut and soy milks and margarines - however these are not a seamless replacement and can alter the taste, texture and preparation of food.

Another example is meat itself. A plethora of soy, seitan, mushroom, gluten and konjac-based meat replacements exist (SE Asian and Indian supermarkets are best for these although most supermarkets stock some brands), however they have their own properties, are not quite the same in flavour or texture and not to everybody's taste. For recent adherents to vegetarianism who are fond of the taste of meat (as I was), this can be troubling.

What this has resulted in is a great culture of exploration and experimentation. Lots of cookbooks were developed by taking an existing cuisine or set of recipes and add radical ingredients, not obvious at first glace, gained from iterative trial, error and creative adjustment. The recipe is just the starting point. We take vegetables, herbs, legumes and other ingredients and add them until we get the satisfaction we want. It can be long and laborious over many meals but it is a lot of fun and when it works the results are dramatic! As an example, the cookbook "Appetite for Reduction" by vegan chef Isa Chandra Moskowitz contains recipes from such combinations as red wine, kalamata olives and tempeh and garlic dressing made from tahini, nutritional yeast flakes and miso - and they taste amazing!

Our well-thumbed cookbooks.

In the world of testing it is well known that exploratory testing methods are more effective at finding defects than adherence to traditional scripted test documentation - I also find them more interesting and motivating - and yet so many testers still draw up test cases with detailed steps in advance and execute them narrowly and slavishly. This also requires the existence of such items as a complete, static requirements specification - which is most often unlikely to exist in all but highly structured waterfall projects. The key attributes of a tester (in my opinion) are curiosity about the software under test, a critical eye, passion for quality, persistence and creativity - so feel free to deviate from the path, come up with new test ideas and go for it!

2) Keep Your Environment Protected, Clean and Maintained (Preferably by You)

My wife and I live with my meat-eating father. We use the same fridge and kitchen. We take steps to protect our environment.

Having uncooked meats and unpackaged herbs, fruits and vegetables (especially to be used in raw salads) in the same fridge is a sanitary nightmare. Contaminated blood from the meats could make us all sick from food poisoning. We deal with this by ensuring that all meats are in sealed plastic containers on a different (lower) shelf to uncovered vegetables.

Also my wife and I are no longer fond of any lingering taste or smell of animal products in our cooking - and sometimes even the best cleaning doesn't get rid of these from cookware. We therefore bought my father his own pots and pans to use and allocated separate kitchen cupboards for him, something that has worked very well.

Our test environments, data and harnesses must similarly be protected and looked after. They should be easily reverted to a pre-test execution state. Although not perfect this helps to provide assurance that the defects we find are product defects and not caused by some external application, unwittingly corrupted data set or misconfiguration of settings. This also allows us to quickly remove test data and final states from an earlier test and re-execute a regression or defect retesting suite quickly and efficiently. I would also recommend that these be maintained by the testing team.

Modern Continuous Integration tools such as Docker, VM tools such as VirtualBox and Vagrant and automated database restores and migration serve to make the above simple and efficient. There are no excuses for having a sloppily maintained, uncontrolled test environment.

3) Have a Variety of Good Tools at Your Disposal. Know When, Where and How to Use Them.

Every decent cook, vegan or not, invests in a set of sharp, durable knives and good non-stick pans. I would also include a good wok. I'm a pretty mediocre cook at best and I still need them...

I do a lot of Asian stir-fry cooking. I like my carrots, capsicum and zucchini cut into strips or julienned and my herbs finely chopped up. My wife likes her eggplant finely cubed. To do this properly, especially with carrots and eggplant, we need sharp, durable knives and choppers of different sizes. These are expensive and need to be regularly sharpened. Similarly among meat eaters, every lover of steak swears by a good set of serrated steak knives.

Sliced bok choy, capsicum and basil with julienned carrot.

Similarly, we have a wide and disparate range of flavourings, herbs and condiments in our kitchen. These include apple cider and balsamic vinegars, kecap manis, soy sauce, miso, various powdered herbs such as basil, chinese five spice and oregano, dukkah and nanami togarashi. They provide a wonderful inclusion to the flavours in our cuisine, however used wrongly they can ruin a meal. I don't always instinctively know the best dishes to use them in (I have a predilection to sweet soy sauce, kecap manis, and use it in enough places for my wife to wonder if I am secretly pregnant...) but I do see it as a priority to learn.

A selection of our condiments

There are many tools available to the tester, from data creation through test planning execution, environment management and setup to automation. Everybody has an opinion of where and when to use them and the internet and books are full of guidance, however the decision of what, when and how to use a tool or tool suite depends entirely on the local context and must be decided by you. We do not have infinite time or budget or expertise available to us and the wrong tooling decisions uncorrected too late can ruin a project.

An example of wild claims and abuse of tools is test automation - far too often seen as a panacea or substitute for the hard graft of testing. Get HP ALM or Selenium Webdriver in, automate everything, we don't need manual testers anymore! Similarly write enough unit tests get 100% unit test coverage and we will never have a quality issue ever again! How often have we heard such complete rubbish?? It is me and the kecap manis again!

So many tools are available at low cost (or free, open source). There is also lots of good information on the internet on examples of how and where to use them. Take advantage of it and get the right tools for your project.

4) We Are (Mostly) Friendly, Come and Meet Us!

When I became a vegan in 2012 I struggled greatly despite my previous experience as a lacto-ovo-vegetarian. Veganism is a much broader and different set of values, not just in terms of how to follow a plant-based diet but also avoiding wool and silk, leather shoes, household cleaning and men's grooming products that contained animal products - even my choice of beer when I am in the pub on social events. It is very hard to do on one's own.

Luckily there are great online and in-person vegan support groups out there. Most large cities contain at least one vegetarian / vegan meetup (here in Sydney, Australia both Vegan Australia NSW chapter and Kym Staton's Sydney Vegan Club run great events) and I find that in almost all cases people are willing to offer advice and relate their experiences to newcomers. Online there are great initiatives (paid and free) such as Animals Australia Unleashed, Colleen Patrick-Goudreau's 30 Day Vegan Challenge and any number of Facebook groups and forums where people can find information and talk with others. Sadly not every group is friendly to newcomers however there is enough support and friendship out there to every person to find a niche.

Similarly, new testers and those without the ability to connect and chat in their workplaces have opportunities. So many cities nowadays have a Testers Meetup via meetup.com, usually running free or low cost talks and events. As an example the Sydney Testers Meetup, for which I am the social media officer has over 2000 members and almost all of its events are free and after work hours. Online Slack channels for testing such as http://testersio.slack.com and the Ministry of Testing Slack are busy with people discussing testing practice and offering tips. Most of the testing thought leaders are also active on Twitter and Linkedin and easy to find. Join up - we were all new once and happy to connect!