This afternoon, Katrina posed the question “What makes a good tester?” but phrased it in a clever activating way: “What makes you a great Tester?”
She had been wrestling with this particular behemoth for some days and found it time to call in the reinforcements.

This added a lot of dust and confusion into the fight, but turned out to bring some interesting threads.

Some remarks that came out of the discussion:

1. Testing is multidisciplinary, what are the attributes that make you a good tester?;
2. Who/what do we compare a good tester too? Non-testers? Bad testers?
3. Core skills: “good” means finding & reporting important problems quickly and cheaply and being able to express/defend your actions.;
4. A test that measures this. An exam? A duel to the death? A multiple choice test?
5. How good you are is directly connected with how valuable information you supply;
6. If you’re in high demand and teams are fighting to get you on theirs;
7. When you are considered an expert of the product;
8. Customer satisfaction is a parameter;
9. Peer assessment;
10. You could be really bad at finding bugs/important information, but inspire everyone around you to not make them in the first place or give their fullest to find them for you;
11. Being confident in your ability may be an indication;
12. The Relative Rule was mentioned.(http://www.developsense.com/blog/2010/09/done-the-relative-rule-and-the-unsettling-rule/)

There were other interesting remarks, but let’s keep it at this.

All of the above seem genuine criteria and yes, it is extremely relative to context. To find a general ‘rule’, however, I’d like to go deeper, broader and try to generalize. Many of the above can be countered with a “what if”. (We’re good at finding alternatives that complicate things)
Can we find common ground on what makes a good tester?

Context is important, but for this exercise we must hold every possible context into account rendering the parameter to become infinite or null. It’s virtually impossible to factor this in.

Skills are important, but as we have an infinite number of contexts, where a very large list of skills has the potential to find that out-of-the-box bug nobody without that particular skill thought about, we can’t calculate non-traditional skills as a factor.
There are a few skills that are always valuable but they are not necessarily crucial to be considered ‘good’.

Person X is a blank slate. You may customize X to your own wishes.  You must build person X to become a tester that could be an asset in any context. Use as few attributes as possible.

This is my person X:

1. The person has a professional demeanour;
2. The person has good written communication skills;
3. The person is a power learner, meaning that (s)he can quickly learn new things and is interested to;
4. The person is aware of his/her own shortcomings.
5. With the above attributes the person will use every method, tool, colleague, hoodwink, trick, question, experiment, ploy,… in his or her power to find important information.

To me, these are the absolute attributes and skills that make a tester good. Or at least good enough to serve in as many possible context I can imagine.

How would your person X look like?

Words are important
I often practice good and thoughtful wording. We’re in the business of gathering information and supplying it. Working in a business where most people have no deep understanding of what their colleagues do, we need to be able to clarify, even translate information in a way that our recipient can understand the message optimally.

This is no easy feat and requires much practice. That’s why I train myself in it and you should too. At some point this summer, the idea popped into my head that “Test should never be used as a noun”.

Since then I’ve had the pleasure to discuss this on Twitter and Skype with Chris(@Kinofrost, QualityExcellentness), Patrick Prill (@TestPappy), Sarah Teugels(@Sarah Teugels) and James (@rightsaidjames).

This is the conclusion I have come to. (which is not necessarily, what the others have found.)

Shaping of ‘a test’

Consider these two definitions (from Context Driven Testing – James bach)

“Testing is the process of evaluating a product by learning about it through exploration and experimentation, which includes: questioning, study, modeling, observation and inference, output checking, etc.”

“A test is an instance of testing.”

To me it felt that the second definition did not bring any value but more likely supported confusion and abuse of the term.

• ‘A test’ is a countable noun. Which makes it dangerous for (non-)testers who like to attach meaning and value to numbers.
• ‘A test’ is easily confused with ‘a check’. It’s much too easy to substitute the difficult to comprehend concept with a more simple one. The question: “Could this information impact value to anyone who matters?” is forgotten and replaced with “Can I push this button and is the background blue?”. ‘A test’ involves human interaction, tacit knowledge, interpreting results,…
• Calling your activity ‘a test’ implicates that it is a thing. Things are tangible and generally mean the same thing. A dog means a dog, even though there’s a few hundred different races. You can further specify what kind of dog it is you’re talking about.
  ‘A test’ involves tacit knowledge, which can’t as easily be expressed.
  In addition, calling it a thing, gives the illusion that it can be repeated. You don’t want other people to misunderstand it that way.

I have been convinced by the aforementioned people that there are indeed cases in which ‘a test’ can be a valuable. It comes with a few caveats though.

– It’s an activity; like ‘a run’ or ‘a play’ it isn’t specific about anything, but gives notion of a certain activity.
– It’s focused on finding important information (to a person who matters).
– It’s requires human intelligence. (tacit and explicit knowledge)
– It’s boundaries are irrelevant. Where it started and where it ended are not important to the concept.

We’ve come up with the following definition, trying to stay true to the nature of Context Driven Testing:
“A test is a human-controlled activity, any activity, that is focused on discovering information that is important to a person who matters”

Which is inclusive with: “Quality is value to some person who matters.”

Conclusion

“When you use ‘test’ as a noun, find a better way to phrase it.”
… and if you don’t, be mindful of what it represents.
Which has it’s exceptions of course, like every heuristic.

I’m open for further discussion, but Twitter didn’t take kind to this blog post format.

I had an amusing experience a few weeks ago. A colleague of mine was explaining how he was automating a regression test set and how it greatly enhanced their testing experience. I believed him, because I know he’s a professional worker with good testing and automating skills.
But I wanted to tease him a bit, as I’ve learned that he sometimes mixes up terminology and definitions.

I asked him: Today, I asked for an extract of a certain table from our database. I visualized it in excel, added some formulas and conditional markup. Next, I sorted on that markup and data errors were immediatley visible.

Was that automated testing or manual testing?

To many people, it’s a confusing question. Hell, it’s even confusing for me.
A better name might be “tool-enabled-checking/testing”. But this doesn’t clarify whether my example is testing or checking.

I noticed that we, testers, see checking and testing as surrogate terminology for automation and manual. This is not completely true, it’s too easy. There’s bound to be a gray area. A twilight zone where testing overflows in checks.

One tests a theory. My theory was that the table I selected might contain errors. I devised a formula in which I could show these mistakes. This is where theoretical testing flows over into the actual checking. The moment I start working in the excel sheet I’m beginning my practical checks. But checking is still a subset of testing. Be they tool-enabled or not, checking has to be done to either, prove or falsify your theories.

There’s a funny ebb and flow kind of feeling to testing and checking as they tend to alternate. While exploring your product you come across new ideas constantly. Further exploring ideas in depth or jumping to the next idea or even exploring ideas within ideas makes the whole concept extremely fluid.

Black on white, in the given example one goes into checking as soon as you start comparing your data to it’s expected result. But while you are doing it, you’re still testing. There is much verification, validation and checking in testing, but it’s the testing itself that’s interesting and which can result in infinitely more important paths to follow.

Be wary, that the above mentioned explainability heuristic does not hold into account false positives and negatives, human epiphanies (oh! That looks weird, let’s take a closer look) and more human errors by the analyst, developer, tester or anyone else whose had a say. It serves to show that “automated” is not the same as checking. It can be used to explain that automated checks are a subset of all possible checks and that all possible checks are a subset of Testing.