Are Test Cases Dead (…yet)?

Software Testing

It’s been a while ago since Rosie asked us this question. (

You can find multiple answers there from testers around the world and the sheer number and diversity of answers continues to baffle me.
Ask any tester for a definition or an explanation of what she does and you’ll get a different answer.

The same is true for designers, developers, analysts and probably any job that requires intellectual and creative thinking.
But why are we different? I don’t see any people in other roles questioning the definition of their craft, or pondering how they should explain themselves to others.
Yet we do so on a weekly basis.
Every week there’s a new blog post, Tweet, Slack discussion or Forum thread about who we are, what we stand for or should stand for and how we can explain or do things differently.

I’m no exception to this. I’m searching for my own answers, trying out different things and experimenting with what I can find or come up with myself.

Are we so misunderstood and mistreated that we have to devise new ways and machinations to seem valuable and become respected? Some of the practices we still cling on to have mutated over the years and have become fetishes.

I have come to believe that Test Cases are such tricks.
To me, Test Cases are a step by step description of a scenario, and in the end an expected result.
Someone, somewhere, needed a way to present the work of his or her testers in easy-to-swallow, countable pieces. That’s nothing preposterous. If you boil it down enough, (and forget all the interesting stuff) testing becomes an overload of information in this scheme:

“If someone does X, Z happens.”

Which is the lowest level possible of an requirement, user story, expectation, or whatever you wish to call it.
Essentially, you get a huge list of check results, but with the actual outcome, instead of an expected.

If you look at testing this way, Test Cases make perfect sense.
Additionally, over the years Test Cases have gotten many different reasons to justify working with them.

  • Newby testers can execute them and learn from them.
  • They can be used to document the product.
  • They can be used to manage the state of the product.

I, however, think they’re a special kind of torture. If ever I get sent below to do the devil’s bidding, it won’t be an apple just out of reach that torments me. No, it’ll be a Test Case management tool that never gets filled.

Over the past projects I’ve participated in I’ve frequently been asked to do Test Cases. Here’s a couple of experiences:

Project 1
Testing of a Healthcare device that freezes mid visit. They asked me to analyze the documentation, create test cases and then test to reproduce the error.

I ignored these instructions, simulated a visit on the device and found the error 5 minutes after getting my hands on the tablet. Test Cases were filed afterwards and never read.

The combined testing effort for this project was 0,5 day. The only thing I contributed was providing information on where a most critical bug that was threatening the project could be found. Nothing more, but I did it in a couple of minutes.

Project 2
Project was failing, product was riddled with bugs. They asked me to analyze the documentation, create test cases and then test to find as many bugs as possible.

I ignored the instructions, started exploring, learning, modeling and logging all the bugs I encountered. When I thought it was needed, I documented some behavior in Test Case format. The Project Manager was happy, but that’s about all I have to show for the effort of writing those scenario’s.

We found all kinds of bugs fast and were able to rapidly learn what was important to the client.

Project 3
Project was failing. There was a severe disconnect between Users and Development team. No testing was done and when the client saw the product for the first time, it was a fiasco.

I was asked to read documentation, write Test Cases, test and log bugs.
However, in this case, the firm I was working at would get ‘points’ for doing any of the following per module: Write Test Cases, Execute Test Cases and Provide a Test Report. The more points we got, the more we could bill.
We put in an immense amount of time to create the scripts and making them executable. Afterwards, we’d pass/fail them in such a way that it’d be enough to get the “done” state and get the points.
In hindsight, we put in a lot of effort to adhere to rules that brought us no extra value except for the analysis work we did.
We could’ve worked faster and better if we hadn’t had to invest time and energy in a task that brought us nothing.
Oh, and yes, we’ve had newby testers and users execute the scripts. The format taught them nothing except to follow a script. They did not know WHY they were doing them.
The things used to manage the state of the project, were the important bugs.

What eventually brought success to the project, was a close understanding and coalition between business and testers.

My Conclusions.

I have yet to come across a context where Test Cases are the single best solution to a problem. In my experience, they divert from the essential tasks.
Rapid information gathering, learning and reporting combined with good working relations with your stakeholders trump just about anything else.

When writing scripts I feel brain-dead, a monkey doing monkey’s work. It feels like I’m wasting my time. I know nobody is interested in the scripts themselves. They might feel more assured if there are several Test Cases, but that’s a false sense of security.
And I haven’t even started executing them. Imagine having to do half a day’s work by a step by step instructional script. Again and again. Day after day.

I want to be a creative, intelligence worker that has my stakeholders best interest at heart. I want to protect them from unwelcome surprises and support my team to deliver true value.
I want my way of working to reflect those goals, to hone and to strengthen them.
Be it in Charters, Mind Maps or Checklists.

Test Cases are dead… to me at least.


One thought on “Are Test Cases Dead (…yet)?

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s