Applied heuristics

Experience Reports, Software Testing

This is an experiment. I’m trying to figure out my understanding of what the word ‘Heuristics’ means to me and whether it’s helpful to me and my craft: Testing.

Therefore, I’ll rehash an old story of how I found a bug and annotate the heuristics used in Bold and see what I can analyse from that afterwards.

Any guidance, ideas, intuitions, sources are very much appreciated.

Finding a Memory Leak

It was past noon. I’d been testing an application that wasn’t too complex for quite a long time. I knew it’s ins and outs and I felt myself getting bored.
I remember going through the customer files rather methodically one by one, click,… click,… click , without a clear aim. You could say I was randomly exploring.

Until something didn’t feel right. Call it surprise, intuition, something was wrong.
I noticed a pattern of loading times slowing down almost unnoticeable. I can’t put to words what triggered it for me. Was it an Observer-expectancy effect? Was it my negative mindset that sharpened my senses? I haven’t got a clue, but I felt I had to focus, zoom in.

I chose a tactic that was less boring than meticulously clicking my way up. Selenium IDE isn’t the best of tools, but it fit my purpose perfectly. I recorded my click and copied it a thousand times. I monitored the behaviour with developer tools, pressed ‘play’ and went for a coffee. After a while, I could identify that it went terribly slow because I could see the latency increase. Eventually I saw it ramping up until it crashed. Pairing with a developer we could conclude that there was a serious memory leak.

Implicit and Explicit heuristics?

A heuristic is “A fallible approach to solve a problem”

The teachings of RST and BBST and possibly how ‘heuristics’ were meant in the science books would identify all the bold phrases as some variation of a heuristic. I’m sure many other heuristics played in my mind that I don’t have words for yet.

However, notice how the first part of my story is almost completely based on hunches and feelings. Something directed me to find this bug and it wasn’t intentional.

In the second part, I took matters into my own hand. I had a goal, I chose a tactic and was able to measure my results. Other tactics, could have been clicking the button myself for the umpteenth time and miss out on a coffee. I might have asked a developer to look into it straight away and maybe had time left for two or three coffees.

I can’t know for sure whether I chose the correct heuristic or tactic for that situation and I’m sure as hell “# of cups of coffee drunk” isn’t the correct way to measure that success.

We can’t go back in time and compare results of heuristics used vs. the ones not used. Though I’m pretty happy with the results of the one I set up.

The Question Remains

Are the ‘Implicit heuristics’, (i.e. feelings, patterns, biases, hunches) which can’t really be controlled, measured,… useful to call heuristics? They fail, yes. They help you notice things, yes. But do they solve problems?

Are ‘Explicit heuristics’, (i.e. repetition to find boundaries, monitoring for abnormalities, pairing to increase understanding) worth a dime without their implicit counterparts?

It’s not up to me to answer these questions. Yet I find it intriguing to ponder over the matter.

IMG_20170718_102211846

The Core of the Testing Role

Software Testing, The Core of Things

This is the second of a couple of “The core of X” -articles that give you -my- ideas on what I think of X. The attempt is to cut away all that surrounds the term and show its core for what it truly is. The essence of it all.
Take from it what you will, love it or leave it. In any case, I would invite you to share what goes around in your head and what you feel while reading. Feel free to correct me or engage in discussion. We’re here to learn.

There’s Always Testing

Independent Test Teams, Dogfooding, Bugbashes, agile Testers, Test Jumpers,… or testing by any other name. There’s always some testing going on in a software project. It might even not be called testing, but it’s there.
It’s in those moments of critical thinking just before someone says “hey now, wouldn’t that shortcut our registration process?” or “Woah, I never thought about it in that way.”

When one ‘thing’: a piece of code, a requirement, a document,… is changed and passed from one person to another: that’s an opportunity for doing testing.
This isn’t always done by a professional tester. Au contraire.
Whether it’s a formalized or accidental process, there’s people jumping in and out of “The Testing Role”. Adding value because of it.

“Everyone tests, but it’s a skill not many professionals care to practice.”

The Core of the Testing Role = Mindset + Skills + Attitude

Be you a tester, developer, analyst, schoolboy or astronaut, to take up the Testing Role effectively, you need to be aware of these three different aspects.

Mindset

Most professionals are focused on Building. Adding, accelerating, stabilizing and all that improvement goodness. At one point or another, these professionals will need to test each other’s improvements. Having that builder’s mindset while doing so is a lousy way to go about it. It’s the equivalent of a mom cheering on her son on the football field:

You want to see them succeed. Confirmation bias and inattentional blindness play their part: Your feedback will be limited.

A tester’s mindset is often seen as a negative or destructive way of thinking. It’s focused on Risk. What can go wrong? How can this be abused? What haven’t we thought of?

It’s about thinking differently. It’s about being engaged with the product with the intent of finding problems.

Getting that mindset consistently right is probably the most important part of this role, yet also the easiest to discard. The path of least resistance doesn’t lead here. It goes everywhere but here, with a rather sharp turn.

It’s a Skill

There’s a lot of different skills involved in testing.

Critical thinking
Systems thinking
Modeling
Risk Analysis
Quick learning
One’s own expectations management
Inter- and intrapersonal skills.
Technical insight
…

Just like any other craft, it might look easy for the unwitting and unpracticed. To become a master though, it takes hours, days of mindful practice.

Attitude

Attitude, to me, is the defining characteristic for a tester.

It makes you speak up in planning meetings to go against popular opinion.
It has you fighting for a bugfix you think is important.
It gives you fresh ideas to go through the same high-risk functionality over and over.
It helps you be different in a world where sameness is worshiped.

Just like Mindset, which is about thinking differently, Attitude is too easily dismissed as part of the Testing Role. Being adamant about quality can be perceived as difficult or not-a-team-player behaviour. I’ve worked with many people who don’t want to bother learning about testing for those exact reasons. It doesn’t make you popular.

Recap

Anyone can take up the Testing Role. Whatever else you do, it takes these three things to be successful at it:

Mindset – Thinking differently when looking at solutions and problems.
Skills – Have the tools to analyse, find, plan for and report risks and issues.
Attitude – The resolve to be consistently different and defend underrepresented notions.

Doing that context switch isn’t easy, isn’t evident and takes practice.
While many people do this role & mindset switch daily, most of the time it is lacking. Practice, investigate and learn how to do it well.

Two Conferences of the East

Conferences, Software Testing

East of where? Why, Belgium of course. Where else would you place the center of the universe?

I’ve written about several conferences before, such as TestBash & Eurostar. It occurred to me that all previous gatherings were placed North of my hometown, that’s to say, until a few weeks ago. A new wind called my name.

After surviving the far and cold North, where I made friends around warm campfires and was taught the dance of ‘socializing while balancing a beer’, my steel winged carriage pointed another direction.
My next adventure lay East. To the haven city of Gdansk and the Transylvanian valley of Cluj Napoca.

I would tell you of my ride across the hills where I battled three packs of wild Romanian dogs, or the dance of fire that initialized me into the ranks of the Polish testing army. But that would be largely exaggerated and are best told at a bar.

None the less:
Two more conferences have been scratched of my bucket-list since then: Romanian Testing Conference & Testing Cup.

18422195_1543969875622917_3589158990241473518_o-900x720.jpg

TestSphere’s workshop: “The Quest for the Ultimate Test Story” in action

The Romanian Testing Conference

Imagine being picked up at the airport in a very expensive Audi. Imagine the driver telling you he arranged a bike for you (you like biking, btw). Imagine that during the ride you oversee a valley filled with houses and old church towers but green hills all around. Imagine a slight hill in that valley with a grand hotel of marble on top.

That’s the first encounter I had with RTC. They had everything planned for me, the bike, the hills, the hotel, my evening dinner and the scenery was groteske. I mean that in the best possible way.

The conference itself was inside the many rooms of the hotel and for several days it was bustling with a 300-400 headed enthusiastic and diverse group of testers.
What struck me about this crowd is that it is much younger and much more evenly distributed across gender. And the hunger for knowledge… Dear lord were they thirsty.

The people I talked to were very eager to hear my stories, were inquisitive, well spoken and remarkable.

I am greatly impressed by the organisation, the participants and the group of speakers that assembled. I would be remiss not to mention I will fondly look back upon the time spent exploring the city with Ard Kramer & Elizabeth Zagroba and Nicola Owen, Mike Noggens, and Keith Klain.

Andrei Contan receiving his prize for telling the Ultimate Testing Story

Testing Cup

I’m a sucker for Poland. Over the last 3 years, I’ve visited these greener pastures a good 4 times. But this time I was reminded that I had been visiting some of the wrong sides. Gdansk is more modern than any other city I visited in Poland by a long shot.

Upon arrival in the old town I was rejoined with Zeger, Andreas, Ard, Marianne and Johan. From that point on, everything went into light speed. That was the moment the stars became white lines flashing in my peripheral view and the limits of the universe became in reach.

I didn’t look back.

It was the second year that Testing Cup grew larger than ‘just their testing competition’, which was incredible, make no mistake. Last year they experimented with several international speakers and this year they invited a larger group.
As a team we immediately set forth to gather all speakers in a whatsapp in a “got to catch ’em all” kind of style.
This medium was the center of much, much absurd humour and self mocking. I loved it. You may find a selection of shared pictures at the bottom of this post.

Much like in Romania, we found a large amount of young, knowledge-hungry and diverse selection of Poland’s best testers.
They demonstrated that in the competition and in our workshops with witty remarks and colourful stories.

Two particular moments will haunt me until years to come:

The party was a surge of positive energy. Having drinks on the beach, standing barefooted in the Baltic sea, enjoying the view of a spectacular fire show and dancing until well in the night in good company.
The party was amazingly good, which was noticed in the sheer number of attendees that trickled in much later than intended the next day.
The first day we had 40 participants (the cap) in our workshop, the second one we had 9.

Another impression about how greatly the attendees LIVED this conference was all the way in the end when the champions of the competition were announced.
I saw a Software Tester run in the middle of the Gdansk football arena waving a huge golden cup above his head. I imagine he felt like Ronaldo. His face showed pure pride and happiness. The crowd was enamored.

Aftermath

What I witnessed at both conferences was a strong organization and an exceptionally large number of volunteers. Even though it felt like they weren’t there, that’s exactly what their strength was. They supported us, carried us and led us to have the best of times. We didn’t have to worry about a thing.
That’s Joanna Mocko, Maciej Chmielarz, Andrei Contan, Andrei Ghinescu, Radek Smilgin for you.

As a speaker, I recommend these conferences with the highest of regards.
Say hi when you’re there. I will be too.

DCBnkCiXkAASuXt

The Core of Automation (Regression Checks)

Software Testing, The Core of Things

This is the first of a couple of “The core of X” -articles that give you -my- ideas on what I think of X. Take from it what you will, love it or take offence. In any case, I would love to hear about what goes around in your head and what you feel while reading. Feel free to correct me or engage in discussion. We’re here to learn.

First things first: Language. For this blogpost, when I say Automation I mean a set of Anti-regression checks. Whether they’re on the Unit, Integration, API, UI,… any level, they’re all the same here. There’s usefulness in dividing them by purpose, use or whatever, but it’s not here. I use ‘checks’ where you may expect ‘tests’. That’s fine. The important thing is the point, all else is for a different time.
Behold, das Point:

The core purpose of Automation is NOT about
finding bugs, saving time, getting to market faster or bringing value faster.
It’s about
Stability and Reliability.

Now don’t get me wrong. The red stuff is greatly helped by what’s in green, but it is not and should not be the main focus.

Should your project not care about Stability and Reliability and only about going to market faster or saving time, you can just as well not write any checks.
If you want to find as many bugs as possible, automation is a rather inefficient way of looking for them.

Your checks are about minimizing the risk that comes from working with multiple people on the same thing. Because… inevitably, someone will make an error. That’s human nature. Ideally, you want your Regression Harness (or Automation Check Suite) to warn you of that error. This way, when a team has a good set of checks, it’ll keep a personal “oopsie-doozie” from becoming a whole-team “WHO THE HELL BROKE OUR STUFF AGAIN?!”.

When you engage in any form of automated checking, know that your focus should be on Reliability and Stability. It will lead you down the good path. The one of continuous integration, saving time, having to do less firefighting and having more control as a team.

I’ve seen people talk about metrics of automation and mention: number of bugs found, time saved, number of deploys per day, number of checks ran,…
These are rather unhelpful.
As a Team member, when it comes to metrics, I’d love to outweigh:

Time lost fighting instability + Energy wasted on firefighting + motivation loss while investigating problems + friction between team members
VS.
Time invested in building a regression harness + problems still occurring after the effort + Effect on team-cohesion and sense of purpose.

Hard to do, right? Though it could be better worded, that’s the only metric that counts for me.
(Though I might be interested in the other metrics sometimes, those moments are rather circumstantial)
Notice, though, the difference of mindset of investigating problems as individuals and building a solution as a team.

But… where does Testing fit into that strategy? In my opinion: Not necessarily.

While Testing can certainly help come up with interesting ideas on improving your check harness, it doesn’t have to be part of it. However:

The information that Testing provides from looking for unexpected, extreme and abusing behaviour can greatly support your automation.

I do believe that Testers should make the occasional step into this territory, just as much as I encourage anyone to make frequent and valuable role-switches. Yet the main focus of the Tester role is on providing valuable information that might have been overlooked, the team was unprotected against or misunderstood.
Make no mistake, if your product is rather complex there’ll be things missed, erred and overlooked.

Also… I’ve yet to meet a tester who didn’t think her project or business is simple.

TL;DR: Regression checking automation is about Stability and Reliability. When done well and consciously, the result can evolve to be: Less bugs, time saved firefighting & problem-solving, going to market faster and adding value faster. A Tester isn’t necessarily an agent in this process, but the information she provides can be invaluable.

IMG_20160725_125941 We all need stability. Sometimes more than other times.

Concerned about boxes

Complaining, Software Testing

Trump, Brexit, Le Pen, The Testing Community, Women in IT, The future of Testers, LinkedIn articles and retweets.

Here’s what scares me the most about recent (and less recent) developments around these examples:

I don’t know who’s making all this happening.
Me, and almost everyone around me apparently share the same views.
Yet there’s an overwhelming number of people working against us.
Who are these people?
Why are their views different and why aren’t we hearing each other?

I often feel like I’m in a huge echo chamber. I keep getting the same things in my twitter-feed, slack channels, fora, mails, magazines,…

As a middle-class, white, European,… , non-religious,… , extravert, male, … , who tries (vehemently) not to be a dick, I look around and genuinely wonder: How could Trump ever get elected? Nothing in the media ever mentioned it to even be remotly possible.
I’m shocked that the UK voted “leave”, because I’d seen nothing but pro-stay voices all day. I’m getting a very one-sided view and it’s not helping anyone.

This happens every day:

I glance over a forum thread and think: “This discussion,… again?!”
Pull open a Slack channel and see someone replying: “What do you exactly mean by Test Cases?”
I click on a link about a new idiotic thing Trump did and can barely keep myself from screaming: “THEN HOW THE HELL DID HE GET ELECTED?!”
Open a “Testing is dead” article and find out that it actually means the opposite.

I’m seeing the same thing again and again. I wonder why nothing is changing.

There’s boxes everywhere: Racism is a problem. Sexism is a problem. Classism is a problem. Extremism, in all its different forms, is a problem. Most -isms are problems.
You’d think we’d all know by now. Hell, there’s enough examples to teach us.

But we like hearing the same thing. We like being right and having no conflicts.
Nice, simple, easy. Bliss.

After going through a very painful cynic stage, I realized that I need to get out of my echo chamber more. As much as I love the people near to me, it’s detrimental to all of us should we not pursue diversity. Even the diversity we may not like.

That’s what testing is, right?
Learning new things, new views, new ways of thinking, having a different mindset,
And using that information
To increase the chance of having better ideas, beliefs, thoughts for you
and people around you.

In my eyes, repetition, conformism, truly is the enemy.
Everywhere.

An interesting thought I had whilst rereading: in our testing field, at least we’re having the discussions. We have various media which we can largely control. Fora, slack, twitter,… We should cherish that and draw upon its potential. Find the people we’re not hearing. Find out what they have to say… and learn.

IMG_20160804_124355