An advert showed up on my LinkedIN today.
This is what it said: 

Dramatic reduction in manual effort.  [The client] identified that automation is approximately 97% faster than their prior manual approach to quality assurance.

It had a link to an article in which a presentation could be viewed. I’m very interested in these adverts for I hope to someday find an honest, clear representation of how a test automation tool can actually be profitable to your project.

I once heard the following being stated: “People who believe in these statistic-based-sales-stories blindly, deserve losing their money on it.”
It was meant as a jest and I admit, I’ve added some more drama to it.

However, it is a deep running frustration of many testers that test automation is sold as automagic. Let me tell you a bit about computers.

Computers don’t think, computers don’t lie, computers don’t tell stories, computers don’t tell the truth, computers can’t give you advice and computers can not (yet) substitute for an intelligent, sentient human being.
They take numbers and they give numbers. They calculate.

It is in our nature to believe they can do more than that. There’s no magic in there, unless we put it there. But I don’t know any magicians.

Don’t get me wrong. I love automation. It helps me speed up the boring, simple, obvious checks that come with every daily build. There’s developers creating these checks, there’s testers creating other kinds of scripts. Most of the time, they give us useful information in the form of “none of the things that worked previously, are now broken”. Sometimes they fail and we get to check whether the script itself needs maintenance, or some functionality actually broke.

The 97% profit, or any time-wise comparisons are complete huey.
Did they calculate in the actual creation of the scripts? The maintenance? The infrastructure? The time it took the developers to adapt the application to accommodate the automation? The tooling cost?

Even if all of this was. Do the tests give the feedback you want? Does it report possible problems or does it give a false sense of quality? Can it replace anything?

I sincerely hope that the people handling the money, who take the decisions about implementing new tools, processes,… take a step back and talk to someone who can put these numbers in perspective.