Testing != Automation

Automation is not the goal .

The goal(s) is to –

  • Make humans effective at the craft of discovering,communicating and advocating risks to user experience,commercial reputation and utility of what we ship.
  • Keep feedback loops to humans as short as possible
  • Provide information to humans that is reliable and consistent( to answer the question…”if we ship now, what are the risks to our customers’ existing experience and utility of our software ?”)
  • Reduce repetitive tasks ( repetitive i.e. following the same steps with the same data with the same intent to achieve the same objective done over and over during a work day), so that cognitive load on humans is lowered and creative intent is sustained

Automation is not just about a Tool set

Automation , before tools, is actually, about –

  • analysis of the architecture
  • change
  • communication
  • facilitation
  • workshopping
  • reflection
  • test data
  • test environments
  • shared understanding
  • business risks
  • business strategy
  • product roadmap
  • commercial significance
  • experiementation
  • probing techniques
  • not Automating everything
  • not dichotomous to “manual” Testing

 

4 thoughts on “Testing != Automation

  1. It only makes sense to test, (automated or otherwise), if you expect to find AND fix defects.

    Otherwise why bother? Ship it and be damned.

    Some test automation anti-patterns I seen in several companies ….

    * The loss of Good bug reports.

    You’re doing it wrong if the developer asks you “I want to fix this defect you reported, how do I reproduce it?” and you answer, “First gather a room full of equipment, a couple of cloudservers, a vast host of dependencies, and then run that script for three days. And no, you can’t connect a debugger or output logging nor will I give you a core dump.”

    * Reliance on the System under test working nearly perfectly.

    Oh gee, the code has a Bug! We never ever expect that!

    The test automation system _relies_ on the code under test working nearly perfectly in all configurations, so now the whole test automation system is toast and every node must be manually recovered!

    1. Absolutely agree John . Able to reliably reproduce a bug is one of the key skills of a Tester , especially in complex Tech environments like Embedded Radio systems 🙂

      Thanks for your comment mate

      1. Sadly, it’s not a problem of embedded systems.

        I have spoken to several companies where “testing” is a checklist item on the plan before it goes out the door…..

        Now obviously there must be another item on the plan straight after it that says “evaluate, and if need be, fix defects found”.

        I’m always gobsmacked by how many managers, in all fields, seem to just completely drop that off the plan.

        Test Automation in some companies I have spoken to, seems to be a way of automatedly ticking that checklist item off….. “We ran the tests, Job Done, Ship It! “

  2. Your last points about before tools is right on target. People forget all the frontend work that needs to take place before even writing a line of code.

Leave a Reply