When Should a Test Be Automated? 06

angelcc216

Brian Marick

Testing Foundations
marick@testing.com


What Do You Lose With Automation?

Creating an automated test is usually more time-consuming (expensive) than running it once manually.1 The cost differential varies, depending on the product and the automation style.

· If the product is being tested through a GUI (graphical user interface), and your
automation style is to write scripts (essentially simple programs) that drive the GUI, an automated test may be several times as expensive as a manual test.

· If you use a GUI capture/replay tool that tracks your interactions with the product andbuilds a script from them, automation is relatively cheaper. It is not as cheap as manual testing, though, when you consider the cost of recapturing a test from the beginning after you make a mistake, the time spent organizing and documenting all thefiles that make up the test suite, the aggravation of finding and working around bugs in the tool, and so forth. Those small "in the noise" costs can add up surprisingly quickly.

· If you’re testing a compiler, automation might be only a little more expensive than manual testing, because most of the effort will go into writing test programs for the compiler to compile. Those programs have to be written whether or not they’re saved for reuse.