When Should a Test Be Automated? 09

angelcc216

Brian Marick

Testing Foundations
marick@testing.com


A note on estimation

I’m asking you for your best estimate of the number of bugs you’ll miss, on average, by automating a single test. The answer will not be "0.25". It will not even be "0.25 ± 0.024". The answer is more like "a good chance at least one will be missed" or "probably none".

Later, you’ll be asked to estimate the lifetime of the test. Those answers will be more like "probably not past this release" or "a long time" than "34.6 weeks".

Then you’ll be asked to estimate the number of bugs the automated test will find in that lifetime. The answer will again be indefinite.

And finally, you’ll be asked to compare the fuzzy estimate for the manual test to the fuzzy estimate for the automated test and make a decision.