|
|
[center]Automation test, silver bullet or snake oil?[/center]
Echo Chih Zhang 2005/5/22
After having done a research on automation test, I'm quite cofused about the profit from testing automation. This article is aim to unveil some theoretical analysis of cost and benifit of automation test.
What's the automation test?
It's a dream for every tester that one day all test cases can be executed automatically and what he should do is just checking the results of those tests to find which fails and report that bug. However it's just a dream, a day dream...
Admittedly, automation test can decrease the pressure of testers especially for those unrealistic schedule. But it doesn't mean that all test cases can be automated and throw away manual tests. Automation test is not test automation!
According to Cem Kaner's theory, automation test can be defined as computer assisted test, which means tester should treat it as a way to find bugs and it just more depends on computer's assistence. Definitely, whichever way tester used, the target of test is to find bugs as more as better. Automation test is one method to achieve it. It can be very useful, but we don't want to confuse ourselves by calling it more than it is.
The direct costs of automation test
When a company decides to automate test, usually it will assign or hire someone to design, code, run and check the automation test cases. The cost of such procedure is quite expensive, not only because the salary company paid and the cost of test tool's license but also the low productivity of automation test. Normally speaking, it takes 3-5 times time to accomplish an automated test case comparing with a manual one and this statistic is based on a skilled automation tester, which dosen't take account of the training expense of automation test.
With automation test furtherly progressing, the maintenace fee can be dramatically increase. It invovles the change of code to maitain the consistency between tested program and test script, also the cost of document maitenace should be taken into account.
The indirect costs of automation test
It is quite obvious that the team can not generate the same amount of manual test cases if they assign one of members to automate test cases. Because most of automation tests take more time than the manual counterparts and they are designed basing on those generated manual test cases.
So a new issue raises, as we all know, the later we find bugs the more we spend to fix them. For example, we have a 10 member's team and they can create and run 100 test cases per week, then one of them turn to automate test cases. So the productivity decrease to 90 per week, and the other 10 test cases will postpone to next week. Correspondingly, the bugs can be found in those 10 test cases will also delay to next week. As time cumulating, the bugs will be found more and more late! For the company side, such situation is not their expectation and they may increase the testing staff in order to achieve the same breadth of testing.
The benifit of automation test
The fantasy is that automation test will make the tester's job simpler and easier and help them contend with unrealistic schedule. It is quite pratical for the regression test. If a new version release tomorrow and there are few bugs fixed today, testers should run most the functional test cases in order to ensure these modification will not affect the stem of that software. Because of the time limitation, it sounds like mission impossible due to the large quantity of functional test cases. Fortunately, the automation test makes it possible to attain.
The most benifit we can achieve from automation tests is that there is a next version to release or a significant change in the code.
Conclusion
Through the analysis of the cost and benifit of automation test, it is not an easy thing to implement automation tests. From the cost efficiency side, it is a long term return of investment. |
|
/1 
关于我们
发表于 2005-5-23 17:30:15
QQ好友和群
QQ空间
腾讯微博
腾讯朋友
微信
收藏
置顶服务
换色服务
