How to choose a regression testing strategy

connie · 发表于 2005-11-29 10:31:31

What is regression testing?

"It's a funny thing about testing - the worse software is when it gets to us, the longer it takes to get it out."
Summary: From my experience, regression testing of any type of an application is the most time-consuming, and is sometimes even a bottleneck. In this article I will try to summarize my research and experience in choosing a strategy and running regression testing suites.

From my experience, regression testing of any type of an application is the most time-consuming, and is sometimes even a bottleneck. In this article I will try to summarize my research and experience in choosing a strategy and running regression testing suites. If SDLC is an iterative development, inserting regression testing into the schedule during the second iteration without any additional resources already becomes a problem (my assumption is that the number of test cases may be doubled from the fist iteration to the second). From my point of view the second iteration is not the right time to automate regression testing. I will not discuss any automation aspects of regression testing in this article. To survive, you have to automate regression testing after developing and stabilizing the application anyway. If you have reached 80% coverage of the existing TC (test cases) by automation (that is an excellent result - most automation gives 50% TC coverage from my own experience) you will still need to run the rest of the TC manually. Even automated tool providers expect the manual TC. For example, "Rational" included a manual TC option in the "Test Manager" 2002 version. So let us assume that we need to run a set of TC manually. The set of TC is growing dramatically during the SDLC process of development. So you can't allocate time and resources to run all existing TC every time you make any changes in the application. To maintain and deliver good software to the client, you must define a proper strategy for regression testing that matches your goals and targets from the beginning. If you are initially involved in the development process and can decrease the variability of the system it will be great but can be discussed in a different article.

Assumptions:
a. If you have a TC, you must run it.
b. The number of TC for every non-trivial system aspires to infinity.
c. Some TC are more important than others.
d. This is not safety-critical software.

Let us begin with some popular definition of regression testing:
1. Regression testing - testing that is performed after making a functional improvement or repair to the program. Its purpose is to determine if the change has regressed other aspects of the program [Myers, 1979]
2. Regression Testing - any repetition of tests (usually after software or data change) intended to show that the software's behavior is unchanged except insofar as required by change to the software or data. [B. Beizer, 1990]
3. Regression Testing - testing conducted for the purpose of evaluating whether or not a change to the system (all CM items) has introduced a new failure. Regression testing is often accomplished through the construction, execution and analysis of product and system tests.
4.Regression testing - re-testing after fixes or modifications of the software or its environment. It can be difficult to determine how much re-testing is needed, especially near the end of the development cycle. Automated testing tools can be especially useful for this type of testing.
5. Regression testing - rerunning test cases, which a program has previously executed, correctly in order to detect errors spawned by changes or corrections made during software development and maintenance.
6. Regression testing - selective retesting of system or component to verify that modifications have not caused unintended effects and that system still complies with its specified requirements. [IEEE 610]

Let us define the most popular strategies for selecting regression test suites:
1. Retest all. Rerun all existing TC. Simple but impossible in the time that we have in our everyday practice.
2. Retest Risky Use Cases. Choose baseline tests to rerun by risk heuristics. Here we are speaking about RUP (Rational Unified Process development activities) for details see: http://www.rational.com/products/rup/
3. Retest By profile. Choose baseline tests to rerun by allocating time in proportion to operational profile.
4. Retest Changed Segment. Choose baseline tests to rerun by comparing code changes (White box strategy).
5. Retest within Firewall. Choose baseline tests to rerun by analyzing dependencies (White box strategy). [R. V. Binder, 1999]
6. Apply hierarchical increment testing (HIT) (close to Retest within Firewall). [John D. McGregor, 2001]
7. Apply Black Box Monkey Testing - [Thomas R. Arnold, 1998]
It is possible to write a separate article about any of these strategies and a separate book about their comparison. If you are interested in details please refer to Binder and McGregor books. I am trying to use the KISS law (keep it stupid simple) and think that when you need to choose something and it depends on your or somebody else's opinion - mistakes can take place. In my regression testing strategy, I am trying to minimize all human factors in common.

c_caibin · 发表于 2008-5-9 13:02:27

说了，慢慢看

k99378 · 发表于 2008-3-8 11:51:24

什么是回归测试？

"这是一个有趣的关于测试-更糟的是软件时，得到对我们来说，时间越长，才获得它" 。
综述：从我的经验来说，回归测试任何类型的应用是最消耗时间，有时甚至是一个瓶颈。在这篇文章中我会尝试总结我的研究和经验，在选择策略和运行回归测试套件。

根据我的经验，回归测试任何类型的应用是最消耗时间，有时甚至是一个瓶颈。在这篇文章中我会尝试总结我的研究和经验，在选择策略和运行回归测试套件。如果 sdlc是一个反复发展，插入回归测试纳入附表在第二次迭代过程不需任何额外的资源，已成为一个问题（我的假设是，有多少测试案例可能会增加一倍，从拳头迭代到第二）。从我的角度来看，第二迭代是不正确的时间进行自动化回归测试。我不会讨论任何自动化方面的回归测试，在这篇文章。为了生存，你必须自动化回归测试后，发展和稳定的应用，无论如何。如果您已达到百分之八十的篇幅阐述了与TC （测试用例）自动化（即是一个很好的结果-最自动化，让百分之五十与T C覆盖率由我自己的经验），你还是需要运行，其余的增距镜手工操作。即使自动化工具提供商期望手册与TC 。举例来说， "理性" ，包括一本手册与TC选项，在"测试经理" ， 2002年版。因此，让我们假定我们需要运行一套TC的手工操作。设置的TC是大幅增长，在sdlc的发展过程。所以你不能拨出时间和资源来运行所有与 TC每一次当你作任何改变，在其申请。保持和提供良好的软件给客户，你必须确定一个适当的策略，回归测试符合你的目标和指标，从一开始的。如果你是最初参与发展进程，并能减少变异的系统，这将是巨大的，但都可以谈，在不同的文章。

假设：
答：如果你有一个总胆固醇，你必须安装它。
乙有多少的TC每非平凡系统渴望无限大。
长一些与TC更重要，比别人。
四这不是安全性至关重要的软件。

让我们开始与一些热门的定义回归测试：
1 。回归测试-测试是演出后，使一个功能改善或修补程式。其目的是要确定，如果改变退化了其他方面的计划[迈尔斯， 1979 ]
2 。回归测试-任何重复的测试（通常是经过软件或数据变动），为了表明该软件的行为是不变的，除非所要求的改变，软件或数据。 [乙beizer ， 1990 ]
3 。回归测试-测试进行，目的是评价与否，要改变这种系统（所有公分项目）引入了新的失败。回归测试往往是通过建设，执行和分析产品和系统的测试。
4.regression测试-再测试后，补丁或修改软件或其环境。因此很难确定有多少复试是必要的，特别是接近尾声的开发周期。自动化测试工具可以得到有用的，尤其是对于这种类型的测试。
5 。回归测试-重新测试案例，其中计划以前执行的，是正确的，以探测误差产生的更改或更正期间所作的软件开发和维护。
6 。回归测试-选择性复检的系统或组件，以确认修改没有造成意外的影响，以及系统仍然遵守其规定的要求。 [的IEEE 610 ]

让我们确定最受欢迎的战略选择回归测试套件：
1 。复试。重新运行所有现有的技术合作。简单，但不可能在一段时间，我们不得不在我们的日常实践。
2 。复试冒险的用例。选择基准测试，以重新由风险启发式。在这里，我们正在谈论的RUP （ Rational统一过程的开发活动），详情请参阅： http://www.rational.com/products/rup/
3 。复试由剖面。选择基准测试，以重新分配的时间比例，以运行剖面。
4 。复试的改变部分。选择基准测试，以重新运行比较更改代码（白盒子战略）。
5 。复试的内部防火墙。选择基准测试，以重新分析相依（白盒子战略）。 [传译五，粘结剂， 1999年]
6 。适用于分层增量测试（热闹）（接近复试的内部防火墙）。 [约翰D理觉， 2001年]
7 。申请黑匣子猴测试-[托马斯传译阿诺德， 1 998年]
这是有可能写一篇单独的文章对上述任何一个策略和一个单独的图书对他们的比较。如果您有兴趣的详细资讯，请参阅粘结剂和理觉书籍。我尝试用吻法（保持愚蠢简单），并认为，当你需要选择的东西，这取决于你的或别人的意见-错误可以发生。在我的回归测试策略，我尝试尽量减少一切人为因素，在常见的。