Defect severity and priority

connie

链接地址：http://www.testingreflections.com/node/view/3040
Submitted by Ainars Galvans on Fri,02/12/2005

Questions are frequently asked regarding evaluating defect priority, severity and differentiating between the two values. I would like to explain my vision, based on best practices in our company and supported by IEEE standard, and some recent publications by recognized experts in testing area.
Issue
I chose IEEE standard, because it address issue of evaluating different properties of the defect. For example CMM talks only about severity and type. IEEE (IEEE Std 1044-1993) consider a lot of different values for an anomaly. It requires “The impact of an anomaly shall be considered at each step of the anomaly process”. However I will only talk about initial evaluation by tester. Further IEEE state that “Identifying the Severity of an anomaly is a mandatory category as is identifying the Project schedule and Project cost impacts of any possible solutions for the anomaly.” The standard only suggest priority to be evaluated: “Additional categories that may be useful are Customer value and Priority.”
IEEE see Priority and Customer Values almost the same values even suggesting in Annex that “Generally, this will be the same as the Customer Value“. It is clear about priority – it is option property to describe the customer value or business value to the company. Now the severity. Annex suggests that severity stands for the impact “on the program operation.”. Project schedule and cost properties evaluate time and money “to address a defect or enhancement”. I would like to stress the term address usage that implies development cost to be extended with any other, including testing and risk of regression.
Solution path
Test engineer could naturally evaluate the value of severity: impact on system operation. It is not so simple with schedule and cost. Test engineer can’t/shouldn’t evaluate development cost and time. What test engineer could evaluate is risk of regression and any other type of defects found upon addressing this one. More over if tester would be able to perfectly evaluate such a risk, then isn’t this the most natural way to evaluate the priority for a developer to fix the problems? However most of the testers will say that they are unable to forecast count and size of defects that developers will implement while addressing the defect.
If you look at IEEE suggested severity values/epxlanations (in samples)you will find that severity exactly maps to hidden defects to be found later (if we skip the sentence about unrecoverable data loss). Isn’t this great? I suggest to only care about other types of defects to be found later while evaluating project cost and schedule.
Everyone knows there is a risk of regression defects. The regression risk for a single defect fix depends on development activities required to address the issue (do we change core or standalone functionality, how many modules involved, etc). So this is again what developers could recognize best.
I believe that we only have got one more type of defects to be detected when the defect is fixed. I call them “blinking defects”. The idea is not so new. Let see what the authors of "Lessons Learned in Software Testing" says about defects to be found later. Lesson 186: “Never budget for just two testing cycles.” Beside describing regression and hidden bugs as well as bugs fixed incorrectly authors point out as the first item “As you learn more about the product you will think of new and better tests”. If it would be so simple we would be able to simply repeat the first cycle before all defects are fixed and avoid finding those types of defects later. I would like to re-design this idea to “As product become functionally stable (quality) you will think of new and better tests“ .
The issue is that you can’t evaluate the product stability evaluating single defect/anomaly. I will address this final issue in my future post “run-log evaluation”.
Conclusions
Tester must naturally evaluate only the severity of the defect in terms of defect impact on the system operation. Tester may suggest other values, such as priority in terms of business value (or even evaluate them, in case when tester is wearing other hats in the project, such as business analyst, system architect, etc).
Optionally tester could set the property describing impact on test data, test environment or even company environment (e.g. impact of local network) along with probability for a tester to occasionally repeat the problem (supposed that he is informed to avoid it). However such an issues seems to be exceptional and such defects could be alternatively processed/escalated without supporting property that evaluates this option.
Additional considerations (next post ideas)
Additionally tester should evaluate risk of what I call “blinking defects” and suggest strategy for defect fix further prioritization based on that. However this is not a value of a single defect/anomaly. “Blinking defects” are property for the set of anomalies detected in a single test case/test procedure or functional unit.

xyzwh

先前我弄不清楚severity和priority的区别，我们提交bug的时候severity是必填字段，priority几乎没关注过。
看了lz帖子后，清楚些了
测是工程师会根据以下几方面来评估severity：
1）对系统操作的影响，而不是项目的进度和成本。
2）回归测试的风险，解决此bug而产生的其他的bug
如果测是工程师能很好的评估severity,就能很好的为开发人员评估出那些bug是需要优先解决的，priority的问题就很好的解决了。
只是对
risk of regression defects
regression risk 有些不懂
为什么要翻译成衰减啊？我一直都理解成回归？？

[ 本帖最后由 xyzwh 于 2008-8-1 18:06 编辑 ]

connie

to jessica2005: 翻译的不错，可以在某些词句上加以斟酌，有些语句需要理顺一下：）

connie

原帖由 jessica2005 于 2005-12-12 15:35 发表
楼上的大哥，应该说说优先级么，这样也好让人家翻译，不然随便找一篇翻译就没有价值了。

大家可以自己在版面选择翻译的内容，贴过来的文章都是挑选过的：）

alice10

不错不错，一下翻译了这么多！

jessica2005

我们要讨论的问题是经常被问到的关于评估错误的优先级，严重程度以及问题如何区别并协同工作。

我将根据公司的实际情况以及IEEE标准的支持以及一些最近出版的并且经过测试领域专家公认的，来说

明我的观点。


    问题
   
    我之所以选择IEEE标准，是因为它是用于评估软件错误的不同特性。打个比方：CMM解决关于严重性

和类型的问题。IEEE(IEEE Std 1044-1993)则考虑到了许多异常问题的不同方面。它要求：处理每个异常

的步骤都要考虑到异常问题所造成的影响。在这里我们仅讨论，测试者对问题作出的初始评估。 此外，

IEEE进一步规定：强制要求定义异常问题的严重性，并且作为解决异常问题造成的额外开销和对项目进度的影响的依据。IEEE标准建议将优先

级作为附加项评估用户使用价值和优先程度。

IEEE将用户使用价值等同于优先度，从附加建议中可见一般：“总的来说，它等同于使用价值"，它明确说明了优先度，是作为描述用户使用价

值和公司商业价值的可选项。用于“定位错误和改进问题”的人力物力衡量项目的进度和花费。

解决途径：

测试工程师客观定义问题的严重度:它并不是简单的进度和花费的问题。它影响到系统的运转。
测试工程师不能或者说不应该评估开发人员的开销和时间。测试工程师要评估的是系统运转的风险和由此产生的其他问题。在者，如果测试工

程师能够很好的评估这样的风险，不正是很客观的评估了开发人员修正问题的优先度么？然而，多数的测试人员会说，他们不能够计算出风险

，或者衡量出问题的范围，而开发人员去定位问题时会发现的。

如果你看看IEEE中对严重度的建议值的说明和样例，你会发现：严重度严密的描述，如果跳过关于不可重现的数据的描述，会错失一些隐含问

题的重现。这不是很棒么？当然，我所指的仅是那些稍后被发现的问题，在评估项目的花费和进度时所需要考虑的。

每个人都知道，问题有个衰退的风险。衰退的风险缘于程序开发人员对问题所作的修改活动。我们是否改变了功能的核心或者是就问题本身的

修改，我们修改所涉及到的模块，诸如此类问题）。因此这又关系到开发人员是否能够很好的鉴别问题。

我认为是我们修改了一个问题的同时，最好是认识到了这个类型的问题。
我称她们为 不可重现的问题。这个论点并不新。让我来看看“软件测试经验”的作者关于后期问题发现所作的说明。
第186课：比较作者关于衰退期、隐含问题以及问题的错误修正所作的描述中指出的作为首要的一条：了解到产品越多，就越要考虑新的更好的测试。

永远不要只做两轮测试的预计。如果在所有问题被修正之前我们仅仅进行第一次测试，就能够避免以后发现问题的话，那就太简单了。
我将他重新描述为：随着产品质量的稳定，需要考虑新的更好的测试。

问题是：不能够以不变的方式的评估产品的稳定性，评估单个问题和异常现象。我将会在下个段落“的评估”，最后的问题中说明此问题

结论：
当问题关乎系统的运转时，测试人员必须客观的评估问题的严重性。测试人员也许会提出其他的建议，比如说涉及到商业价值的优先级，当测

试人员做出了商业价值的评估，就好像他们在项目中担任着商业评估师或者系统架构师的责任。测试人员依靠偶然重现的问题（假定他见多识

广能够每每重现），随意的测试数据来描述对测试环境甚或是对公司环境（局域网）的影响。
然而，此类问题也有例外，虽然没有这些特性说明的支持，也会转变或者是等级升高。

附加的事项：
此外，测试人员需要评估我们称之为“不可重现”的问题，为这些问题设计策略以此进一步确定优先级。然而，这不是单个问题或是异常事件

的值。“不可重现”是测试过程中或者功能模块中检测出的一组异常事件的特性。

jessica2005

楼上的大哥，应该说说优先级么，这样也好让人家翻译，不然随便找一篇翻译就没有价值了。

jessica2005

我们要讨论的问题是经常被问到的关于评估错误的优先级，严重程度以及问题如何区别并协同工作。

我将根据公司的实际情况以及IEEE标准的支持以及一些最近出版的并且经过测试领域专家公认的，来说

明我的观点。


    问题
   
    我之所以选择IEEE标准，是因为它是用于评估软件错误的不同特性。打个比方：CMM解决关于严重性

和类型的问题。IEEE(IEEE Std 1044-1993)则考虑到了许多异常问题的不同方面。它要求：处理每个异常

的步骤都要考虑到异常问题所造成的影响。在这里我们仅讨论，测试者对问题作出的初始评估。 此外，

IEEE进一步规定：强制要求定义异常问题的严重性，并且作为解决异常问题造成的额外开销和对项目进度的影响的依据。IEEE标准建议将优先级作为附加项评估用户使用价值和优先程度。

IEEE将用户使用价值等同于优先度，从附加建议中可见一般：“总的来说，它等同于使用价值"，它明确说明了优先度，是作为描述用户使用价值和公司商业价值的可选项。

connie

to 李洁：
you can choose the articles by yourself here or anywhere else.
welcome to the family

李洁

Do you want me to translate chinese ?
whatever you want me translate chinese or english ,it's no problem to me!
please answer me as soon as you can!!!!

xiaoye_china

to :upstair
  you are right!  But I want to know what your really means?  
thank you!

connie

to xiaoye_china:you mean explain or translate?

xiaoye_china

Would  you  find  somebody  to  explain this article?