Effective software testing
//有效的软件测试
Chapter 1. Requirements Phase
//第一章 需求阶段
The most effective testing programs start at the beginning of a project, long before
any program code has been written. The requirements documentation is verified
first; then, in the later stages of the project, testing can concentrate on ensuring the
quality of the application code. Expensive reworking is minimized by eliminating
requirements-related defects early in the project's life, prior to detailed design or
coding work.
The requirements specifications for a software application or system must
ultimately describe its functionality in great detail. One of the most challenging
aspects of requirements development is communicating with the people who are
supplying the requirements. Each requirement should be stated precisely and
clearly, so it can be understood in the same way by everyone who reads it.
If there is a consistent way of documenting requirements, it is possible for the
stakeholders responsible for requirements gathering to effectively participate in the requirements process. As soon as a requirement is made visible, it can be tested and clarified by asking the stakeholders detailed questions. A variety of
requirement tests can be applied to ensure that each requirement is relevant, and
that everyone has the same understanding of its meaning.
//最有效的测试是从项目一开始就展开的，远远在任何代码编写完成之前。首先是需求文档被验证，然后再后面的阶段中，测试可以集中确保应用程序代码的质量。在软件生命周期中，通过剔除和需求相关的缺陷可以减少昂贵的重复劳动，这样比在设计或编码工作时发现这类缺陷优越得多。
软件应用程序或者系统的需求规格说明书一定是最终描述功能的细节。在建立需求的过程中最重要的挑战之一就是和提供需求人员进行交流。每条需求都需要描述的精准和清晰，这样可以让任何读需求的人可以有相同的理解。如果有一套文档需求的统一的方法，那么利益相关者在需求阶段被集中并有效参与需求过程就可能成为他们的一种责任。只要需求是可见的，那么就是可以被测试的，可以通过问利益相关者问题来阐明的。各种各样的需求测试可以被申请，用以确保每个需求是中肯的，每个人都有相同的理解，没有二义性。
Item 1: Involve Testers from the Beginning
//第一节 在项目开始时引入测试
Testers need to be involved from the beginning of a project's life cycle so they can
understand exactly what they are testing and can work with other stakeholders to
create testable requirements.
//测试人员需要在软件生命周期的起始阶段就进行参与，这样可以让他们更准确的理解他们要测试的东西并且可以和利益相关者一起打造可测的需求。
Defect prevention is the use of techniques and processes that can help detect and
avoid errors before they propagate to later development phases. Defect prevention
is most effective during the requirements phase, when the impact of a change
required to fix a defect is low: The only modifications will be to requirements
documentation and possibly to the testing plan, also being developed during this
phase. If testers (along with other stakeholders) are involved from the beginning of the development life cycle, they can help recognize omissions, discrepancies,
ambiguities, and other problems that may affect the project requirements' testability, correctness, and other qualities.
//缺陷预防是技术和流程，这些技术和流程可以帮助缺陷在后面开发阶段被传播前就被发现和避免。缺陷预防是在需求阶段最有效的，因为需求的改变而修复缺陷的影响也最小：仅仅是需求文档的修改或者测试计划的修改，也或者是在开发阶段。如果测试人员（还有利益相关者）在开发周期一开始就介入的话，他们可以帮助识别出遗漏、偏差以及含糊不清之处，以及其他影响项目需求的可测试性、正确性及其他质量问题。
A requirement can be considered testable if it is possible to design a procedure in
which the functionality being tested can be executed, the expected output is
known, and the output can be programmatically or visually verified.
//一个需求被认为是可测的：如果可以设计一个过程，这个过程中功能可以被执行测试，期望结果是可知的，并且输出结果可以被程序化地和可视地验证。
***************
以上是这两天的翻译成果，后面继续努力，翻译不对的地方也欢迎各位指正。
Testers need a solid understanding of the product so they can devise better and
more complete test plans, designs, procedures, and cases. Early test-team
involvement can eliminate confusion about functional behavior later in the project
life cycle. In addition, early involvement allows the test team to learn over time
which aspects of the application are the most critical to the end user and which are
the highest-risk elements. This knowledge enables testers to focus on the most
important parts of the application first, avoiding over-testing rarely used areas and
under-testing the more important ones.
Some organizations regard testers strictly as consumers of the requirements and
other software development work products, requiring them to learn the application
and domain as software builds are delivered to the testers, instead of involving
them during the earlier phases. This may be acceptable in smaller projects, but in
complex environments it is not realistic to expect testers to find all significant
defects if their first exposure to the application is after it has already been through
requirements, analysis, design, and some software implementation. More than just
understanding the "inputs and outputs" of the software, testers need deeper
knowledge that can come only from understanding the thought process used during the specification of product functionality. Such understanding not only increases the quality and depth of the test procedures developed, but also allows testers to provide feedback regarding the requirements.
//测试人员对产品要有比较扎实的理解，这样他们就可以较好地做出完整的测试计划、测试设计、测试过程和测试用例。早期的团队参与可以剔除在开发周期后期的关于功能行为的混淆。另外，早期的参与可以让测试团队有更多的时间在学习软件，了解什么是对最终用户最关键的，这也是最高风险的。这些知识可以让测试人员在一开始就知道软件的最重要的部分，避免在不重要的地方过度测试而在重要的地方测试力度不足。
一些机构把测试人员就是严格意义上的软件需求和其他工作产品的消费者，在交付给测试人员软件版本时要求他们去学习应用和领域，而不是在更早的阶段就引入他们。这在小项目中式可以接受的，但在复杂的环境下，没有测试人员去发现重要的缺陷，而是在经过了需求、设计、编码甚至一些部署后才首次发现这些缺陷，那是不现实的。不但要理解软件输入输出，测试人员需要更深层的知识，这些知识只能来自于对需求规格说明书中产品功能的思考过程，这种理解不但提高质量和测试过程的深度，而且也让测试人员对需求提供反馈。
The earlier in the life cycle a defect is discovered, the cheaper it will be to fix it.
Table 1.1 outlines the relative cost to correct a defect depending on the life-cycle
stage in which it is discovered.[1]
//在生命周期中越早发现缺陷，越节省修复的费用。
表1中，列出了在生命周期中的不同阶段修复的代价：
[1] B. Littlewood, ed., Software Reliability: Achievement and
Assessment (Henley-on-Thames, England: Alfred Waller, Ltd.,
November 1987).
Table 1.1. Prevention is Cheaper Than Cure: Error Removal CostMultiplies over System Development Life Cycle
PhaseRelative Cost to Correct
Definition $1
High-Level Design$2
Low-Level Design$5
Code$10
Unit Test$15
Integration Test$22
System Test$50
Post-Delivery$100+
Item 2: Verify the Requirements
// 验证需求
In his work on specifying the requirements for buildings, Christopher Alexander[1]
describes setting up a quality measure for each requirement: "The idea is for each
requirement to have a quality measure that makes it possible to divide all solutions to the requirement into two classes: those for which we agree that they fit the requirement and those for which we agree that they do not fit the requirement." In other words, if a quality measure is specified for a requirement, any solution that meets this measure will be acceptable, and any solution that does not meet the measure will not be acceptable. Quality measures are used to test the new system against the requirements.
//在Christopher Alexander 从事的明确需求的工作中，他描述了为每个需求搭建质量度量标准：每条需求都有一个质量度量标准，这样才可能把所有的需求方案分成两种类型：什么可以满足需求，什么不满足需求。换句话说，对于需求的质量度量标准时明确的，那么测试的任何方案都是可接受的，没有度量标准的都是不可以接受的，质量度量标准用来测试新系统而不是需求。
Testers need a solid understanding of the product so they can devise better and
more complete test plans, designs, procedures, and cases. Early test-team
involvement can eliminate confusion about functional behavior later in the project
life cycle. In addition, early involvement allows the test team to learn over time
which aspects of the application are the most critical to the end user and which are
the highest-risk elements. This knowledge enables testers to focus on the most
important parts of the application first, avoiding over-testing rarely used areas and
under-testing the more important ones.
Some organizations regard testers strictly as consumers of the requirements and
other software development work products, requiring them to learn the application
and domain as software builds are delivered to the testers, instead of involving
them during the earlier phases. This may be acceptable in smaller projects, but in
complex environments it is not realistic to expect testers to find all significant
defects if their first exposure to the application is after it has already been through
requirements, analysis, design, and some software implementation. More than just
understanding the "inputs and outputs" of the software, testers need deeper
knowledge that can come only from understanding the thought process used during the specification of product functionality. Such understanding not only increases the quality and depth of the test procedures developed, but also allows testers to provide feedback regarding the requirements.
//测试人员对产品要有比较扎实的理解，这样他们就可以较好地做出完整的测试计划、测试设计、测试过程和测试用例。早期的团队参与可以剔除在开发周期后期的关于功能行为的混淆。另外，早期的参与可以让测试团队有更多的时间在学习软件，了解什么是对最终用户最关键的，这也是最高风险的。这些知识可以让测试人员在一开始就知道软件的最重要的部分，避免在不重要的地方过度测试而在重要的地方测试力度不足。
一些机构把测试人员就是严格意义上的软件需求和其他工作产品的消费者，在交付给测试人员软件版本时要求他们去学习应用和领域，而不是在更早的阶段就引入他们。这在小项目中式可以接受的，但在复杂的环境下，没有测试人员去发现重要的缺陷，而是在经过了需求、设计、编码甚至一些部署后才首次发现这些缺陷，那是不现实的。不但要理解软件输入输出，测试人员需要更深层的知识，这些知识只能来自于对需求规格说明书中产品功能的思考过程，这种理解不但提高质量和测试过程的深度，而且也让测试人员对需求提供反馈。
The earlier in the life cycle a defect is discovered, the cheaper it will be to fix it.
Table 1.1 outlines the relative cost to correct a defect depending on the life-cycle
stage in which it is discovered.[1]
//在生命周期中越早发现缺陷，越节省修复的费用。
表1中，列出了在生命周期中的不同阶段修复的代价：
[1] B. Littlewood, ed., Software Reliability: Achievement and
Assessment (Henley-on-Thames, England: Alfred Waller, Ltd.,
November 1987).
Table 1.1. Prevention is Cheaper Than Cure: Error Removal Cost
Multiplies over System Development Life Cycle
Phase
Relative Cost to Correct
Definition
$1
$2
$5
$10
$15
$22
$50
$100+
Attempting to define the quality measure for a requirement helps to rationalize
fuzzy requirements. For example, everyone would agree with a statement like "the
system must provide good value," but each person may have a different
interpretation of "good value." In devising the scale that must be used to measure
"good value," it will become necessary to identify what that term means.
Sometimes requiring the stakeholders to think about a requirement in this way will lead to defining an agreed-upon quality measure. In other cases, there may be no agreement on a quality measure. One solution would be to replace one vague
requirement with several unambiguous requirements, each with its own quality
measure.[2]
[2] Tom Gilb has developed a notation, called Planguage (for
Planning Language), to specify such quality requirements. His
forthcoming book Competitive Engineering describes
Planguage.
//试图为每条需求定义质量标准可以帮助识别模糊的需求，比如说，所有的人都赞成这句话“系统一定要提供好的值”，但每个人对“有的值”理解却各不相同，设定“好的值”的度量标准是一定需要的，定义小组的意思是非常必要的。有时候也需要利益相关者按照这样的方式思考需求，这样才可能定义一个大家看法一致需求标准。另外一方面，可能在质量准则上没有达成一致，一个解决方案就是用几个明确的需求代替模糊的需求，每个需求都有自己质量准则。
It is important that guidelines for requirement development and documentation be
defined at the outset of the project. In all but the smallest programs, careful
analysis is required to ensure that the system is developed properly. Use cases are
one way to document functional requirements, and can lead to more thorough
system designs and test procedures. (In most of this book, the broad term
requirement will be used to denote any type of specification, whether a use case
or another type of description of functional aspects of the system.)
In addition to functional requirements, it is also important to consider
nonfunctional requirements, such as performance and security, early in the process:
They can determine the technology choices and areas of risk. Nonfunctional
requirements do not endow the system with any specific functions, but rather
constrain or further define how the system will perform any given function.
Functional requirements should be specified along with their associated
nonfunctional requirements. (Chapter 9 discusses nonfunctional requirements.)
//在项目之外建立一套对需求开发和文档定义的指南是非常重要的。除了最小的程序，几乎所有的程序都需要进行仔细的分析以确保系统被正确的开发。Use Cases 是一种文档化功能需求的方法，可以指导整个系统设计和测试的过程。（在这本书中，大部分的广义术语“需求”将用来标识任何类型的规范，不管是use cases 还是其他类型的系统功能描述。）
除了功能需求，非功能需求的考虑也是非常重要的，比如性能和安全。在早期的过程中：
他们可以决定技术的选择和领域风险。非功能需求不是赋予系统任何的特殊功能，而是限制或进一步定义所给定的功能需求的表现的情况。功能需求应该和他们相关的非功能需求是并驾齐驱的。（在第九章中会讨论非功能需求）
Following is a checklist that can be used by testers during the requirements phase
to verify the quality of the requirements.[3],[4] Using this checklist is a first step
toward trapping requirements-related defects as early as possible, so they don't
propagate to subsequent phases, where they would be more difficult and expensive
to find and correct. All stakeholders responsible for requirements should verify that
requirements possess the following attributes.
[3] Suzanne Robertson, "An Early Start To Testing: How To Test
Requirements," paper presented at EuroSTAR 96, Amsterdam,
December 2–6, 1996. Copyright 1996 The Atlantic Systems
Guild Ltd. Used by permission of the author.
[4] Karl Wiegers, Software Requirements (Redmond, Wash.:
Microsoft Press, Sept. 1999).
//以下是一些检查点，测试人员可以在需求阶段验证需求的质量。使用这些检查点是迈向获取尽可能早的需求缺陷的第一步，以使得这些缺陷不再后续阶段继续派生下去，如果那样将是修复起来很困难并且代价比较大。所有对需求负责的利益相关者也应该按照下面的属性来验证需求。
· Correctness of a requirement is judged based on what the user wants. For
example, are the rules and regulations stated correctly? Does the requirement
exactly reflect the user's request? It is imperative that the end user, or a
suitable representative, be involved during the requirements phase.
Correctness can also be judged based on standards. Are the standards being
followed?
//正确性。需求的正确性是根据用户所想的来判断的。举例来说：
是所有的规则和规程的状态都是正确的吗？
需求准确反映了用户的需求吗？
在需求阶段引入终端用户或者合适的代表，是非常重要的，正确性也可以根据一些标准来判断，标准被遵循了吗？
· Completeness ensures that no necessary elements are missing from the
requirement. The goal is to avoid omitting requirements simply because no
one has asked the right questions or examined all of the pertinent source
documents.
Testers should insist that associated nonfunctional requirements, such as
performance, security, usability, compatibility, and accessibility,[5] are
described along with each functional requirement. Nonfunctional
requirements are usually documented in two steps:
[5] Elfriede Dustin et al., "Nonfunctional Requirements," in
Quality Web Systems: Performance, Security, and
Usability (Boston, Mass.: Addison-Wesley, 2002), Sec.
2.5.
//完整性可以确保没有必要的元素从需求中遗漏。目标是避免仅仅因为没有人问正确的问题或者检查源文件引起的遗漏的需求。
测试人员应该坚持：相关的非功能需求（性能、安全、可用性、适应性、可到性等）也应该一起在需求中被描述。非功能需求一般在以下两步中被文档化下来：
1. A system-wide specification is created that defines the nonfunctional
requirements that apply to the system. For example, "The user
interface of the Web system must be compatible with Netscape
Navigator 4.x or higher and Microsoft Internet Explorer 4.x or
higher."
2. Each requirement description should contain a section titled
"Nonfunctional Requirements" documenting any specific
nonfunctional needs of that particular requirement that deviate from
the system-wide nonfunctional specification.
//1、一个系统宽度的需求规格说明书是这样被创建的：定义给功能需求以适应系统。比如说，“一个网站系统的用户界面必须和Netscape Navigator 4.x 或更高的版本以及 Microsoft Internet Explorer 4.x 或更高的版本兼容。”
2、每个需求描述应该包含一个章节叫做“非功能需求”，记录任何具体非功能需求，避免偏离非功能需求的规范。
· Consistency verifies that there are no internal or external contradictions
among the elements within the work products, or between work products. By
asking the question, "Does the specification define every essential subjectmatter
term used within the specification?" we can determine whether the
elements used in the requirement are clear and precise. For example, a
requirements specification that uses the term "viewer" in many places, with
different meanings depending on context, will cause problems during design
or implementation. Without clear and consistent definitions, determining
whether a requirement is correct becomes a matter of opinion.
· Testability (or verifiability) of the requirement confirms that it is possible
to create a test for the requirement, and that an expected result is known and
can be programmatically or visually verified. If a requirement cannot be
tested or otherwise verified, this fact and its associated risks must be stated,
and the requirement must be adjusted if possible so that it can be tested.
// 一致性验证是在工作产品中或者工作产品之间的各元素中，要保持一致，没有内部或者外部条件。可以通过提问一下问题：“在需求规格说明书中是否规范定义了所有将来要用的重要事务？” 我们可以判断用在需求中的元素是否清晰明确。比如：在需求中很多地方用到了“viewer”，根据上下文的意思不同，这个词也有不同的意思，这就会在设计或者实现时引起问题。没有清楚前后一致的定义，判断需求是否正确会变得非常困难。
可测试性（可验证性）是确定需求能否变成一个测试用例，期望结果是否可知，并且是否可以按部就班地进行验证，验证的过程是可视的。如果一个需求是不能被测试或被验证的，那么这个事实和它相关的风险是必须被声明的，而且这个需求在可能的情况下一定要进行调整，使得可以被测试。
*******************
周末愉快！！！
· Feasibility of a requirement ensures that it can can be implemented given
the budget, schedules, technology, and other resources available.
· Necessity verifies that every requirement in the specification is relevant to
the system. To test for relevance or necessity, the tester checks the
requirement against the stated goals for the system. Does this requirement
contribute to those goals? Would excluding this requirement prevent the
system from meeting those goals? Are any other requirements dependent on
this requirement? Some irrelevant requirements are not really requirements,
but proposed solutions.
· Prioritization allows everyone to understand the relative value to
stakeholders of the requirement. Pardee[6] suggests that a scale from 1 to 5 be
used to specify the level of reward for good performance and penalty for bad
performance on a requirement. If a requirement is absolutely vital to the
success of the system, then it has a penalty of 5 and a reward of 5. A
requirement that would be nice to have but is not really vital might have a
penalty of 1 and a reward of 3. The overall value or importance stakeholders
place on a requirement is the sum of its penalties and rewards— in the first
case, 10, and in the second, 4. This knowledge can be used to make
prioritization and trade-off decisions when the time comes to design the
system. This approach needs to balance the perspective of the user (one kind
of stakeholder) against the cost and technical risk associated with a proposed
requirement (the perspective of the developer, another kind of
stakeholder).[7]
// 需求的可行性是确保所给出的是预算、时间、技术以及其他的可用资源是可执行的。
必要性验证在需求说明规格书的每个需求都是和系统相关的。测试是相关还是必要，测试人员检查需求是否支持在需求中阐述的目标。这些需求是为了这些目标而制定的吗？是否包括不符合这些目标的需求？任何别的需求都是依赖这些需求吗？一些不相关的需求不是真正的需求，而是被提议的方案。
优先顺序让大家理解利益相关者的需求的相对价值。Pardee 建议可以用从1-5的级别来区别对需求来说是好的回报还是坏的惩罚。如果一个需求对于一个成功的系统来说是绝对至关重要的，那么它就有5级惩罚和5级奖励。一个需求也是不错的但不是至关重要的，那可能是有1一级惩罚和3级奖励。对于需求，综合值和利益相关者认为的重要性的综合指标是对惩罚和奖励的总结——第一个例子中，值为10，在第二个例子中值为4。这些知识可以用来决定在接下来的设计中先后顺序。这种方法需要权衡用户（一种利益相关者）的观点和所提议的需求相关的费用和技术的风险（开发人员的观点，另外一种利益相关者）。
· Unambiguousness ensures that requirements are stated in a precise and
measurable way. The following is an example of an ambiguous requirement:
"The system must respond quickly to customer inquiries." "Quickly" is
innately ambiguous and subjective, and therefore renders the requirement
untestable. A customer might think "quickly" means within 5 seconds, while
a developer may think it means within 3 minutes. Conversely, a developer
might think it means within 2 seconds and over-engineer a system to meet
unnecessary performance goals.
// 清晰性验证需求是通过一种准确的可用度量的方法进行描述的。下面是一个含糊的需求的例子：“系统必须快速响应客户的需求”。“快速”是一个固有的含糊词语，带有主观意愿，这将导致这个需求不可测试性。一个客户可能认为5秒内是“快速”，而有的开发人员认为在3秒内是“快速”，也有的开发人员认为是2秒内，在整个需求中没有明确的性能目标。
· Traceablity ensures that each requirement is identified in such a way that it
can be associated with all parts of the system where it is used. For any
change to requirements, is it possible to identify all parts of the system
where this change has an effect?
// 可跟踪性验证每个需求都是通过这样的方法可用识别：系统的所有部分都可以根据他过去的位置被关联到。需求的任何变化，可以在识别系统的所有部分都是在哪里变化了吗？
To this point, each requirement has been considered as a separately
identifiable, measurable entity. It is also necessary to consider the
connections among requirements— to understand the effect of one
requirement on others. There must be a way of dealing with a large number
of requirements and the complex connections among them. Suzanne
Robertson[8] suggests that rather than trying to tackle everything
simultaneously, it is better to divide requirements into manageable groups.
This could be a matter of allocating requirements to subsystems, or to
sequential releases based on priority. Once that is done, the connections can
be considered in two phases: first the internal connections among the
requirements in each group, then the connections among the groups. If the
requirements are grouped in a way that minimizes the connections between
groups, the complexity of tracing connections among requirements will be
minimized.
// 从这点上看，每个需求被理解分解为可以识别和度量的单元。考虑到需求之间的联系，理解一个需求对另一个需求的影响，也是非常必要的。一定有一种办法来处理大量的需求和他们之间的复杂联系。Suzanne Robertson 建议，与其试图同时解决所有问题，不如把需求分解为多个可以管理的组。这样可以把需求分解成子系统，或者根据优先级进行按顺序发布。一旦这样做，需求的联系就可以被考虑成两个阶段：第一是每组需求内部的联系，第二阶段是每组之间的联系。如果需求是通过最小化组之间的联系来分组的，那么需求之间的联系的复杂性就降到最低。
[8] Suzanne Robertson, "An Early Start to Testing," op. cit.
Traceability also allows collection of information about individual
requirements and other parts of the system that could be affected if a
requirement changes, such as designs, code, tests, help screens, and so on.
When informed of requirement changes, testers can make sure that all
affected areas are adjusted accordingly.
As soon as a single requirement is available for review, it is possible to start testing that requirement for the aforementioned characteristics. Trapping requirementsrelated
defects as early as they can be identified will prevent incorrect
requirements from being incorporated in the design and implementation, where
they will be more difficult and expensive to find and correct.[9]
[9] T. Capers Jones, Assessment and Control of Software Risks
(Upper Saddle River, N.J.: Prentice Hall PTR, 1994).
After following these steps, the feature set of the application under development is
now outlined and quantified, which allows for better organization, planning,
tracking, and testing of each feature.
// 可跟踪性 允许每个需求的信息的关联和需求变化对其可以产生影响的系统的其他部分，比如设计、编码、测试、帮助等等，当获悉需求发生了变化，测试人员可以确保有影响的地方都相应地做了调整。只要一个需求可以用来评审，那么测试需求上述的特性就是有可能的。跟踪需求的状态的缺陷要在他们尽可能早地被识别，要让不正确的需求在设计和执行阶段尽早剔除，以免越往后面修复代价也越大。
以下几步之后，正在开发的应用程序的功能是提取大纲和量化，以允许为每个特性更好地组织、计划、跟踪、和测试。
| 欢迎光临 51Testing软件测试论坛 (http://bbs.51testing.com/) | Powered by Discuz! X3.2 |