（续）Theme Two: Planning the Testing Effort 典型测试错误（二）——计划测试工作

pele

How about avoiding testing altogether?

  不做测试会怎么样？

  At a conference last year, I met (separately) two depressed testers who told me their management was of the opinion that the World Wide Web could reduce testing costs. "Look at [wildly successful internet company]. They distribute betas over the network and get their customers to do the testing for free!" The Windows 95 beta program is also cited in similar ways.

  在去年的一个会议上，我（分别）遇到两个沮丧的测试员，他们告诉我他们的管理是基于这样一种意见：万维网（World Wide Web）可以减少测试成本。“看看非常成功的网络公司”。他们在网络上分发β版，让客户免费给他们做测试！”。Windows 95的β程序也是这样的。

  Beware of an overreliance on beta testing. Beta testing seems to give you test cases representative of customer use - because the test cases are customer use. Also, bugs reported by customers are by definition those important to customers. However, there are several problems:

  要当心对β测试的过分依赖。因为测试用例是客户使用的，所以β测试似乎是给了你客户使用的代表用例。另外，客户报告的错误也是对客户重要的。但是，有几个问题：

  1. The customers probably aren't that representative. In the common high-tech marketing model, beta users, especially those of the "put it on your web site and they will download" sort, are the early adopters, those who like to tinker with new technologies. They are not the pragmatists, those who want to wait until the technology is proven and safe to adopt. The usage patterns of these two groups are different, as are the kinds of bugs they consider important. In particular, early adopters have a high tolerance for bugs with workarounds and for bugs that "just go away" when they reload the program. Pragmatists, who are much less tolerant, make up the large majority of the market.

  客户可能不是代表。在一个普通的高科技市场营销模型中，β用户，特别是那种“将产品放到网站上让他们下载”的情况，是早期的采用者。他们喜欢摆弄新技术。他们不是实用主义者，不是那种愿意等到新技术被证明是安全可靠后才采用的人。这两种类别的使用方式是不同的，就像他们认为 bug 的重要程度是不同的一样。特别地，早期的采用者对于能够用变通方法解决的 bug和重新加载程序就能消失的 bug有较强的容忍性。但容忍性较差的实用主义者占据了市场的大部分。

  2. Even of those beta users who actually use the product, most will not use it seriously. They will give it the equivalent of a quick test drive, rather than taking the whole family for a two week vacation. As any car buyer knows, the test drive often leaves unpleasant features undiscovered.

  即使是那些实际使用产品的β用户，大多数也不会认真地使用。他们会给一个类似于试驾车的快速测试，而不是带着整个家庭休假两周。很多购买汽车的人都知道，试驾车经常会遗漏一些令人不愉快的特性。

  3. Beta users - just like customers in general - don't report usability problems unless prompted. They simply silently decide they won't buy the final version.

  β用户象客户一样，除非特别要求，一般不会报告可用性错误。他们只是暗自决定不去购买最终产品。

  4. Beta users - just like customers in general - often won't report a bug, especially if they're not sure what they did to cause it, or if they think it is obvious enough that someone else must have already reported it.

  β用户象客户一样，常常不会报告 bug ，尤其是当他们不能确定是什么操作导致了错误，或者是他们认为这个错误很明显，其他人肯定已经报告了。

  5. When beta users report a bug, the bug report is often unusable. It costs much more time and effort to handle a user bug report than one generated internally.

  当β用户报告错误时，错误报告常常无法使用。处理一个用户的错误报告比一个内部产生的错误报告要花费多得多的时间和精力。

  Beta programs can be useful, but they require careful planning and monitoring if they are to do more than give a warm fuzzy feeling that at least some customers have used the product before it's inflicted on all of them. See [Kaner93] for a brief description.

  β程序可能是有用的，但是需要仔细的计划和监督，否则它们在激怒所有β客户之前，除了带来一种模糊的、兴奋的感觉，认为至少有一些客户在使用产品之外，不会后其他收获。参见[kaner93]以获取一个简要描述。

  The one situation in which beta programs are unequivocally useful is in configuration testing. For any possible screwy configuration, you can find a beta user who has it. You can do much more configuration testing than would be possible in an in-house lab (or even perhaps an outsourced testing agency). Beta users won't do as thorough a job as a trained tester, but they'll catch gross errors of the "BackupBuster doesn't work on this brand of 'compatible' floppy tape drive" sort.

  β测试有用的一种情况是配置测试。对于任何古怪的配置，你都可以找到一个使用此配置的β用户。你可以做比机构内部实验室（或者甚至是外包给测试机构）多的配置测试。β用户不会象一个训练有素的测试员一样做完整的测试，但他们可以捕捉到大致错误，像“BackupBuster在这个品牌的兼容磁带驱动器上不能工作”。

  Beta programs are also useful for building word of mouth advertising, getting "first glance" reviews in magazines, supporting third-party vendors who will build their product on top of yours, and so on. Those are properly marketing activities, not testing.

  β程序也有助于建立口头的广告，获得杂志的“第一印象”评论，支持第三方供应商在你的产品上构建他们的产品等等。这些都是正常的市场营销活动，不是测试。

  Planning and replanning in support of the role of testing

  计划和重新计划测试的支持作用

  Each of the types of testing described above, including functional testing, reduces uncertainty about a particular aspect of the product. When done, you have confidence that some functional areas are less buggy, others more. The product either usually works on new configurations, or it doesn't.

  上面所描述的包括功能测试在内的各种类型的测试，减少了产品某一方面的不确定性。在执行完毕后，你可以确信某些功能领域的错误较少了，其他的还比较多。产品通常将在新配置中起作用，或者是不起作用。

  There's a natural tendency toward finishing one testing task before moving on to the next, but that may lead you to discover bad news too late. It's better to know something about all areas than everything about a few. When you've discovered where the problem areas lie, you can test them to greater depth as a way of helping the developers raise the quality by finding the important bugs.

  有一种很自然的倾向，就是在进行到下一个测试任务之前先完成一个任务，但这可能导致你过晚地发现坏消息。对所有领域都了解一些比深入了解几个领域更重要。如果你发现了问题在哪个地方，你可以更深入地测试它们，通过发现重要 bug来帮助开发人员提高质量。

  Strictly, I've been over-simplistic in describing testing's role as reducing uncertainty. It would be better to say "risk-weighted uncertainty". Some areas in the product are riskier than others, perhaps because they're used by more customers or because failures in that area would be particularly severe. Riskier areas require more certainty. Failing to correctly identify risky areas is a common mistake, and it leads to misallocated testing effort. There are two sound approaches for identifying risky areas:

  严格地说，我对将测试的作用描述为减少不确定性是太简单了。更恰当的说法是“风险加权”的不确定性。产品中某些领域比其他领域更有风险，也许是因为它们由更多客户使用或是因为那个领域的故障更严重。危险性高的区域需要更好的稳定性。不能正确地识别危险区域是一个常犯的错误，它导致测试工作的不恰当分配。

  1. Ask everyone you can for their opinion. Gather data from developers, marketers, technical writers, customer support people, and whatever customer representatives you can find. See [Kaner96a] for a good description of this kind of collaborative test planning.

  向每一个能够找到的人征询意见。从开发人员、市场人员、技术写作人员、客户支持人员和你能找到的每一个客户代表那里收集意见。查看[Kaner96a]以获得关于这种协同测试计划的描述。

  2. Use historical data. Analyzing bug reports from past products (especially those from customers, but also internal bug reports) helps tell you what areas to explore in this project.

  使用历史数据。分析以前产品的 bug 报告（特别是来自客户的，但也要包含内部 bug 报告）可以帮助你辨别在这个项目中还需要探索哪些领域。

  "So, winter's early this year. We're still going to invade Russia."

  “今年冬天来得很早。但我们还是要入侵俄国。”

  Good testers are systematic and organized, yet they are exposed to all the chaos and twists and turns and changes of plan typical of a software development project. In fact, the chaos is magnified by the time it gets to testers, because of their position at the end of the food chain and typically low status. One unfortunate reaction is sticking stubbornly to the test plan. Emotionally, this can be very satisfying: "They can flail around however they like, but I'm going to hunker down and do my job." The problem is that your job is not to write tests. It's to find the bugs that matter in the areas of greatest uncertainty and risk, and ignoring changes in the reality of the product and project can mean that your testing becomes irrelevant.

  好的测试员是有计划、有组织的，但他们受到计划，特别是软件开发项目计划的各种混乱、各种意外转折的影响，因为他们处于食物链的最后一环，而且通常地位比较低。一个不幸的反应是固执地坚持测试计划。从感情上讲，这会令人很满意：“他们可以随意胡乱摆弄，但我要坐下来做我的工作。”但问题是你的工作不是编写测试。而是在最不确定和危险的领域发现 bug 。忽略产品和项目的实际变化可能意味着你的测试变得无关紧要。

  That's not to say that testers should jump to readjust all their plans whenever there's a shift in the wind, but my experience is that more testers let their plans fossilize than overreact to project change.

  这不是说测试员在有任何变化时都应该匆忙地重新调节他们的计划，但我的经验是很多的测试员都让计划僵化而不是对项目变化起过度的反应。

鱼鳞

自己翻译的吗？顶！

ioiojoyer

bucuo

ioiojoyer

up 一下

fuzhijuan

向你学习