Good testers know when to quit
By: Kristy Pryma ComputerWorld Canada (09 Aug 2002)By the time a piece of software hits your hard drive, it's safe to say that hundreds or even thousands of hours have been invested in its creation. A key part of the development process is testing, which should cover both how the application is structured and whether or not it does what it has been designed to do. These processes are often referred to as white box and black box testing.
Boris Beizer, a Huntingdon Valley, Penn.-based veteran software tester and author of several books on the subject including Black Box Testing, Software Testing Techniques and Software System Testing and Quality Assurance, dislikes the terms "white box" and "black box" because to him they sound too simplistic and mysterious. Beizer prefers the terms structural and behavioural testing.
White box or structural testing is designed to test the way that a piece of software is built.
"It has to be done by a programmer who knows about the guts of the software," Beizer said, explaining why synonyms for this methodology include clear box or glass box testing.
Black box or behavioural testing is a methodology that treats the system as if it were a black box, with no knowledge of the internal code being used during the testing process, and generally focuses on testing functional requirements. This process is also known as functional testing, opaque box testing and closed box testing, and is often completed by someone other than the developer.
"An application can be complete, highly automated and have good theory behind it, but might be totally irrelevant to what it's supposed to do. Imagine testing an application that's supposed to be a word processor, but has been developed as an accounting package. Every instruction that's been written reacts in the way that the developer designed it, but the application doesn't behave the way that it's supposed to for it to be a word processor. That's behavioural testing," Beizer said.
These testing methodologies are complementary, Beizer said, noting that both are done in any well run organization.
"What process you're using depends on where you are in the development cycle," he explained. "Early in the cycle, when you're dealing with small components, you rely mostly on structural testing executed by the programmer, but once you're closer to the end of the cycle, you rely more on behavioural testing."
The key to good testing, Beizer said, is knowing when to say when. Software testing and quality assurance is philosophically very different than testing that has traditionally worked well for other industries including manufacturing.
"You've got to get to the point of making a trade-off with software - you're going to ship software with bugs, because it's not always cost effective to make every little fix. For a company like Boeing, they should be spending billions to fix every single bug, but should a company shipping a word processor spend billions to fix every bug that might affect a few users with a certain configuration? That's where good companies do the trade-off and try to debug to the point of diminishing returns," he said.
Several years ago, Beizer worked with WordPerfect, which he said at the time had a reputation of the best software quality on the market. He used the Word Perfect product along with his Brother laser printer, which he discovered had an Epson printer emulator mode that would make his laser printer act like a dot matrix printer. He discovered that when using printing in this mode, the justification in his WordPerfect documents would be off, which he attributed to a bug in WordPerfect's printer driver.
"I was able to tie that down, and went to WordPerfect with the problem. They came back to me and said that they had studied the bug and came to the conclusion that I was the only person in the world to ever use their laser printer to look like a dot matrix and that it was unlikely that anyone else would ever run into the problem, so they said they weren't going to fix it. That was a wonderful answer, because a good company's philosophy should be to expect bugs," he said.
Alex Samurin, a Toronto-based tester pointed out that besides the traditional black and white box testing, grey box testing has become important for Web and Internet applications. Grey box testing is a strategy based partly on the internals of an application; testing is done from the outside of the product, as in black box testing, but testing choices are informed by knowledge of the application's underlying components. According to Samurin, this grey box methodology is fairly new, and is not yet fully embraced by the testing community, but widespread acceptance is in many ways irrelevant.
"From my point of view, it doesn't matter what colour the cat is as long as it catches the mouse," Samurin said. 从软件驱动硬件的那一刻起，成百上千甚至成千上万的时间被投资于此项创造(software development)中。测试已成为软件开发周期中重要的一环。它不仅应涵盖应用程序组织结构方面的问题也涉及应用程序是否达到预先设计的要求。这些方法经常被称为白盒与黑盒测试。
Boris Beizer，居住于Huntingdon Valley的一名软件测试高手，并且也是多部测试著作（包括黑盒测试、软件测试技巧、 软件系统测试及质量保证）的知名作家，并不喜欢“黑盒”，“白盒”这样的术语，因为听上去太过于单纯，太过于神秘。Beizer更喜欢这样的术语，结构化（structural）和行为（behavioural）测试。
白盒或是结构化测试被设计来测试一段代码如何被构建。
“它往往由知道软件内核的程序员来进行测试，” Beizer说道，所以它才有了玻璃盒或透明盒子测试的别名。
黑盒测试是一种策略。可以把系统看成一个黑盒一样，在测试过程中毋需知道内部代码。而把主要精力放在测试功能需求上。该测试方法也被称为功能测试，不透明测试或闭盒测试。通常由开发人员以外的人来完成。
“一个应用程序也许完工了，且有高度的自动化，在其背后有非常好的理论支持，但是它也许完全与它应有的功能不相关。”想象测试一个文字处理应用程序，但它却被开发成了一个会计包。每个指示都要说明开发人员应该把程序设计成什么样子。程序不能被设计成不需要设计的东西，就像上面提到的文字处理程序那样。这就是行为测试。”
测试策略往往是补充性质的。不可能所有的都做，Beizer说。
“我们所用的方法依赖于我们处在软件开发周期的什么阶段”，他解释道，“早期处理小的组件我们需要开发人员来做白盒或者说结构性测试。但是一旦接近开发周期末尾，我们就更依赖黑盒或者说行为测试”。
做好测试的关键是要知道何时软件测试和质量保证从哲学上不同于传统工业包括制造业。
要知道软件的平衡点---对软件作每个很小的修补不总是能带来效率。像Boeing这样的公司花重金修复每个Bug，但是否值得这样做值得商榷。
七年前，Beizer就是测试WordPerfect的工程师。这也是被他称为当时市场上质量最好声誉最好的软件产品。他用Word Perfect连接上自己兄弟的激光打印机。他发现这种打印机在Epson priteing emulator模式下，简直就变成了一个点阵式打印机了。如此的话，肯定对WordPerfect将来使用者的选择，打了个大大的问号。这就是当年他为WordPerfect打印机驱动程序所奉献的一个Bug！
"当他提交这个WordPerfect问题时，他们在研究了这个bug后说，我可能是世界上仅有的那一个用激光打印机还用点阵模式的。可能世界上不太会有这样的情况，所以他们没有修复它。这是个非常精彩的回答。因为这就是一个优秀企业对想要的bug的哲学。"Beizer说.
居住于Toronto 的Alex Samurin 指出了除了传统的黑白盒测试以外 ，还有一种叫灰盒测试的。它对于网络和互联网应用程序非常重要。灰盒测试策略上部分基于内部程序；测试也可从产品外部来做。但是测试的选择需要知道程序内部组件的知识。按照其说法，灰盒测试是一种非常新的策略。还未被测试界广泛接受， 但在其他不相关领域已有广泛的支持。
从我的观点上来说，白猫黑猫抓住老鼠就是好猫。Samurin如是说。
[ 本帖最后由 B2CPC 于 2005-12-11 14:56 编辑 ] :)well done.
thx very much. Supporting... 非常感谢! Supporting too long very good “According to Samurin, this grey box methodology is fairly new, and is not yet fully embraced by the testing community, but widespread acceptance is in many ways irrelevant. ”－－最后一句有些疑问
按照其说法，灰盒测试是一种非常新的策略。还未被测试界广泛接受， 但在其他不相关领域已有广泛的支持（个人意见：在许多方面来说，普遍的接受灰盒测试还是需要时间的）。 very good Too long to read inconvenient
页:
[1]
