Exploratory Testing After 23 Years

skinapi

链接请见：
http://www.logigear.com/newslett ... _after_23_years.asp

Exploratory Testing After 23 Years
Cem Kaner, Professor, Florida Institute of Technology
Twenty-three years ago, I coined the phrase "exploratory testing" (ET) I didn't invent the practice but (as far as I can tell) I was the first public advocate of it.

As ET has become fashionable, a host of consultants and academics have started writing and speaking about it. Some of them have helped move the concepts of exploration and testing forward in interesting ways. My upcoming talk to the Conference of the Association for Software Testing asks what we know now: What is exploratory testing and how do we do it? This note is a preview (a teaser, really) for that talk.

James Bach pointed out the essential characteristic of exploratory testing-the explorer is cognitively present. She actively, purposefully, curiously investigates the software under test, always with the responsibility of deciding, minute by minute, what is the most promising path to whatever she has chosen to investigate. There are no artificial limits on exploration. The tester is free to use whatever sources of information are available, including specifications, technical support records, competitors' implementations of comparable software, and (of course) experiments (tests) that reveal information empirically. There are no limits on test techniques that explorers can use-for example, any degree of automation is fine. However, the explorer doesn't simply rerun old tests and trust that they'll reveal whatever is interesting. She tests to learn. She will probably scrutinize the behavior of the program as it is tested, looking for new ideas about how it might fail, how it might be further tested or measured, and how useful these tests are at this point in development. Test execution might be automated-thinking is not.

The antithesis of exploration is scripted testing, in which the tester (or a machine) follows a set of procedures laid out long ago, comparing the observed behavior with whatever results the test designer considered relevant or interesting back then. The cognition happened back then, not now. For a strong recent example of advocacy of an anti-exploratory (fully scripted) approach to system testing, read Lisa Crispin's Testing Extreme Programming.

The scope of exploration is the same as the scope of testing itself.

The old definition of testing-execution of a program with the intent of exposing coding errors-is far too narrow. I used to push this definition too, but my work went beyond it and so did the work of most other skilled testers. When the definition doesn't describe the practice, I think we should value the practice and fix the definition. Here is what I work from now.

Software testing is a technical investigation conducted to provide stakeholders with empirical information about the quality of a software product, system or service.

The investigation includes research necessary to design information-revealing tests and interpret their results, building tools and data sets to make those tests possible, and communication necessary to report results to stakeholders in an effective way. Like any skilled tester, the explorer does all of these. The difference is that the explorer does them in any sequence that is useful, mixing research, design, execution, interpretation, and communication so as to keep exposing new information and keep up with the ongoing changes in market, platform, design, and implementation of the software under test.

What are the skills of the explorer? And how do these, and the mind set that applies them, differ from the traditional approach? Come to the conference and we'll explore these questions together. The way we have designed CAST, keynotes start the discussion rather than containing it.

About the author: Cem Kaner, J.D., Ph.D., is a Professor of Software Engineering at the Florida Institute of Technology and Director of the Center for Software Testing Education & Research.

brilliantking

Exploratory Testing After 23 Years
探索性测试，23年后的今天

Cem Kaner

Twenty-three years ago, I coined the phrase "exploratory testing" (ET) I didn't invent the practice but (as far as I can tell) I was the first public advocate of it.
二十三年前，我造出了“探索性测试”（ET）这个词。我并没用将其用于实践，不过，尽我所言，我是公开传播探索性测试思想的第一人。

As ET has become fashionable, a host of consultants and academics have started writing and speaking about it. Some of them have helped move the concepts of exploration and testing forward in interesting ways. My upcoming talk to the Conference of the Association for Software Testing asks what we know now: What is exploratory testing and how do we do it? This note is a preview (a teaser, really) for that talk.
随着ET的流行，大批的技术顾问和学者开始撰写和宣讲与之相关的话题了。他们中的一些人用绘志绘色的方式将探索与测试的概念向前发展了。我即将在软件测试协会会议上做的一个报告要问到我们现在所知道的：什么是探索性测试，我们如何进行这种测试？这篇笔记是对该报告的一个前瞻（实际上是一个引子）。

James Bach pointed out the essential characteristic of exploratory testing-the explorer is cognitively present. She actively, purposefully, curiously investigates the software under test, always with the responsibility of deciding, minute by minute, what is the most promising path to whatever she has chosen to investigate. There are no artificial limits on exploration. The tester is free to use whatever sources of information are available, including specifications, technical support records, competitors' implementations of comparable software, and (of course) experiments (tests) that reveal information empirically. There are no limits on test techniques that explorers can use-for example, any degree of automation is fine. However, the explorer doesn't simply rerun old tests and trust that they'll reveal whatever is interesting. She tests to learn. She will probably scrutinize the behavior of the program as it is tested, looking for new ideas about how it might fail, how it might be further tested or measured, and how useful these tests are at this point in development. Test execution might be automated-thinking is not.
James Bach 指出了探索性测试的核心特征：测试者要有灵感。她主动地，目的明确地，用好奇的眼光考察着所测试的软件，带有责任感地，时刻决定着对她所选择的要测试的东西，什么是最佳的途径。在探索性测试中没有人为的限制。测试者可以自由地使用任何可以用的资料，包括规格说明，技术支持记录文档，竞争对手的同类软件，而且当然还有传达测试信息的测试经验。测试者所用的测试技术也是没有限制的，比如，把测试工作进行任何程度的自动化都是可以的。尽管如此，测试者决不能简单地重复运行老的测试用例并且认定用这个用例可以找到所关心的东西。测试者测试的目的是用习。很有可能她要细心地观察软件在测试时的行为，对于软件如何运行失败，如何进一步测试或衡量，以及现在所有的测试用例如何有助于开发，她都要寻找新的思路。应该自动地执行测试用例，光想是没有用的

The antithesis of exploration is scripted testing, in which the tester (or a machine) follows a set of procedures laid out long ago, comparing the observed behavior with whatever results the test designer considered relevant or interesting back then. The cognition happened back then, not now. For a strong recent example of advocacy of an anti-exploratory (fully scripted) approach to system testing, read Lisa Crispin's Testing Extreme Programming.
对探索性测试相对的是脚本式测试。在脚本式测试中，测试者（或机器）遵循以前所制定出来的一系统的步骤进行测试，将所观察到的结果与测试用例的设计者期望的相关的结果进行对比。之后，就会有某种意识产生而非现在。最近有一种宣传，针对系统测试的与探索性测试相对（完全脚本化）的测试方法，可以阅读　Lisa Crispin 的《极限编程测试》。

The scope of exploration is the same as the scope of testing itself.
探索性测试的应用范围与测试本身是相同的。

The old definition of testing-execution of a program with the intent of exposing coding errors-is far too narrow. I used to push this definition too, but my work went beyond it and so did the work of most other skilled testers. When the definition doesn't describe the practice, I think we should value the practice and fix the definition. Here is what I work from now.
关于软件测试的旧的定义——对一个程序执行测试旨在发现代码中的缺陷——现在看来太狭义了。我过去也常常推崇这个定义，但现在我所做的已经远远抛下这个定义了，而且大多数的其他娴熟的测试者的工作也和我一样。当理论无法与实践相符时，我想我们应该认识到实践的价值并且修改理论。下面就是我从现在所要做的工作。

Software testing is a technical investigation conducted to provide stakeholders with empirical information about the quality of a software product, system or service.
软件测试是一门对软件进行检视的技术，用来根据测试实验向相关人员提供关于软件产品，系统或服务的消息。

The investigation includes research necessary to design information-revealing tests and interpret their results, building tools and data sets to make those tests possible, and communication necessary to report results to stakeholders in an effective way. Like any skilled tester, the explorer does all of these. The difference is that the explorer does them in any sequence that is useful, mixing research, design, execution, interpretation, and communication so as to keep exposing new information and keep up with the ongoing changes in market, platform, design, and implementation of the software under test.
这种检视包括为设计揭示软件信息的测试用例，解释这些用例的测试结果，构建测试工具和数据集以便让测试得以实施所必需的对软件本身的研究，以及有效地向相关人员报告测试结果所必需的交流技巧。

What are the skills of the explorer? And how do these, and the mind set that applies them, differ from the traditional approach? Come to the conference and we'll explore these questions together. The way we have designed CAST, keynotes start the discussion rather than containing it.
进行探索性测试的人员需要什么技能呢？如何运行这些技能？应用这些技能的思路与传统的测试思路有怎样的区别？请来参加这次会议，我们会一道讨论这些问题。

About the author: Cem Kaner, J.D., Ph.D., is a Professor of Software Engineering at the Florida Institute of Technology and Director of the Center for Software Testing Education & Research.
关于作者： Cem Kaner 佛罗里达理工学院教授，软件测试教学与研究中心主任。

[ 本帖最后由 brilliantking 于 2006-5-8 17:45 编辑 ]

skinapi

1、For a strong recent example of advocacy of an anti-exploratory (fully scripted) approach to system testing, read Lisa Crispin's Testing Extreme Programming.
这一句翻译应该没什么问题，需要说明的是，根据我的个人理解，这里提到的scripted testing并不一定是指一定写脚本，强调的是执行前执行的步骤以及数据都是准备好的，也就是我们通常说的测试用例已经设计好了的。
2、The way we have designed CAST, keynotes start the discussion rather than containing it.
这里提到的CAST是指一个测试方面的国际会议，意思是说CAST的活动形式是通过一些提纲挈领的东西引起大家的热烈讨论，而不是上面的人讲下面的人听。

ddhengbb

DX谢谢了,不过我们现在毕业设计需要三篇2,3千字的相关外文翻译,正在迷惑中.
DX能否给提供一下.多谢了.

brilliantking

原帖由 ddhengbb 于 2006-5-10 08:53 发表
DX谢谢了,不过我们现在毕业设计需要三篇2,3千字的相关外文翻译,正在迷惑中.
DX能否给提供一下.多谢了.

前面的贴子里有，仔细找找，呵呵。

brilliantking

原帖由 skinapi 于 2006-5-9 17:50 发表
1、For a strong recent example of advocacy of an anti-exploratory (fully scripted) approach to system testing, read Lisa Crispin's Testing Extreme Programming.
这一句翻译应该没什么问题，需要说明的 ...

谢谢斑竹悉心指正！对第二句，我虽然知道是什么意思，但就是找不到合适的词来表达 containing it，所以就没去翻译那句了。第一句中的 fully scripted 我确实是望文生意了，实践经验太少啊sdlkfj7

李逍遥

ding !