Steve Swanson is very new to testing. I predict he has a great future. He has already noticed that the common idea of boundary testing is almost content-free. Michael Bolton and I do a whole session on how to turn boundary testing into a much more intellectual engaging activity. At the end of his post, he identifies one of the major weaknesses of the classic notion of boundary testing. This confirmed for me that he is a mind to watch.
Steve questions the idea of naming test techniques:
What�s the point of having names for things? To me having a name limits what you see and limits creativity. If you feel that certain things are not to be considered boundary tests, then maybe you won�t do them. Maybe you are pigeon-holing yourself into constraints that do not need to be there. Furthermore it seems that everyone has a different idea of what a �boundary� test is. If that is the case then why even have a name for it?
Dear Steve,
I’ve been studying testing for 19 years, and let me tell you, a lot of things people write about it are fairy tales. This is the first reason why you are confused about what’s in a name: most people (not everyone) are also confused, and thus just copy what they see other people write, without thinking much about it.
To use an example from my own history, I used to talk about the difference between conformance testing and deviance testing. I learned this distinction at Apple Computer. For about five years I talked about them, until I one day I realized that it is an empty and unhelpful distinction. It was not a random realization, but was part of a process of systematically doubting all my terminology and assumptions about testing, in traditional Cartesian fashion. I just couldn’t find a good enough reason to retain the distinction of testing into conformance and deviance.
It looks like you are on a similar path. If you continue on it, the kind of thinking you are doing, will A) lead you to better resources to draw from about testing, and B) allow you to become one the leaders helping us out of this morass, C) ensure that you will be attacked by certain bloggers from Microsoft and elsewhere who just hate people who apply philosophical methods to such an apparently straightforward and automatable task like testing. (Speaking of philosophy of terminology, you may be interested in the Nominalism vs. Realism conversation, or how the pragmatism movement swept aside that whole debate, or how the structuralists and post-modernists study the semiotics of discourse. All these things relate to the issues you raise about terminology.)
I will tell you now what book you need to read that will help you more than any other on this planet: Introduction to General Systems Thinking, by Gerald M. Weinberg. It’s what I consider to be the fundamental textbook of software testing, yet not 1 in 100 testers knows about it.
A quick answer to your issue with names…
Terminology is useful for at least these reasons:
A term can be a generative tool. It can evoke an idea or a thought process that you are interested in. (This is different from using a term to classify or label something, which as you point out limits us without necessarily helping us.) An example of the generative use of terms in this way is the HAZOP process which uses “guidewords” to focus analysis. Even a generative usage is susceptible to bias, which is why I use multiple, diverse, overlapping terms.
A term can serve as a compact way to direct a co-worker. When I manage a test team, I establish terminology so that when I say “do boundary testing” I can expect the tester to know what I’m asking him to do without necessarily explaining every little thing. The term is thus attached to training, dialog, and shared experiences. (This needn’t be very limiting, although we must guard against settling into a rut and having only a narrow interpretation of terms.)
A term can serve as a label indicating the presence of a bigger story under the surface, much like a manilla folder marked “boundary testing” would be expected to hold some papers or other information about boundary testing. The danger, I think you’ve noticed, is that ignorant testers may quite happily pass folders back and forth that are duly labelled but are quite perfectly empty. You have to open the folders on a regular basis by asking “can you describe, specifically, what you did when you did ‘exploratory testing’? Can you show me?”
A term can hypnotize people. (I’m not recommending this; I’m warning you against it). Terminology, especially obscure terminology, is often used in testing to give the appearance of knowledge where there is no knowledge, in hopes that the client will fall asleep in the false assumption that everything is oooookkkkkkaaaayyyyyy. You appear not to be susceptible to such hypnosis. (Adam White has a similar resistance.)
I expect to see more example of skeptical inquiry on you blog, as you wrestle with testing, Steve. I hope you find, as I do, that it’s a rewarding occupation.