Risk-based continuous integration: testing as project risk
链接请见：http://www.testingreflections.com/node/view/3680
Risk-based continuous integration: testing as project risk.
Submitted by Ainars Galvans on Tue, 09/05/2006 - 09:42. Continuous Integration | test management
I’m going to invent a new term “Risk-based continuous integration” am I? I believe that practice is not so widely used, or at least not formalized. I hope to hear from You if anyone openly practice something similar. Any comments, argues and even attacks are also welcome. I plan to blog technical details of the practice, but hope to receive some feedback to understand level of detailed I should provide. Meanwhile I will try to describe the term in general.
You know: risk-based testing; testing as project risk mitigation investment; ROI from testing – I’m afraid those topics indirectly contributed to the wrong perception of testing role on the project. Quality itself – the back hole on the project – frequently underestimated, seems to become the project risk. I believe that continuous integration, if managed appropriate, is the right mitigation for the phenomena I would like to call “testing as project risk”. What I mean by “managed appropriate” – it is what the topic is about.
What’s so different
Continuous integration is “process that completely rebuilds and tests an application frequently”. It does not matter how does we prioritize feature adding to the code. What I’ve seen so far (referred to Agile methods) is that features are added according to business priority. Simplified description would be the following: we start new project, collect requirements (user stories) and prioritize them 1,2,3,4... then we add them one-by-one until we run out of the money to add the next one. Well - the priorities may be changed as well as new items add on-the-fly, but the idea is that next feature to add is determined as the one with the biggest business priority. There is a lot of sense behind this approach, but I’m going to suggest a different one. So this whole blog is about custom prioritization within continuous integration.
Testing as project risk
I’m switching to a different story now. I have analyzed what Project manager (PM) thinks about testing in my company (I plan to publish details soon). The main conclusion is the following: they see testing as risk for the project to delay. They have no idea how many defects we are going to report and how much time it will take to fix even the most significant of them (not to mention that they don’t commit to fix everyone of them). More over, they don’t know about it even when testing is said to be “started”. This is especially true because of the continuous integration – they are adding new features until the last minute and adding bugs along the way. Even if we/they write comprehensive automated tests which ensures any regression defect is fixed at the same time as it is added (during build), they are still adding defects with the new code which they add. And it may and typically happen that the first added features are the most straight-forward and so have less defects. And testers are reporting more and more defects as project is progressing. More over testers (and also customers) are learning more and more and find more and more issues in the code that was OK yesterday, but incomplete/ insufficient/ inappropriate/ wrong today. We could call that adding more features or fixing defects, either way if we look at initial feature list – implementation progress is slowing down. Unhappy customer, finger-pointing and all the issues that Agile methods intended to solve.
Testing-risk-based development
Practice I want to share is simple. Just like we base test planning on the risks identified by developers, they – developers should base implementation based on risks identified by testers. The features that testers feel to cause most defects and/or require more time/resources for testing should be implemented first. Actually developers could in most cases evaluate the risk of causing defect even better than tester. I’ve heard developer saying – this feature is tricky one, there are so many possible cases I can’t test them all. I know you – testers will succeed at finding the most critical cases and will find the most critical defect. It would be logical to start with implementing such a features. Even if the developer has the power to decide on implementation order I know a few of them who would start with the trickiest ones.
Balancing with business priority
So if we are implementing only those most tricky features, it could appear we never implement those – most significant to the customer. There should be some balance introduced. It means that through the project we change prioritization approach – we start with most tricky and end up with most significant (to the customer), what we do in the middle should be managed carefully.
Issue – business want us to use different order
It could appear that business user will be against such an approach, as we are implementing less significant features before implementing the most significant ones. I has no this issue with my practice as we use iterative development and customers don’t see the next iteration until it get some stability. I would like to believe that it is possible to convince customers who are used to Agile methodologies that the balanced prioritization approach are the best also for them: start with features which they don’t feel certain about not with the ones that they know how exactly it should work.
页:
[1]
