Practical Testing
原文出处：http://www.lenholgate.com/archives/000306.html这是一系列文章，最新的一篇“15 - Testing payback”是11月1日发表的。觉得这一系列文章都很不错。如果有谁有空可以报名翻译一下。
目前为止共15篇章，以后可能陆续会增加，如果有所增加，我会及时添加在本文之后。
One of the common complaints about TDD and unit testing that I see is that the examples used aren't real. People often say that the examples used are toys and that writing such tests adds little or no value. To be honest, I often find myself agreeing with them.
One of the problems of adding unit tests to an existing code-base or driving a new project with TDD is deciding exactly where to spend your testing efforts. This is more of an issue when adding tests to existing code as I personally find that the safety of TDD on new code becomes slightly addictive...
Anyway, in an attempt to show how adding unit tests to existing code can be worthwhile I've decided to write a series of blog entries on Practical Testing of real world code...
The code I'll be using for first examples is a class that is used by the socket server framework; CCallbackTimer. This class provides a very light-weight timer manager that runs on its own thread and is reasonably challenging to test. I have a subtle bug that I need to fix in it and reproducing the conditions that cause the bug are quite difficult; first run your PC for 49.7 days...
The class lives in one of our tools libraries and is coupled to several other classes and one other library. First I'll present the code and the fragment of the library that's required to use it. Then I'll add a test harness for the library and write a simple test for the class. Then I'll finally start working on the bug fix and some tests that prove it's fixed and make sure it stays fixed.
I'll use this entry as an index that I'll update as I post the future entries.
1 - Introduction - where we describe the code that we'll test and explain the problems we're hoping to fix.
2 - The first test - where we create a test harness and a unit test.
3 - Test 2, Enter the mocks - where we write our first real test and struggle to write a reliable tests when time is involved...
4 - Taking control of time - where we tell the object under test where to source its time data from, rather than just allowing it to decide for itself.
5 - Testing shouldn't be this hard - where we make the time source more complicated to enable us to control the multi-threaded nature of the class under test.
6 - Tests refactored - where we fix the bug that we discovered in part 5, clean up the tests and finally write the test that shows up the problems that happen when GetTickCount() rolls over to 0.
7 - Fixing the tick count wrap bug - where we finally put in a fix for the bug that shows up when GetTickCount() rolls over to 0.
8 - Once more, with tests first - where we see what the code might look like if we had developed test first rather than by following the HITIW methodology.
9 - More tests, more development, notice the order? - where we write a couple more tests for the new version of the timer queue and do just enough development to make one of them pass.
10 - Fixing the tick count wrap bug, again - where we fix the tick count bug again, only better.
11 - Moving away from the simplest thing - where we add back some real world functionality so that the timer queue can support more than one timer.
12 - Threading is orthogonal - where we realise that the threaded aspect was orthogonal to the real work and add it back in an optional way.
13 - Missing functionality - integrating the new code into a client of the old code exposes some missing functionality.
14 - Bitten by the handle reuse problem - where we use the tests we've built to support a redesign to remove the handle reuse problem.
15 - Testing payback - where we use the tests we've built to support a redesign to improve the performance of the code. 呵呵，connie辛苦了，发了不少文章啊，我第一个来支持！
我带头翻译下
我把上面的文章翻译了一下，翻译的不好，希望大家不要笑话我啊，多多提意见阿，这可是我的处女作！嘿嘿！ 莹莹又谦虚了，呵呵，加分鼓励一下，希望再接再厉。^_^ 呵呵，一定一定！激动ing!
Practical Testing: 1 - Introduction
I'm writing some blog entries that show how to write tests for non trivial pieces of code. This is part 1.The code that we're going to test is CCallbackTimer. This is a class that lives in our Win32Tools library. It’s used by users of our SocketTools library. The class provides a very light-weight timer manager that gets around the various issues we had with using standard Win32 timers.
The timer is designed to be used in vast quantities. Let’s have a timer, or two, for each connection and let’s have 10,000 connections... Because of this we wanted the timer to be something that we could create lots of without caring.
The threading rules and potential 'weight' of WaitableTimers meant that they weren't of use to us.
The fact that we needed to support NT 4 and Windows 95/98 meant that we couldn't use TimerQueues, which is a pity because they’re otherwise a pretty good match.
The class implements a timer queue. Timers are placed on the queue and the class deals with making sure they're kept in the correct order (soonest first). A thread sleeps for as long as is necessary for the first timer and then wakes up and calls a user supplied callback function. The thread then goes back to sleep until the next timer expires. You can cancel and reset timers before they go off and you can pass user data to the callback function.
The code is less than ideal. It was written during a fairly high pressure phase of a project and that shows. I feel it's a bit too complicated for its own good and there's been a known bug in the code since the start.
The current complexity and the bug is why we're interested in writing tests. We want a test that proves the existence of the bug so that we can fix the bug and make the test pass. We'd like to have additional tests in place before we do that so that we can make sure that we don't break any of the existing functionality when we fix the bug.
The class is reasonably cohesive (it does just one thing) and at first sight doesn't appear to be too highly coupled to other concepts; it only uses basic types and its own handle type in its function signatures. However when we tried to break the code out into the smallest project that would compile we discovered that it uses a surprising number of other classes to get its work done.
The Win32Tools library is generally where we keep code that makes Win32 primitives a little easier to use; we have events, threads, critical sections, a wide/narrow string that is the correct width depending on your UNICODE settings, etc. The callback timer uses a thread object because it runs in its own thread, it uses a critical section to allow correctly syncronised access to its data structures, an auto reset event so that the thread can wait for the next timeout and yet still be controllable if we want to shut down the timer queue. The code that the callback timer object uses directly also uses other code; most things throw one of our standard exception types and most things use _tstring for strings. This isn't too bad really since all of these things live in the same library and the user of CCallbackTimer doesn't really have to understand many of them to use CCallbackTimer.
The class also uses another library; our C++Tools library. This was originally seen as a place to keep stuff that isn't platform specific and yet isn't specific to any particular business need. We found there wasn't that much that fell into this category; or at least we've been less aggressive at refactoring and harvesting this kind of code. The callback timer object uses CNodeList from this library. CNodeList is our home grown invasive doubly linked list. It's used in the callback timer because when we want to cancel or reset a timer we have direct access to the node in the list and thus removals are simple. I'd really like to be rid of this dependency...
So, there we have it. The code that we're intending to test can be downloaded from here. I'm presenting them in 'lowest common denominator' format, i.e. VC 6 projects, since I can't be bothered to maintain 3 project file formats and all of the later tools will load and convert the earlier projects. Load the Win32Tools.dsw workspace and you're away.
Now on to the bug: The callback timer uses GetTickCount() to work out when now is and when its timers need to go off. It uses absolute times in milliseconds from the GetTickCount() time continuum. It does this because when a timer goes off it needs to work out how long it needs to sleep for it and simply say the next timer is due in (absolute time of next timer - now) milliseconds. Unfortunately GetTickCount() wraps from a big number to 0 every 49.7 days and the code we have ignores this fact; assuming that 'later' is always bigger. The code is broken in several places, the breaks are reasonably easy to fix, but proving that the code is broken and writing a test to expose the break is non trivial.
In the next posting we'll write the first test.
接下来，谁来继续。
Practical Testing: 2 - The first test
I'm writing some blog entries that show how to write tests for non trivial pieces of code. This is part 2.In part 1 I introduced the code that we're going to write a test for. Now we'll move towards the all important first test. The first test is, in my opinion, vitally important; the first test proves that you can build a test harness that can create the object. This sounds easy, but try picking a class from your current project at random and writing a test that constructs an instance of that class. The test should link in as little additional code as possible and should result in an instance that is fully usable; so if you use some freaky two phase construction method, like having to call a function to Initialise() the object after creating it then you need to do that as well...
Writing the first test exposes all of the explicit and implicit coupling that tethers your object to the rest of your code base. If you're trying to test existing code that was never designed with testing in mind then it's quite often the case that you need to link in some global objects, do some seemingly unrelated initialisation, or link with every library in your project... When you have the first test running you know that you CAN test; everything from there on is just typing ;)
So, we'll add a test harness for the Win32Tools library. This is a console mode exe project that contains the test code and a simple driver that runs the tests. I don't use C++Unit; convince me why I need it... I expect that NUnit and JUnit are more useful as they can use reflection to create the scaffolding. It's my opinion that you should put the least things between you and getting to your first test. If you feel the need to refactor it all later to use a wiz-bang framework then fine, but get started first...
The test project is simple, it includes Test.cpp, the test driver, and CallbackTimerTest.cpp and .h, the actual unit test. The driver calls TestAll() on the CCallbackTimerTest object and catches a range of exceptions. If the test doesn’t throw an exception then all is well and we've passed. It's simple and I've been meaning to do it all in a cleverer way, but I've just not found the need. For me, if I have 10 tests then they should all pass. If 1 fails then the test harness is broken. I don't need to see that 8 out of 10 pass, if the first one fails, that's it, that's where I need to work next...
The test harness exe links with the library under test. It also links with any libraries that the library under test depends on, like C++Tools. It also links with a library of test harness helper code, TestTools.
Getting to the first test means we need to pull in a little more of the real implementation of Win32Tools. We need SEHException code so that the test harness can report on any structured exceptions that the tests throw.
Looking at the test for CCallbackTimer you'll see that it's very simple. We log that the test has started, we create an instance of the object and we log that the test has finished. Hardly rocket science and probably something that would earn the scorn of those who despise unit tests but: We now have a framework for writing more tests. We could add a unit test for any of the other classes in Win32Tools with ease. We can add a new test for CCallbackTimer just by adding a new function to the unit test... We've proved that we can create the class in relative isolation. We've exposed any hoops we might need to jump through, or classes we might need to create before we can create our object under test.
The first test is important, sometimes I stop here. Often it's not worthwhile to add more tests right away, but it is worthwhile to have a test that proves that your object isn't becoming implicitly or explicitly coupled to other code. If you suddenly find a bug in a class that has a test harness you're more likely to write a test to help you fix the problem, if you have to create a new project, build a test harness for the object and then work out what else you need to link to get the exe to run then you're more likely to just tinker with the problem and try and debug it in place within the application. In choosing where to spend your test energy you get a lot of bang for the buck by writing the first test for an object...
Code is here. Win32Tools is the workspace that you want and Win32ToolsTest is the project that you should set as active.
Practical Testing: 3 - Test 2, Enter The Mocks
I'm writing some blog entries that show how to write tests for non trivial pieces of code. This is part 3.Last time we wrote the first test. It doesn't test much, but it proves we can test. Now we'll write a real test for real functionality and along the way we'll start to deal with some of the issues that come up when you're trying to test multi-threaded code.
To test the timer functionality we need to write a test that sets a timer and then waits for it to go off. This first test should be able to verify that the timer was triggered and that the callback was executed. The function we will be testing is SetTimer() this has the following signature:
void SetTimer(
const Handle &hnd,
DWORD millisecondTimeout,
DWORD userData = 0);
The Handle type is what the CCallbackTimer uses to identify an instance of a timer. There are two constructors for a Handle but we'll ignore the second one for now as that's part of the unfortunate complexity that we mentioned earlier. The constructor we're interested in for the purposes of this test is this one:
explicit Handle(Callback &callback);
So, to call SetTimer() we need to create a Handle and to do that we need to create a Callback... The Callback is just an interface that the timer queue uses to alert us to the fact that our timer has expired. The interface looks like this:
class Callback
{
public :
virtual void OnTimer(
Handle &hnd,
DWORD userData) = 0;
virtual ~Callback() {}
};
To be able to call SetTimer() we will need something that implements the Callback interface; that something will be a mock object with the sole purpose of helping us to write tests.
As I've said before, I like to keep my mock objects in a library project that's in a subdirectory of the project that the mocks provide functionality for. So, if the interface lives in Win32Tools the mock for that interface lives in Win32ToolsMock... This is a useful way to structure the code because you will quite likely need to use the Win32ToolsMocks when writing tests for libraries that depend on interfaces from the Win32Tools library. When I started with unit testing I kept my mocks in the test harness project but this made them hard to reuse in other test harnesses. The new way is better.
Our first mock only has to implement a single function, OnTimer(), but what does it do once the function is called? Most of my mocks inherit from a base class from the TestTools library; CTestLog. The test log allows derived classes to log messages to it as they are used. At various points during a test we can query the mock object's logs to make sure that the correct functions have been called in the correct order and with the correct arguments.
Our implementation of the mock's OnTimer() could be as simple as this:
void CLoggingCallbackTimerHandleCallback::OnTimer(
CCallbackTimer::Handle &hnd,
DWORD userData)
{
LogMessage(_T("OnTimer: ") + ToString(userData));
}
But to be able to test reliably we need more control. Suppose our test looks something like this:
void CCallbackTimerTest::TestTimer()
{
const _tstring functionName = _T("CCallbackTimerTest::TestTimer");
Output(functionName + _T(" - start"));
CCallbackTimer timer;
CLoggingCallbackTimerHandleCallback callback;
CCallbackTimer::Handle handle(callback);
timer.SetTimer(handle, 100, 1);
// what to do here?
callback.CheckResult(_T("|OnTimer: 1|"));
Output(functionName + _T(" - stop"));
}
We set a timer using our mock object for the callback, we then do something and then check the test log contains the expected result. The question is, what should we do in between. We need to wait for the timer to expire; we could stick a Sleep() in there, but that adds an element of uncertainty to our test. For the delay to be effective in all situations it needs to be long enough to always be long enough, no matter what machine we're running the test on or what the CPU loading is at the time. A test that only works sometimes is worse than no test at all. If developers start wasting time debugging bugs in the unit tests then they'll soon lose interest in testing.
Unfortunately, since we're dealing with time, we can't solve this problem completely; at least not this time. We can make sure that the test executes as fast as possible though, and still keep the delay long enough to make the test reliable. The trick is to add some code to the mock so that we can wait for the timer to go off. A manual reset event will do. If we change OnTimer() to this:
void CLoggingCallbackTimerHandleCallback::OnTimer(
CCallbackTimer::Handle &hnd,
DWORD userData)
{
LogMessage(_T("OnTimer: ") + ToString(userData));
m_event.Set();
}
Then we can add a function to the mock so that we can wait for the event to be set:
bool CLoggingCallbackTimerHandleCallback::WaitForTimer(
DWORD timeoutMillis)
{
return m_event.Wait(timeoutMillis);
}
Our event class returns true if the event is signaled within the time limit and false if it isn't. Since the wait will end as soon as the event is signaled and the event is signaled when the timer callback is called we can delay the test until the timer expires and provide a timeout that is reliable but that doesn't slow down the test run.
The test code becomes this:
timer.SetTimer(handle, 100, 1);
THROW_ON_FAILURE(functionName, true == callback.WaitForTimer(200));
callback.CheckResult(_T("|OnTimer: 1|"));
So now we have a test that's reasonably reliable. We could increase the delay to something larger and improve the reliability at the expense of slowing the reporting of failure situations. Unfortunately we can't make the test any better than this until we remove our reliance on time; and that will have to wait until the next posting...
Code is here. Same rules as before.
Practical Testing: 4 - Taking control of time
I'm writing some blog entries that show how to write tests for non trivial pieces of code. This is part 4.We have a test for SetTimer() but it's not as robust as we'd like. The problem is that the class under test is runs its own thread which reacts based on time and this makes our testing harder and less predictable. What's more it actually makes testing for our known bug practically impossible; to test for our bug we'd have to have a test which called GetTickCount() to determine the current value and which then slept so that it could execute the test at the point when the counter rolled over to 0. That would mean that, on a bad day, the test could take 49.7 days to run...
I've written about the problems in testing time related code before. The solution we need now is the same one that I recommended then. We need to apply a level of indirection. Rather than having the object go off to a well known place to get an indication of the current time we need it to go to a place that we've provided. Once we are in control of where the object goes to get the resource it needs we can substitute the resource with one that we can manipulate in the ways that we require for testing. So, in this case, rather than embedding calls to GetTickCount() directly into the code we should pass the object an interface that allows it to obtain the time values that it needs. This is classic parameterize from above; the object doesn't decide, its creator does. If we create an interface like this:
class IProvideTickCount
{
public :
virtual DWORD GetTickCount() const = 0;
protected :
~IProvideTickCount() {}
};
We can pass something that implements it to the timer queue in its constructor and it can call through the interface when it requires the current tick count. Once this indirection is in place we can pass in a mock object for testing and that mock object can define time however we decide is appropriate for the test in hand.
Unfortunately parameterize from above tends to make all of the object wiring explicit and, generally, somewhat more ugly and complicated. In this particular situation, where there's really only one 'real' implementation of the interface and the parameterization is purely for testing I find that it's often useful to make the parameterization optional. Rather than having a single constructor which takes an IProvideTickCount implementation we can have two, one that does and one that doesn't. The object can provide the default, real, implementation when none is supplied and when we need to test we can plug in our own version. Of course, should the code in question turn out to be a performance hot spot we could go one further and #define the indirection just for testing (but I prefer to let profiling lead the way in this kind of optimisation).
So, our object under test ends up looking a bit like this:
static const CTickCountProvider s_tickProvider;
CCallbackTimer::CCallbackTimer(
const IProvideTickCount &tickProvider)
:m_shutdown(false),
m_tickProvider(tickProvider)
{
Start();
}
CCallbackTimer::CCallbackTimer()
:m_shutdown(false),
m_tickProvider(s_tickProvider)
{
Start();
}
We implement a mock tick count provider for testing and needn't worry about the exposed wiring when using the code for real.
The next issue is how to implement the mock implementation of the interface; something like this springs to mind:
class CMockTickCountProvider :
public IProvideTickCount,
public JetByteTools::Test::CTestLog
{
public :
CMockTickCountProvider();
void SetTickCount(
const DWORD tickCount);
// Implement IProvideTickCount
virtual DWORD GetTickCount() const;
private :
volatile DWORD m_tickCount;
// No copies do not implement
CMockTickCountProvider(const CMockTickCountProvider &rhs);
CMockTickCountProvider &operator=(const CMockTickCountProvider &rhs);
};
It simply returns the tick count that we tell it to. Our test can then become this:
void CCallbackTimerTest::TestTimer()
{
const _tstring functionName = _T("CCallbackTimerTest::TestTimer");
Output(functionName + _T(" - start"));
CMockTickCountProvider tickProvider;
CCallbackTimer timer(tickProvider);
CLoggingCallbackTimerHandleCallback callback;
CCallbackTimer::Handle handle(callback);
tickProvider.SetTickCount(1000);
timer.SetTimer(handle, 100, 1);
// Prove that time is standing still
THROW_ON_FAILURE(functionName, false == callback.WaitForTimer(1000));
callback.CheckResult(_T("|"));
tickProvider.SetTickCount(1100);
THROW_ON_FAILURE(functionName, true == callback.WaitForTimer(100));
callback.CheckResult(_T("|OnTimer: 1|"));
Output(functionName + _T(" - stop"));
}
And at last we have control of time. Our test for tick count rollover is now within our grasp. We could set our tick count to just before the rollover, set a timer that expires after the rollover and, well, observe and then fix the problems.
This particular approach can be applied to all services that an object uses and it tends to result in a flexible, decoupled and far more granular design for code. Once your services are provided via interfaces you can mock them up for testing or demo purposes. When working with services that provide changing data; such as live, "ticking", financial market data, it's so much easier to prove that the code works as expected if you can mock up a data source and have that source provide just the data you require as and when you require it. Gathering the data to supply is made easy due to the same indirection that makes the mock provider possible; rather than mocking up a provider, you simply instrument a real service provider and have that save down live data that you can then edit and use as test data. In summary; cool. ;)
The CCallbackTimer can now be controlled completely by the test; well, almost... Since we're unit testing we know about how the object is implemented; these are white box tests. Looking at that implementation, or the results of the test log from the mock time source, will show us that our current time source causes the object to spin in a busy loop, waiting around 100 ms per loop. Time is standing still, but our object expects it to be moving forward at its usual rate. We have control, but not quite enough; we can't determine how many times our mock is accessed, we just know that when we change the time to after the timeout then the timer will go off. I think it would be better to have a little more control...
Code is here. Same rules as before.
[ Last edited by connie on 2005-11-10 at 10:05 ]
Practical Testing: 5 - Testing shouldn't be this hard
I'm writing some blog entries that show how to write tests for non trivial pieces of code. This is part 5; the one where we find a bug we weren't expecting...Last time I slipped an interface between our object under test and its source of time. This allowed us to provide a mock time source that we could control from within our test. This moved us nearer to being able to write the test we need to prove that the code doesn't currently handle GetTickCount() wrapping to 0 after 49.7 days of machine up-time. I pointed out that the test was still not quite deterministic enough for me as the CCallbackTimer object's timeout management thread was spinning in a way that we couldn't control and so we couldn't easily know reliably exactly what the test log from the mock time provider would contain.
We can fix this problem quite easily by allowing the mock time provider to provide time a certain number of times and then block. This will allow us to completely control the way the timeout management thread runs: We can set the time provider to supply time for 2 calls and then block and then wait for the time provider to process both calls and we then know that the timeout management thread is blocked within our time provider and that the objects are in a certain state. Often you don't need this level of control within a test but it's useful to know how to achieve it for those situations when you do.
The mock time provider can be called from 2 threads. The thread on which it was created; such as during a call to SetTimer() on the CCallbackTimer and the thread that the timer queue runs internally to manage timeouts. It's only the internal thread that we want to control so we'll take a note of the thread that the mock time provider is created on and allow all calls from that thread to work all of the time.
Next we'll add an events, so that we can wait for the timer to be in a particular state, and a counter so that we can allow just a certain number of calls. For the counter we use a simple helper class, CWaitableCounter which implements a counter that manages a couple of events so that you can wait for the counter to reach zero, or wait for it to move away from zero. We also use a critical section because, even though we know that only one thread will be using the code path that we're protecting in this test we may want to use the mock time provider in situations where multiple threads access it and it's easier to add a critical section now than waste time trying to work out why it doesn't work at a later date.
GetTickCount() now looks like this:
DWORD CMockTickCountProvider::GetTickCount() const
{
if (m_mainThreadId != ::GetCurrentThreadId())
{
CCriticalSection::Owner lock(m_criticalSection);
if (0 == m_counter.GetValue())
{
m_blockedCallEvent.Set();
}
m_counter.WaitForNonZero();
m_blockedCallEvent.Reset();
LogMessage(_T("GetTickCount: Another Thread: ") + ToString(m_tickCount));
m_counter.Decrement();
}
else
{
LogMessage(_T("GetTickCount: Main Thread: ") + ToString(m_tickCount));
}
return m_tickCount;
}
It's fairly complicated but that's because we're trying to test a piece of code that is fairly hard to test and we want a lot of control over the test environment... The function works like this; for calls that originate on a thread that isn't the thread that the object was created on, if the counter is zero it sets and event to let us know that the call is about to block and then waits on the counter and, most likely, blocks. As soon as the wait returns (meaning the counter is not zero) it resets the 'blocked' event. The blocked event is used so that the test can wait for a call to block within the time provider. We use this to synchronise the test with the state of the timer queue thread. Next we update the test log and finally we decrement the counter and return the value.
This mock time provider allows us to selectively allow calls to occur, to wait for those calls to complete and to wait for a call to block. With this level of control we can write some tests that are more reliable; our first test now looks like this:
void CCallbackTimerTest::TestTimer()
{
//1
const _tstring functionName = _T("CCallbackTimerTest::TestTimer");
Output(functionName + _T(" - start"));
CMockTickCountProvider tickProvider;
tickProvider.SetTickCount(1000);
CCallbackTimer timer(tickProvider);
CMockTickCountProvider::AutoRelease releaser(tickProvider);
THROW_ON_FAILURE(functionName, true == tickProvider.AllowCalls(1, s_delay));
tickProvider.CheckResult(_T("|GetTickCount: Another Thread: 1000|"));
//2
CLoggingCallbackTimerHandleCallback callback;
CCallbackTimer::Handle handle(callback);
timer.SetTimer(handle, 100, 1);
tickProvider.CheckResult(_T("|GetTickCount: Main Thread: 1000|"));
THROW_ON_FAILURE(functionName, true == tickProvider.AllowCalls(1, s_delay));
tickProvider.CheckResult(_T("|GetTickCount: Another Thread: 1000|"));
THROW_ON_FAILURE(functionName, true == tickProvider.WaitForBlockedCall(s_delay));
//3
// Prove that time is standing still
THROW_ON_FAILURE(functionName, false == callback.WaitForTimer(0));
callback.CheckResult(_T("|"));
tickProvider.SetTickCount(1100);
THROW_ON_FAILURE(functionName, true == tickProvider.AllowCalls(1, s_delay));
tickProvider.CheckResult(_T("|GetTickCount: Another Thread: 1100|"));
THROW_ON_FAILURE(functionName, true == callback.WaitForTimer(s_delay));
callback.CheckResult(_T("|OnTimer: 1|"));
Output(functionName + _T(" - stop"));
}
The test does three things; the code between //1 and //2 sets up the environment for testing and gets the timer queue thread to a well known state (waiting for an infinite time or until the timer queue state changes). Between //2 and //3 we set a timer and allow the timer queue thread to spin once and then block. The timer queue thread would have retrieved a value for 'now' of 1000, compared this to the first timeout to occur of 1100 and slept for 100ms before checking the time again, and blocking. After //3 we set 'now' to 1100 so the timer will go off, allow the timer queue thread to run again so that it picks up the new time and signals the timer. The timer queue thread then goes back into an infinite wait and the test completes.
Note that we've added a helper class CMockTickCountProvider::AutoRelease to unblock the timer queue thread in the event of a test failure exception. Since the CCallbackTimer object is destroyed when an exception occurs and the thread inside it may be blocked the destruction will block as the object can't shut its worker thread down. The AutoRelease object will get destroyed before the timer queue and its destructor sets the time provider to allow lots of calls; thus unblocking the timer queue.
Unfortunately we still rely on a hard coded timeout within the tests; s_delay is set to a value at the top of the test harness and used when we're expecting an event to be set within a short period of time. If the tests are working correctly then s_delay could be set to INFINITE, however this would mean that test failure would cause the test to hang rather than actually fail. By setting s_delay to a smaller value we can force the test to fail by throwing an exception rather than hanging; this is better if the test is part of an automated build, but introduces a level of uncertainty. I find that setting it to 1000 seems to work fine for me whilst I'm developing tests, and 60000 removes any chance of false failure during automated test runs. Setting the value to INFINITE is useful if you want to debug into the test code and you don't want your timeouts to go off and tear the test harness down around you.
The code above is ripe for refactoring, but we'll leave that for later. First we'll write some more tests in case the repeated structure that's obvious in this test needs to change for other tests... Then we'll refactor.
So, we have a test for a single timer, we should add tests for multiple timers and for cancelling timers before, and after, they expire. We can then add our test for the tick count wrap and any others that we think about.
The test for multiple timers is relatively easy to write. We can set several timers to go off at different times and then run the timer queue manually as we have done for the single timer and check that the timers go off at the right times and in the right order. Something like this:
void CCallbackTimerTest::TestMultipleTimers()
{
const _tstring functionName = _T("CCallbackTimerTest::TestMultipleTimers");
Output(functionName + _T(" - start"));
CMockTickCountProvider tickProvider;
tickProvider.SetTickCount(1000);
CCallbackTimer timer(tickProvider);
CMockTickCountProvider::AutoRelease releaser(tickProvider);
THROW_ON_FAILURE(functionName, true == tickProvider.AllowCalls(1, s_delay));
tickProvider.CheckResult(_T("|GetTickCount: Another Thread: 1000|"));
CLoggingCallbackTimerHandleCallback callback1;
CCallbackTimer::Handle handle1(callback1);
timer.SetTimer(handle1, 100, 1);
tickProvider.CheckResult(_T("|GetTickCount: Main Thread: 1000|"));
THROW_ON_FAILURE(functionName, true == tickProvider.AllowCalls(1, s_delay));
tickProvider.CheckResult(_T("|GetTickCount: Another Thread: 1000|"));
THROW_ON_FAILURE(functionName, true == tickProvider.WaitForBlockedCall(s_delay));
///
CLoggingCallbackTimerHandleCallback callback2;
CCallbackTimer::Handle handle2(callback2);
timer.SetTimer(handle2, 200, 2);
tickProvider.CheckResult(_T("|GetTickCount: Main Thread: 1000|"));
THROW_ON_FAILURE(functionName, true == tickProvider.AllowCalls(1, s_delay));
tickProvider.CheckResult(_T("|GetTickCount: Another Thread: 1000|"));
THROW_ON_FAILURE(functionName, true == tickProvider.WaitForBlockedCall(s_delay));
///
CLoggingCallbackTimerHandleCallback callback3;
CCallbackTimer::Handle handle3(callback3);
timer.SetTimer(handle3, 150, 3);
tickProvider.CheckResult(_T("|GetTickCount: Main Thread: 1000|"));
THROW_ON_FAILURE(functionName, true == tickProvider.AllowCalls(1, s_delay));
tickProvider.CheckResult(_T("|GetTickCount: Another Thread: 1000|"));
THROW_ON_FAILURE(functionName, true == tickProvider.WaitForBlockedCall(s_delay));
CLoggingCallbackTimerHandleCallback callback4;
CCallbackTimer::Handle handle4(callback4);
timer.SetTimer(handle4, 150, 4);
tickProvider.CheckResult(_T("|GetTickCount: Main Thread: 1000|"));
THROW_ON_FAILURE(functionName, true == tickProvider.AllowCalls(1, s_delay));
tickProvider.CheckResult(_T("|GetTickCount: Another Thread: 1000|"));
THROW_ON_FAILURE(functionName, true == tickProvider.WaitForBlockedCall(s_delay));
// Prove that time is standing still
THROW_ON_FAILURE(functionName, false == callback1.WaitForTimer(0));
THROW_ON_FAILURE(functionName, false == callback2.WaitForTimer(0));
THROW_ON_FAILURE(functionName, false == callback3.WaitForTimer(0));
THROW_ON_FAILURE(functionName, false == callback4.WaitForTimer(0));
callback1.CheckResult(_T("|"));
callback2.CheckResult(_T("|"));
callback3.CheckResult(_T("|"));
callback4.CheckResult(_T("|"));
tickProvider.SetTickCount(1100);
THROW_ON_FAILURE(functionName, true == tickProvider.AllowCalls(1, s_delay));
tickProvider.CheckResult(_T("|GetTickCount: Another Thread: 1100|"));
THROW_ON_FAILURE(functionName, true == tickProvider.WaitForBlockedCall(s_delay));
THROW_ON_FAILURE(functionName, true== callback1.WaitForTimer(s_delay));
THROW_ON_FAILURE(functionName, false == callback2.WaitForTimer(0));
THROW_ON_FAILURE(functionName, false == callback3.WaitForTimer(0));
THROW_ON_FAILURE(functionName, false == callback4.WaitForTimer(0));
callback1.CheckResult(_T("|OnTimer: 1|"));
callback2.CheckResult(_T("|"));
callback3.CheckResult(_T("|"));
callback4.CheckResult(_T("|"));
tickProvider.SetTickCount(1160);
THROW_ON_FAILURE(functionName, true == tickProvider.AllowCalls(1, s_delay));
tickProvider.CheckResult(_T("|GetTickCount: Another Thread: 1160|"));
THROW_ON_FAILURE(functionName, true == tickProvider.WaitForBlockedCall(s_delay));
THROW_ON_FAILURE(functionName, false == callback1.WaitForTimer(0));
THROW_ON_FAILURE(functionName, false == callback2.WaitForTimer(0));
THROW_ON_FAILURE(functionName, true== callback3.WaitForTimer(s_delay));
THROW_ON_FAILURE(functionName, true== callback4.WaitForTimer(s_delay));
callback1.CheckResult(_T("|"));
callback2.CheckResult(_T("|"));
callback3.CheckResult(_T("|OnTimer: 3|"));
callback4.CheckResult(_T("|OnTimer: 4|"));
tickProvider.SetTickCount(1201);
THROW_ON_FAILURE(functionName, true == tickProvider.AllowCalls(1, s_delay));
tickProvider.CheckResult(_T("|GetTickCount: Another Thread: 1201|"));
// No more timers pending so the thread goes into an infinite wait until new timers are added.
THROW_ON_FAILURE(functionName, false == callback1.WaitForTimer(0));
THROW_ON_FAILURE(functionName, true== callback2.WaitForTimer(s_delay));
THROW_ON_FAILURE(functionName, false == callback3.WaitForTimer(0));
THROW_ON_FAILURE(functionName, false == callback4.WaitForTimer(0));
callback1.CheckResult(_T("|"));
callback2.CheckResult(_T("|OnTimer: 2|"));
callback3.CheckResult(_T("|"));
callback4.CheckResult(_T("|"));
Output(functionName + _T(" - stop"));
}
Unfortunately running this test locates a bug in the code under test. The timer queue isn't being maintained in lowest to highest order. Timers 3 and 4 are being inserted in the queue after timer 2 rather than before... This bug probably doesn't show up in our current usage of the timer queue as all the timers tend to be set to very low values, but it's definitely a bug. Looking at the code in question the debug Output statements give away the fact that I obviously had problems getting things to work here... Looks like I stopped before I actually fixed the problem :(
Anyway, in the best tradition of TDD, we have a failing test, so it's OK to stop for a break... Code is here. Same rules as before.
Practical Testing.doc
包含目前已经写了的15篇文章。前面贴的几篇给大家看看是否对你有帮助，如果有帮助，可以从这里下载附件。文章的相应链接在本贴的第一篇已经详细列出。 先感谢莹莹的劳动,能留下你的QQ或者MSN吗? 想翻译后面,又觉得文章太常了。
[ Last edited by jody on 2005-11-10 at 16:55 ] 嘿嘿，多谢jody，我也是刚翻译，大家一起努力！
MSN：roselover_wy@hotmail.com
我已经着手翻译 Introduction了，所以请接下面翻译！嘿嘿！ 我耐心等待 要不是英语水平不够 我也上了 嘿嘿，实在不好意思，最近非常忙，所以翻译可能要延后了，真是对不起大家阿！
页:
[1]
