Hi, you mentioned this so I think it really warrants a good thread and discussion to itself:
I have no idea what you are trying to say. The test in question tests the Preferences object. It is not possible to replace it with a stub because then the test becomes meaningless.
Aside from a stub, for some cases, a test mode or other things are very very useful. It's good that you recognize where something itself is being tested that the thing should not be changed.
However, things around it can and should be changed. If we have code that is a monolithic mess, then it will be hard to deal with and maintain. It will also be harder to understand intentions and desired use of the API, etc. I think you hand encountered all of these when you did your cleanup pass on the preferences.
Given the state of things when you started, the current code is pretty nice. However we do have an opportunity to improve it even more and get things quite solid and maintainable. What it sounds like to me is that we have some good opportunity for a better understanding of Test Driven Development (TDD). http://en.wikipedia.org/wiki/Test-driven_development
The main aspects I am thinking of here are in testability and API design. If we have some code that can't be tested, then it is a sign that the code needs to be re-worked. Specifically here it would appear that the code is doing too much "all-in-one" and could benefit from some modular breakup.
In this specific case, the persistence of the preferences can be split up. For some testing of the preferences class I would think that some testing should be on the actual setting and fetching of values. A *separate* set of testing can focus on the actual backing store of things that currently persist to a file. I would rank this as even more appropriate for these classes since the stated design intent is to have the ability to hook to different backing down the road.
So for a quick overview of factors to look a to try to glean an idea of what might be done in this case:
* Look at splitting the backing store from the main class that others use. (a singleton/facade helps us here) * Consider having the unit tests run with the XDG environment set to point to other locations. (less desirable) * Consider a *compile-time* flag to have an explicit file used for the preferences when inside a unit test, instead of the fallback defaults * Some other unit tests might benefit from a preferences instance that either breaks in determined ways or reads from some canned setup.
Splitting the backing store might be a bit more work than we need now. The XDG environment variables *could* be used to affect testing, but that's probably not as stable of an approach. I think compile-time flags to enable "read-only" and/or "read from a special .xml" might be the most helpful. The "read-only" mode should work for most tests, but reading from an explicit file might allow tests to be run using canned data.
If you have a bit of time to look at this, one of the things I've done in the past on other projects is some simple changes to add a call that will clear out the preference type objects and fill them from an explicit file. This makes it very easy to switch starting state of things on a case-by-case basis even if multiple tests are run inside of a single executable.
Let me know if this has helped you start to get some ideas. We probably should bounce things around a bit in the chat room, then get some details up on the wiki so that others can have guidelines as they work on other code too.
Thanks.