Re: [Inkscape-devel] "make check" doesn't run unit tests
On Dec 8, 2008, at 6:14 PM, Krzysztof Kosiński wrote:
Jon A. Cruz wrote:
>
> At the moment there are reasons to keep things separate...
>
> ... however with CxxTest, it's simple to have things individually
> *and* combined. We might have some of that already set up.
>
OK, but what are those reasons to keep CxxTests separate? As I see
it, this
has a number of disadvantages:
1. Lots of error-prone makefile boilerplate needed
2. Test logs are scattered all over the build directory
3. Building and linking the tests takes more time
4. Each test has to specify which libraries it needs
5. We need the test-stubs.cpp file with a stub preferences
implementation
The only advantages I see are:
1. If some test crashes, others can run (but then it's an obvious
error
either in the testand should be corrected right away)
2. The number of failed tests shown after "make check" is slightly
more
meaningful
3. Higher chance of conflicts between test headers (I encountered
only one
and corrected it already)
My single-test implementation uses a variable named
CXXTEST_TESTSUITES, to
which all test headers are appended in Makefile_inserts. The only
test-related thing left in Makefile_inserts is the list of test
headers. I
think it's more convenient and simpler than the exisitng system.
Disadvantage 1: only when a new subdirectory is created. Happens so
rarely as to not be a strong factor.
Disadvantage 2: logs can be directed to specific locations as needed.
Not so strong an issue. (mainly the current behavior is due to nobody
needing it to be different, as opposed to an inability to be different)
Disadvantage 3: on my linux box this time is only 8 seconds for an
average test run of everything, and 33 seconds if I touch every
single test source file. So seems like a non-issue
Disadvantage 5: This sounds like a problem in the recent introduction
of the preferences. Should be made simpler.
Advantage 1: not really an advantage.
Advantage 2: this number is so off at the moment as to not be a factor.
Advantage 3: strongly possible.
Other advantages:
4: Since all tests are in .h files, they do not need to be explicitly
added to the Makefile_instert, since they could be picked up due to
placement in the resulting .cpp
5: an individual developer is likelier to only be working in a subset
of the code. When doing so, they can work within a directory subtree
and have test results for just that directory subtree. (this was one
of the main reasons things were split to begin with)
6: certain tests can be run with dedicated mock object. (This has not
yet been leveraged).
And most importantly, overall the most "error prone" operations are
only done when adding a new test source file (*-test.h). That happens
very rarely, with the addition of tests to an existing test suite
source is far more common.