Jon A. Cruz wrote:
Disadvantage 1: only when a new subdirectory is created. Happens so rarely as to not be a strong factor.
Actually, this needs to be done not when a directory is created, but when tests are added to a directory that doesn't have any. Currently there are tests only in 6 directories. The single-test-executable solution just ensures that adding a test to a directory that doesn't have any unit tests yet is equally simple as adding one to a directory that has some - it's much simpler to maintain, and eliminates the need to maintain a list of test log files to be removed during "make distclean". It's also consistent with the Windows build system which uses a single test executable.
Jon A. Cruz wrote:
Disadvantage 2: logs can be directed to specific locations as needed. Not so strong an issue. (mainly the current behavior is due to nobody needing it to be different, as opposed to an inability to be different)
OK, so how to redirect all logs to a single file?
Jon A. Cruz wrote:
Disadvantage 5: This sounds like a problem in the recent introduction of the preferences. Should be made simpler.
A stub implementation of preferences also existed before I did the rewrite, the only difference was that it required less code.
Advantage 1: not really an advantage. Advantage 2: this number is so off at the moment as to not be a factor. Advantage 3: strongly possible.
The risk of conflicts can be mitigated if tests are kept contained in their respective objects, e.g. any objects the test needs are members of the test suite object, rather than global.
Jon A. Cruz wrote:
4: Since all tests are in .h files, they do not need to be explicitly added to the Makefile_instert, since they could be picked up due to placement in the resulting .cpp
I don't really understand what do you mean by this - the tests have to be listed in Makefiles regardless of the approach, because they have to be passed as arguments to cxxtestgen.pl. Wildcarding cannot be used because Automake doesn't support wildcards or pattern rules.
Jon A. Cruz wrote:
5: an individual developer is likelier to only be working in a subset of the code. When doing so, they can work within a directory subtree and have test results for just that directory subtree.
Your response to disadvantage 3 indicates that running a subset of the tests won't gain us much time, and running the whole test suite ensures that no unforeseen side effects were introduced. Also, you can't run "make check" in a subdirectory, you'd have to run the tests manually after building them.
My original disadvantage 3 doesn't really hold because on multi-core machines it may be faster to compile the tests in parallel (make -j). Either way, the differences would be minimal.
Regards, Krzysztof Kosiński