On 29/04/06, Jon A. Cruz <jon@...18...> wrote:
On Apr 28, 2006, at 2:53 PM, Ben Fowler wrote:
[ snip ]
Actually... rather than "test infected" we really want to be getting focused on "test driven development"
However, to point out things, you've hit some mix of features and uses of unit tests.
Suppose I say that I agree with you 100% and agree to drop the term 'test-infected' as a human readable alias for 'Test Driven Development' ...
Basically, a "unit test" tests some unit of software, usually a module of some sort. It will run through API's and such to ensure that base functionality is as expected.
Then we have "integration test", where pieces are put together and system is checked for overall operation.
"Regression tests" are those that are done to make sure that things that were once fixed stay fixed, and don't "regress" to be broken again. Often unit tests can be run as part of regression testing.
... I would have to argue that unit tests should have been completed long before getting to 'packaging', and arguably integration tests as well. (I will leave regression testing on one side as I personally think that regression tests need to be more rigorous and should be fully automated, independent of individual developers; and I wouldn't mind if they took hours or days to complete, so I would expect that regression testing is prescribed as mandatory only for release candidates perhaps also being done irregularly during development).
Once one has unit tests in, maximizing their benefit is good. Running all unit tests before checkin is a very good thing. Running them each and every developer build is even better.
I may be mixing two independent points here, but I would explain this in terms such as "Never require your folk to choose between the easy way and the right way".
Urging people to do unit tests may even be counter productive, but showing how to do them and stating that they are done personally before checking in could be a much better way of achieving this very good thing. Care should be taken, however, that we do not break the Inkscape model of "patch first and question later", and also (in my opinion) that we do not dissuade people who wish to contribute to the code, but whose drawing skills are better than their C++; until we have really good extensions and plug-ins systems - a discipline in a mediaeval sense - then people with both drawing and coding skills are very valuable, indeed there should be a Welcome Mat or Red Carpet out for them.
Perhaps the all target should be configured so that a plain make as in:
$ make -sC obj_dir
should imply an invocation of 'make -sC obj_dir check', and if a dev wanted to avoid the 'check' step then he or she would have to specifically ask for 'make -sC obj_dir inkscape inkview'
Test Driven Development goes on to say that you shouldn't bother writing anything that you don't code the unit tests for first. If you can't create a unit test for some function, then you either don't need that function, or don't understand what you're doing.
I would claim to be test-inf.., ahem, fully in favour of Test Driven Development, though probably not an Xtremist, and would merely add that a set of unit tests (of the type we are discussing) focussed on coverage would be very useful indeed to anyone who is considering making changes to part of the code that he or she does not fully understand, which is probably all of us.
(My original point about smoke testing was intended to speak to 'acceptance testing' - the dev certifying that the build was good enough to be worth a tester spending time on - but your points are somewhat more general in that they cover what we do every day).
I respectfully submit these two small patches to the unit test system that make it more effective when building outside the source tree (object dir build):
Index: src/Makefile_insert =================================================================== --- src/Makefile_insert (revision 11475) +++ src/Makefile_insert (working copy) @@ -338,8 +338,8 @@ echo '#define INKSCAPE_VERSION "$(VERSION)"' > inkscape_version.h
test_all_includes = \ - attributes-test.h \ - color-profile-test.h \ + $(srcdir)/attributes-test.h \ + $(srcdir)/color-profile-test.h dir-util-test.h \ extract-uri-test.h \ verbs-test.h Index: src/Makefile.am =================================================================== --- src/Makefile.am (revision 11475) +++ src/Makefile.am (working copy) @@ -232,7 +232,7 @@ $(svg_test_svg_includes) \ $(xml_test_xml_includes) \ $(test_all_includes) - $(top_srcdir)/cxxtest/cxxtestgen.pl --template=selfname.tpl -root -o test-all.cpp \ + $(top_srcdir)/cxxtest/cxxtestgen.pl --template=$(srcdir)/selfname.tpl -root -o test-all.cpp \ $(libnr_test_nr_includes) \ $(svg_test_svg_includes) \ $(xml_test_xml_includes) \
It is possible that there are other patches needed - I haven't been able to establish whether the other lines in 'test_all_includes' need the $(srcdir) pre-pended.
Ben