Hello inkscapers,
I could contribute to the development of the code in my spare time if the following idea makes sense and is of value. While talking in the #irc dev channel and in the last developer meeting, it turned out that the test suite in testfiles directory is somewhat messy and a refactoring could be desirable for some reasons. Here's my proposal. First of all, tests should be grouped in coherent and standard categories. A standard classification is:
end to end testing ---> integration testing ---> unit testing
This classification, translated into a dir tree would be: ----------------------------------------------------------
testfiles |______end2end_tests | |_______gui_tests | |_______cli_tests | |______integration_tests | |______unit_tests | |______CMakeLists.txt | |______CTestCustom.cmake.in | |______README.md
[SUBDIR] gui_tests: empty for now, just add a TODO/FILL-ME note inside it.
[SUBDIR] cli_tests: *) rendering tests are made with CLI (as the CMakeLists.txt confirms), so they should be moved inside cli_tests. Therefore, move here svg files from "rendering_tests" and move rendering_tests/test.sh as cli_tests/rendering.sh *) leave here the current content of cli_tests *) add an "images" dir containing all the graphic files
[SUBDIR] integration_tests: it will contain cpp or script files in the form: operation1_operation2_..operationN.cpp (or .sh, or .py etc).
[SUBDIR] unit_tests: a software unit can be meant both as: 1) (see testfiles/src/curve-test.cpp) a self-referential "concept" 2) (see testfiles/src/uri-test.cpp) as a class, or even a single function, from the API. The important thing is to separate unit tests from integration tests, which are sequences of different tasks. From what I see, the "testfiles/src" dir currently contains unit tests (if cpp files perform integration tests, they should be moved to the testfiles/integration_tests dir) In addition, do move lpe images here (inside an "images" dir). Add a note on the README that lpe stuff is here (as it partially is, actually).
[FILE] CMakeLists.txt
[FILE] CTestCustom.cmake.in
[FILE] README.md: note: only ONE readme will be used for describing this test suite. It will describe what the tests do and how to launch them separately. It will also contain a very short mention - one or two lines - of the model (e2e tests ---> integration tests ---> unit tests) used.
--------------------------------------------------------------------------
There are two possible concrete values/advantages in this new struct:
1) The reordered code would be much more readable, especially in the perspective of finding and running on the fly a specific test for a specific need or in the perspective of adding new ones, or deleting obsolete ones.
2) Integration tests could significantly speed up bug fixing, other than giving a more robust validation procedure of the released software. Let me show an example. An integration test could consist in the same steps described in a submitted bug:
open a file ---> modify a curve ---> apply a filter ---> export to png
In the integration_tests dir, there could be, together with test files, one (or more than one) file called "integration_test_template.cpp", which is meant as an example, with basic operations done sequentially (open, modify, filter, export etc.). Then, if another example called "integration_test_custom.cpp" is created by the developer (but not added to the source tree and added to .gitignore), it would contain code modeled on the basis of "integration_test_template.cpp" and it would be automatically compiled because as a part of the CMakeLists directives. In this way, we have a ready-to-use environment for testing the API. So, when a nasty bug is described together with its steps, a "integration_test_custom.cpp" file can be created on the fly by using chunks of code taken from "integration_test_template.cpp" in order to replicate the steps.
What do you think? And do you think that the integration_tests template would be really useful? In this last case, could you share an example (or more than one) of a bug that can be managed with "integration_test_custom.cpp" so as to have a useful case study?
If the idea appears good and collects approval, I can start making a fork and a new branch for it on my local clone.