On 13-11-2014 1:21, Martin Owens wrote:
Johan E,
I've looked at the inkscape-rendertest branch.
There is a pdiff codebase and a few other bits, but I notice no use of Pillow(was PIL) to process the images. We have access to PIL.ImageChomp to select the difference rectangles and further processing (with some good guides online) for pulling out more information if we need to.
http://stackoverflow.com/questions/1927660/compare-two-images-the-python-lin...
If we went with pdiff instead of Chomp and CMP in python, was there a reason?
The original python script I believe could do a binary compare, or a perceptual compare (using perceptualdiff). No need to use perceptualdiff specifically. We do need a fuzzy-ness threshold, otherwise the compares will fail too often. Imagemagick seems to provide cool stuff there too...
You should be aware of the different types of compares we should do: - compare with pass-reference, if available (pass) - compare with fail-references, if avail (fail) - compare with patched svg file (that should render to what is intended) (pass)
Desirables for the tests: - XML output, parsable by xUnit Jenkins plugin. GoogleTest output is fine, CTest output, many other formats all using XML. - Timeout: if Inkscape takes too long, fail the test - Performance: add execution time to output (like GoogleTest does) - If ok: don't output too much - If not ok, different from pass reference: output difference picture, indicating the differences - If not ok, same as fail-ref: say that output is equal to fail ref
I can think of many other nice-to-haves, but let's get this working first. Most important thing is to get the tests running on the Jenkins server. It is easiest if the python script can be called, something like:
runtests.py --inkscape=/bla/bla/inkscape --XML=rendertest-results.xml (--options=optionsfile-that-is-stored-in-bzr?)
Thanks so much,
- Johan