Bryce Harrington wrote:
...
It's a good idea to also make use of the command line access to verbs, this would indeed allow for some very interesting testing possibilities. The main problem would be how to test it. I would imagine comparing the SVG result to a reference output would be most interesting (after all, just looking good usually isn't good enough when executing verbs). But to compare SVGs it would be necessary to parse them and write comparison routines for different elements, which might be a bit much to do as part of this project.
As a first order, a simple diff | wc -l could be used to estimate the size of change. md5sum of the svg file can be used to test output identity. There's probably also tools for extracting the elements of an XML file, and a simple listing of the SVG elements could be useful for comparisons (e.g., after adding a group, check to see if a new group object is present).
Unfortunately none of those would account for changes like increased/reduced precision or alternative encodings for paths.
Alternatively the resulting bitmaps could be compared, this would be a lot less work (it would be needed for the SVG conformance tests anyway). And it would still capture bugs resulting in visible problems.
The Cairo test suite has a simple bitmap diff tool that could be reused here. There are also other similar tools (I used one once from a video camera motion sensor project I found on SourceForge.)
Thanks for the suggestions.
...
Along these lines of using bitmap diff tools, you could create a suite of regression tests using past problematic .svg's. This could be done by scouring launchpad for Fixed bugs with attachments that are not patches (you'd need to write a spider script to do this, or use a tool like bugbuddy, since the advanced search doesn't allow for this), downloading the svg's and prepending the bug ID to their filenames. ...
That would indeed also be a potentially interesting source of test data. I already looked at launchpad's APIs, unfortunately they're not very advanced (yet), so it would indeed need some tricks. But I will definitely look into it.