Git submodule policy / address sanitizer
Dear Inkscape Team,
since gitlib is up and running, the question arizes if we want to use submodules, and if so which folder structure should we use.
I want to add address / memory sanitizer support and use https://github.com/arsenm/sanitizers-cmake. It is just a few files and I could copy them verbatim, but it would make sense to add it as a submodule.
In case you don't know what address/memory sanitizer is: it creates instrumented builds (about 2x slower) which check for all sorts of invalid memory usage, like access to freed memory or reading uninitialized variables. The cmake scripts above enable this by a cmake define, similar to switching between debug and release builds. In my experience it saves a lot of time in tracking down obscure bugs in complex C++ code. I right now have a debug/release deviation I want to track down.
Best regards,
Michael
Thanks Michael, deviation between debug/release is boring! Best!
On Sat, 2017-06-10 at 15:28 +0200, Michael Soegtrop via Inkscape-devel wrote:
Dear Inkscape Team,
since gitlib is up and running, the question arizes if we want to use submodules, and if so which folder structure should we use.
I want to add address / memory sanitizer support and use https://github.com/arsenm/sanitizers-cmake. It is just a few files and I could copy them verbatim, but it would make sense to add it as a submodule.
In case you don't know what address/memory sanitizer is: it creates instrumented builds (about 2x slower) which check for all sorts of invalid memory usage, like access to freed memory or reading uninitialized variables. The cmake scripts above enable this by a cmake define, similar to switching between debug and release builds. In my experience it saves a lot of time in tracking down obscure bugs in complex C++ code. I right now have a debug/release deviation I want to track down.
Best regards,
Michael
Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ Inkscape-devel mailing list Inkscape-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/inkscape-devel
My experience is that git submodules are really cumbersome to work with. Duplication of code is not nice but if sanitizers-cmake is no available as a system package maybe it is just easier to copy the files to Inkscape repository.
2017-06-10 16:45 GMT+02:00 Jabier Arraiza <jabier.arraiza@...2893...>:
Thanks Michael, deviation between debug/release is boring! Best!
On Sat, 2017-06-10 at 15:28 +0200, Michael Soegtrop via Inkscape-devel wrote:
Dear Inkscape Team,
since gitlib is up and running, the question arizes if we want to use submodules, and if so which folder structure should we use.
I want to add address / memory sanitizer support and use https://github.com/arsenm/sanitizers-cmake. It is just a few files and I could copy them verbatim, but it would make sense to add it as a submodule.
In case you don't know what address/memory sanitizer is: it creates instrumented builds (about 2x slower) which check for all sorts of invalid memory usage, like access to freed memory or reading uninitialized variables. The cmake scripts above enable this by a cmake define, similar to switching between debug and release builds. In my experience it saves a lot of time in tracking down obscure bugs in complex C++ code. I right now have a debug/release deviation I want to track down.
Best regards,
Michael
Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ Inkscape-devel mailing list Inkscape-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/inkscape-devel
Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ Inkscape-devel mailing list Inkscape-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/inkscape-devel
Christoffer;
What is bad with git submodules in your experience?
Mvh
/Olof ----------------- Är du systemutvecklare? Spana in https://cilamp.se
On 10 June 2017 at 20:12, Christoffer Holmstedt <christoffer.holmstedt@...400...> wrote:
My experience is that git submodules are really cumbersome to work with. Duplication of code is not nice but if sanitizers-cmake is no available as a system package maybe it is just easier to copy the files to Inkscape repository.
2017-06-10 16:45 GMT+02:00 Jabier Arraiza <jabier.arraiza@...2893...>:
Thanks Michael, deviation between debug/release is boring! Best!
On Sat, 2017-06-10 at 15:28 +0200, Michael Soegtrop via Inkscape-devel wrote:
Dear Inkscape Team,
since gitlib is up and running, the question arizes if we want to use submodules, and if so which folder structure should we use.
I want to add address / memory sanitizer support and use https://github.com/arsenm/sanitizers-cmake. It is just a few files and I could copy them verbatim, but it would make sense to add it as a submodule.
In case you don't know what address/memory sanitizer is: it creates instrumented builds (about 2x slower) which check for all sorts of invalid memory usage, like access to freed memory or reading uninitialized variables. The cmake scripts above enable this by a cmake define, similar to switching between debug and release builds. In my experience it saves a lot of time in tracking down obscure bugs in complex C++ code. I right now have a debug/release deviation I want to track down.
Best regards,
Michael
Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ Inkscape-devel mailing list Inkscape-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/inkscape-devel
Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ Inkscape-devel mailing list Inkscape-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/inkscape-devel
-- Christoffer Holmstedt
Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ Inkscape-devel mailing list Inkscape-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/inkscape-devel
Just a reminder, address sanitizer can miss overflows if a read/write ends up pointing to another valid memory location. A more fool proof tool is pointer bounds checking. It's in GCC but requires a Skylake processor or later.
On Sat, Jun 10, 2017 at 2:01 PM, Olof Bjarnason <olof.bjarnason@...400...> wrote:
Christoffer;
What is bad with git submodules in your experience?
Mvh
/Olof
Är du systemutvecklare? Spana in https://cilamp.se
On 10 June 2017 at 20:12, Christoffer Holmstedt <christoffer.holmstedt@...400...> wrote:
My experience is that git submodules are really cumbersome to work with. Duplication of code is not nice but if sanitizers-cmake is no available as a system package maybe it is just easier to copy the files to Inkscape repository.
2017-06-10 16:45 GMT+02:00 Jabier Arraiza <jabier.arraiza@...2893...>:
Thanks Michael, deviation between debug/release is boring! Best!
On Sat, 2017-06-10 at 15:28 +0200, Michael Soegtrop via Inkscape-devel wrote:
Dear Inkscape Team,
since gitlib is up and running, the question arizes if we want to use submodules, and if so which folder structure should we use.
I want to add address / memory sanitizer support and use https://github.com/arsenm/sanitizers-cmake. It is just a few files and I could copy them verbatim, but it would make sense to add it as a submodule.
In case you don't know what address/memory sanitizer is: it creates instrumented builds (about 2x slower) which check for all sorts of invalid memory usage, like access to freed memory or reading uninitialized variables. The cmake scripts above enable this by a cmake define, similar to switching between debug and release builds. In my experience it saves a lot of time in tracking down obscure bugs in complex C++ code. I right now have a debug/release deviation I want to track down.
Best regards,
Michael
Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ Inkscape-devel mailing list Inkscape-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/inkscape-devel
Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ Inkscape-devel mailing list Inkscape-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/inkscape-devel
-- Christoffer Holmstedt
Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ Inkscape-devel mailing list Inkscape-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/inkscape-devel
Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ Inkscape-devel mailing list Inkscape-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/inkscape-devel
On Sat, Jun 10, 2017 at 03:28:02PM +0200, Michael Soegtrop via Inkscape-devel wrote:
Dear Inkscape Team,
since gitlib is up and running, the question arizes if we want to use submodules, and if so which folder structure should we use.
I want to add address / memory sanitizer support and use https://github.com/arsenm/sanitizers-cmake. It is just a few files and I could copy them verbatim, but it would make sense to add it as a submodule.
IMHO, in the interest of keeping things simple while people are climbing learning curves for git, let's hold off on remotes and submodules for now.
Also, in general, I know we've been apt to pull stuff into the main codebase but going forward as much as possible let's try to keep external dependencies completely external, with detection logic in cmake. The more we can slim down the codebase the easier it'll be for new folks to learn and less work it'll be for busy old timers to maintain over the long run.
Regarding testing tools, improving code quality for Inkscape is extremely valuable work, so the more the merrier. We just need to be cognizant of setting them up as default off, with the appropriate cmake stuff to detect the test dependency and give the user a config option to switch them on.
In case you don't know what address/memory sanitizer is: it creates instrumented builds (about 2x slower) which check for all sorts of invalid memory usage, like access to freed memory or reading uninitialized variables. The cmake scripts above enable this by a cmake define, similar to switching between debug and release builds. In my experience it saves a lot of time in tracking down obscure bugs in complex C++ code. I right now have a debug/release deviation I want to track down.
We had set up Coverity at one point, which also checked similar stuff. I think that's probably defunct currently, though. We didn't really have enough people following up on problems for it to work.
The tool almost doesn't matter - the real value is the people who work on following up on issues the testing finds. The first run can typically identify a LOT of work items that'll take a lot of labor to work through; ongoing runs can be less work but still need folks to follow up on stuff, and unfortunately a lot of regular developers will be too busy with other stuff to bother so it helps to have dedicated people familiar with the testing tool.
So, as a rule of thumb, you'd want to see at least, say, 3 people committing to watch and follow up on issues found by a particular tool, if adoption of the tool is going to be invasive in terms of infrastructure requirements or adjustments to development processes.
Bryce
Dear Bryce,
Also, in general, I know we've been apt to pull stuff into the main codebase but going forward as much as possible let's try to keep external dependencies completely external, with detection logic in cmake. The more we can slim down the codebase the easier it'll be for new folks to learn and less work it'll be for busy old timers to maintain over the long run.
The piece I suggested is exactly that: cmake code which detects and implements sanitizer support. But it is just 10 or so cmake files and can also be copied.
So, as a rule of thumb, you'd want to see at least, say, 3 people committing to watch and follow up on issues found by a particular tool, if adoption of the tool is going to be invasive in terms of infrastructure requirements or adjustments to development processes.
I think the sanitizers are different. It is more a tool to help developers debug their code in case they see strange behavior like Debug/Release mode deviations.
Best regards,
Michael
On Sun, Jun 11, 2017 at 11:51:31PM +0200, Michael Soegtrop wrote:
Dear Bryce,
Also, in general, I know we've been apt to pull stuff into the main codebase but going forward as much as possible let's try to keep external dependencies completely external, with detection logic in cmake. The more we can slim down the codebase the easier it'll be for new folks to learn and less work it'll be for busy old timers to maintain over the long run.
The piece I suggested is exactly that: cmake code which detects and implements sanitizer support. But it is just 10 or so cmake files and can also be copied.
If it is entirely inside the cmake directories, then that does remove most of the worry, although 10 files sounds like a lot of cmake...
So, as a rule of thumb, you'd want to see at least, say, 3 people committing to watch and follow up on issues found by a particular tool, if adoption of the tool is going to be invasive in terms of infrastructure requirements or adjustments to development processes.
I think the sanitizers are different. It is more a tool to help developers debug their code in case they see strange behavior like Debug/Release mode deviations.
They're not different, same concept applies - make sure other people besides yourself will be committing to use it before you add it to the repo.
I don't mean to pick on this tool in particular, there's just been a bit of a history of tests, build systems, frameworks, and so on being championed by one person but never really catching on beyond them, so that when their attentions moved on the bits were left in place to atrophy.
Bryce
On 06/11/2017 04:51 PM, Michael Soegtrop via Inkscape-devel wrote:
Dear Bryce,
Also, in general, I know we've been apt to pull stuff into the main codebase but going forward as much as possible let's try to keep external dependencies completely external, with detection logic in cmake. The more we can slim down the codebase the easier it'll be for new folks to learn and less work it'll be for busy old timers to maintain over the long run.
The piece I suggested is exactly that: cmake code which detects and implements sanitizer support. But it is just 10 or so cmake files and can also be copied.
Another solution here would be to have the CI job that did the sanitizer check download the CMake files on the fly from another source. This way we wouldn't have to have them in our repo at all, just when the job is run. We should save them as an artifact of that job for debugging though.
Ted
participants (7)
-
Bryce Harrington -
Christoffer Holmstedt -
Jabier Arraiza -
Michael Soegtrop -
Olof Bjarnason -
Ted Gould -
Yale Zhang