Bizarro Makefile_insert usage
What is the purpose of using Makefile_insert to pretend that everything is in the src directory instead of using the SUBDIRS directive?
I'm intending to change the Automake build system to link the executables from single object files instead of static libraries so that we may finally put files in the proper directories (e.g. remove widgets/ and dialogs/ and move those to ui/widget and ui/dialog/, as well as join helper/, util/ and svg/), and I would like to know whether I should also sanitize general Makefile usage while I'm at it.
Regards, Krzysztof Kosiński
On Feb 20, 2009, at 7:56 AM, Krzysztof Kosiński wrote:
What is the purpose of using Makefile_insert to pretend that everything is in the src directory instead of using the SUBDIRS directive?
I'm intending to change the Automake build system to link the executables from single object files instead of static libraries so that we may finally put files in the proper directories (e.g. remove widgets/ and dialogs/ and move those to ui/widget and ui/dialog/, as well as join helper/, util/ and svg/), and I would like to know whether I should also sanitize general Makefile usage while I'm at it.
It's not "Bizarro", but rather rational.
Track down the paper "Recursive Make Considered Harmful" by Peter Miller.
The key point is to avoid recursive make. Not only does it slow builds down, but it often causes incorrect building and linking.
Also see if you can check history of things. widgets/ and dialogs/ vs. ui/widgets and ui/dialogs comes from early attempts to recreate things all-at-once instead of a more gradual C++ replacement path. For the most part the outdated files can be removed as long as they can be found to not be actively used.
Jon A. Cruz wrote:
Track down the paper "Recursive Make Considered Harmful" by Peter Miller. The key point is to avoid recursive make. Not only does it slow builds down, but it often causes incorrect building and linking.
I skimmed over that paper, and while incomplete depndencies don't happen in Automake, the point about parallel builds are valid, so I'll leave the current non-recursive setup alone.
For the most part the outdated files can be removed as long as they can be found to not be actively used.
The main problem here is that some files have to be in dialogs/ and some in ui/dialog/ because the static libs build from those libraries are in different places in the link order. Moving files around without fixing the Makefiles first will result in link failures. What I want to do is to link the "inkscape" executable from .o files, instead of having an .a file for each directory.
Regards, Krzysztof Kosiński
On Fri, Feb 20, 2009 at 07:56:59AM -0800, Krzysztof Kosiński wrote:
I'm intending to change the Automake build system to link the executables from single object files instead of static libraries
I believe this will slow down the link time. Linking is already often the longest part of compilation for developers (such as when making changes to a single .cpp file).
(I haven't actually checked whether it noticeably slows down the final link -- I'm still compiling now, but I'm about to go out, so thought I'd send this before quantifying. The reason I think it will slow down linking is that I believe that creating a .a file involves creating a symbol index to speed up symbol lookups; though maybe this only helps people who do `make' rather than `make inkscape'; or maybe the difference isn't particularly significant.)
so that we may finally put files in the proper directories (e.g. remove widgets/ and dialogs/ and move those to ui/widget and ui/dialog/, as well as join helper/, util/ and svg/),
I'll note that svn doesn't handle renames very well, e.g. if any developers have any uncommitted changes to a file that gets renamed, or if other developers change files that you've renamed locally but haven't committed yet.
That's not to say that it's never worth reorganizing, just that you might be careful about doing it.
Technically, library membership and directory membership are orthogonal when using non-recursive make. Of course there are human reasons why we might want to arrange directories according to library makeup, but it does mean that you can at least experiment with different library organization without doing renames.
pjrm.
Peter Moulder wrote:
I believe this will slow down the link time. Linking is already often the longest part of compilation for developers (such as when making changes to a single .cpp file).
The link time did increase a few seconds (I ahven't done any quantative tests), but the benefit of not having to care about the link order is much more important. Now files can be placed in the directories they belong to, instead of the directories they had o be in in order not to break the linking. This was the case with e.g. calligraphic-profile-rename.cpp, which had to be in dialogs/ instead of ui/dialog/ because it was used in toolbox.cpp. There are more examples. One idea we could try to reduce the link time is to create a large static library from all the .o files.
I'll note that svn doesn't handle renames very well, e.g. if any developers have any uncommitted changes to a file that gets renamed, or if other developers change files that you've renamed locally but haven't committed yet.
OK, I'll keep that in mind. The files I'm moving right now are rather stable, so I hope there won't be any conflicts.
Regards, Krzysztof Kosiński
On Feb 20, 2009, at 4:27 PM, Krzysztof Kosiński wrote:
The link time did increase a few seconds (I ahven't done any quantative tests), but the benefit of not having to care about the link order is much more important. Now files can be placed in the directories they belong to, instead of the directories they had o be in in order not to break the linking. This was the case with e.g. calligraphic-profile- rename.cpp, which had to be in dialogs/ instead of ui/dialog/ because it was used in toolbox.cpp. There are more examples. One idea we could try to reduce the link time is to create a large static library from all the .o files.
On thing I'd strongly suggest is to look at how to layer things.
That is, from natural use there could be a .a structure that comes out from the nature of the classes themselves.
Certain items might be appropriate for separation, such as croco, libnr, etc.
The layering of GTK+ might be another simple inspiration: http://www.gtk.org/documentation.html * GLib * GObject * Pango * ATK * GdkPixbuf * GDK * GTK
On Fri, Feb 20, 2009 at 04:27:49PM -0800, Krzysztof Kosiński wrote:
The link time did increase a few seconds (I ahven't done any quantative tests), but the benefit of not having to care about the link order is much more important. Now files can be placed in the directories they belong to, instead of the directories they had o be in in order not to break the linking.
...
One idea we could try to reduce the link time is to create a large static library from all the .o files.
I don't think that that would work, as it's much the same as creating a large binary from all the .o files. More precisely, it will still help the case of building multiple binaries (such as the default targets of inkscape and inkview), but I expect that compiling just inkscape after a single .cpp file change will be slower with this approach than with the existing library arrangement.
If the existing library assignment is a working one, then another idea (or at least starting point) would be to try moving files but keep the existing library assignment: i.e. keep existing foo_libfoo_a_SOURCES lines but just adjust leading directories of individual .cpp files as appropriate.
pjrm.
Peter Moulder wrote:
If the existing library assignment is a working one, then another idea (or at least starting point) would be to try moving files but keep the existing library assignment: i.e. keep existing foo_libfoo_a_SOURCES lines but just adjust leading directories of individual .cpp files as appropriate.
That would be a management nightmare. Currently there is no good cure for the long link time. The real solution would be to make parts of Inkscape less dependent on each other, so that the static libraries built from directories would really represent something, instead of being just an arbitrary grouping of files. This process is ongoing, but it's a slow one (it's labelled as "inkcore" on the roadmap).
Regards, Krzysztof Kosiński
participants (3)
-
Jon A. Cruz -
Krzysztof Kosiński -
Peter Moulder