On Fri, Mar 21, 2014 at 07:19:44PM +0000, Isobel Knowles wrote:

On Thu, 20 Mar 2014 13:21:16 -0700 mathog <mathog@...1176...> wrote:

...

Lately I have noticed that when I come back to Inkscape on another day, modify a single cpp (today's was drawing-text.cpp) and "make" it starts rebuilding all sorts of things that would seem to be utterly unrelated. In fact, the build is so extensive it may well be a complete rebuild. Here is just a fraction of the things it compiled:

... CXX sp-mesh-gradient.o CXX sp-mesh-patch.o CXX sp-mesh-row.o CXX sp-object.o CXX sp-path.o CXX sp-pattern.o CXX sp-rect.o CXX sp-shape.o CXX sp-stop.o CXX sp-style-elem.o CXX sp-text.o CXX sp-tref.o CXX sp-tspan.o CXX sp-use.o CXX style.o ...

If the same file is later modified again, "make" the next time does the expected thing and just compiles that one file and then links everything.

Any idea why it is doing that? It almost seems like the Make files now have some concept of "old" object files and anything over a day old gets rebuilt.

Trunk 13168, Ubuntu 12.04.4 LTS

Configure was run with

export CPPFLAGS="-DWITH_SVG2" ./configure --disable-strict-build

Dear mathog,

I think this is an automake oddity (I remember this happening some years back), but to rule out other matters can you run this script instead of executing make (you can supply arguments as before):

I notice we have cmake as a secondary build system in the tree. Perhaps after 0.91 is done and out the door we should consider migrating fully over to cmake. I've had very good experience with it in other projects and it seems to handle dependencies a lot more sensibly IMHO.

I don't know what implication cmake has for windows or osx, but have a recollection that cmake works at least as well on those platforms as autoconf, and perhaps better?

Bryce

On Sat, Mar 22, 2014 at 05:52:06AM +0100, Krzysztof Kosiński wrote:

2014-03-22 5:40 GMT+01:00 Bryce Harrington <bryce@...961...>:

...

I notice we have cmake as a secondary build system in the tree. Perhaps after 0.91 is done and out the door we should consider migrating fully over to cmake. I've had very good experience with it in other projects and it seems to handle dependencies a lot more sensibly IMHO.

The CMake build system we have in the tree is incomplete, and its scripting language is such a joke that in my experience using CMake is only a modest improvement over Autotools.

I concur that it's not functional as implemented currently, but calling it "a joke" is merely being rude. I'm all in favor of considering other options but not if we're going to do it merely by slinging mud. Change is important, but we must remain civil and stay on a technical level, so that we can reach a consensus decision that everyone as a group can get behind.

I will point out the cmake scripts appear to be doing a lot more than they need to be doing. It likely could be slimmed down quite a bit.

Bryce

On Fri, 21 Mar 2014 22:15:23 -0700 Bryce Harrington <bryce@...961...> wrote:

On Sat, Mar 22, 2014 at 05:52:06AM +0100, Krzysztof Kosiński wrote:

...

2014-03-22 5:40 GMT+01:00 Bryce Harrington <bryce@...961...>:

...

I notice we have cmake as a secondary build system in the tree. Perhaps after 0.91 is done and out the door we should consider migrating fully over to cmake. I've had very good experience with it in other projects and it seems to handle dependencies a lot more sensibly IMHO.

The CMake build system we have in the tree is incomplete, and its scripting language is such a joke that in my experience using CMake is only a modest improvement over Autotools.

I concur that it's not functional as implemented currently, but calling it "a joke" is merely being rude. I'm all in favor of considering other options but not if we're going to do it merely by slinging mud. Change is important, but we must remain civil and stay on a technical level, so that we can reach a consensus decision that everyone as a group can get behind.

I will point out the cmake scripts appear to be doing a lot more than they need to be doing. It likely could be slimmed down quite a bit.

Dear mathog,

This is somewhat faster for me (tested on a 1.7GHz Celeron):

#!/bin/sh -e rm -f basics filetimes* deliberations* ( pwd ; mount ; make --version ; ntpq -c peers ) > basics find . -type f -printf '%i %TY-%Tm-%TdT%TH:%TM:%TS %p\n' | \ sort -k2 | xz -c > filetimes.xz make -p --debug=basic,verbose $@ | xz -c > deliberations.xz

or you could pop the deliberations into /tmp (assuming that is tmpfs) and compress with xz -e them only before emailing, or you could examine the deliberations by hand. All I'm really looking for is what make thinks it should be doing vs. the facts of the case. If it is an automake issue it will come down to automake injecting spurious dependencies into the Makefile. Often these are not at all obvious, hence the need to examine it's deliberations.

On the other hand the use of dot whatnot directories to store build products can eke out all kinds of timing differences due to NFS attribute caching. (I'm assuming you are building over NFS). Some time back I had quite a wild goose chase in this regard. It turned out to be ntpd hunting between two authoritative time servers which happened to differ slightly in timing. This was caused by different interpretations of a leap-second by different ntpd versions. If the NFS server and client use NTP and have an even number of servers at the same stratum this can happen. It was only noticeable because the timing of attribute cache refreshes tends to differ significantly between directories. When I put the objects in the same directory as the source files it didn't happen. Weird, hey? Perhaps you can investigate how time differs between your servers?

This kind of problem can be isolated by building on a local filesystem (which will be a lot faster anyway).

On another note, did you perhaps ^C a compile just before this happened? I could well imagine gcc -M partially writing it's dependency file, only for it to be mis-interpreted my make the next time around. Quite how that would have the effect you described, I'm not sure.

Dear Chaps,

I'm about ready to upgrade my all-in-one Makefile patch to the latest trunk version. I'm sure there are lots of things wrong with it and I'm open to suggestions on how to re-arrange things for your comfort, but one thing is for sure there are no hidden dependencies which might lead to oddities like this.

When it comes down to it, if the commands the build system actually executes are simple and clearly visible it makes tracking down this sort of behaviour a lot easier. A customised build system derived from generic rules inevitably contains more complexity than a hand-crafted makefile.

I've been looking at MySQL recently. It uses CMake and although the CMakeLists.txt files are quite small CMake generates quite a few very long makefiles. There seems to be very little use of pattern rules in these makefiles. Here is an example from MySQL 5.6.14 of the rules in just one generated makefile for turning just one C source file into an object and associated debugging files:

mysys/CMakeFiles/mysys.dir/mf_dirname.c.o: mysys/CMakeFiles/mysys.dir/flags.make mysys/CMakeFiles/mysys.dir/mf_dirname.c.o: mysys/mf_dirname.c $(CMAKE_COMMAND) -E cmake_progress_report /src/mysql-5.6.14/CMakeFiles $(CMAKE_PROGRESS_9) @echo "Building C object mysys/CMakeFiles/mysys.dir/mf_dirname.c.o" cd /src/mysql-5.6.14/mysys && /bin/gcc $(C_DEFINES) $(C_FLAGS) -fPIC -o CMakeFiles/mysys.dir/mf_dirname.c.o -c /src/mysql-5.6.14/mysys/mf_dirname.c

mysys/CMakeFiles/mysys.dir/mf_dirname.c.i: cmake_force @echo "Preprocessing C source to CMakeFiles/mysys.dir/mf_dirname.c.i" cd /src/mysql-5.6.14/mysys && /bin/gcc $(C_DEFINES) $(C_FLAGS) -fPIC -E /src/mysql-5.6.14/mysys/mf_dirname.c > CMakeFiles/mysys.dir/mf_dirname.c.i

mysys/CMakeFiles/mysys.dir/mf_dirname.c.s: cmake_force @echo "Compiling C source to assembly CMakeFiles/mysys.dir/mf_dirname.c.s" cd /src/mysql-5.6.14/mysys && /bin/gcc $(C_DEFINES) $(C_FLAGS) -fPIC -S /src/mysql-5.6.14/mysys/mf_dirname.c -o CMakeFiles/mysys.dir/mf_dirname.c.s

mysys/CMakeFiles/mysys.dir/mf_dirname.c.o.requires: .PHONY : mysys/CMakeFiles/mysys.dir/mf_dirname.c.o.requires

mysys/CMakeFiles/mysys.dir/mf_dirname.c.o.provides: mysys/CMakeFiles/mysys.dir/mf_dirname.c.o.requires $(MAKE) -f mysys/CMakeFiles/mysys.dir/build.make mysys/CMakeFiles/mysys.dir/mf_dirname.c.o.provides.build .PHONY : mysys/CMakeFiles/mysys.dir/mf_dirname.c.o.provides

mysys/CMakeFiles/mysys.dir/mf_dirname.c.o.provides.build: mysys/CMakeFiles/mysys.dir/mf_dirname.c.o

That particular makefile contains 163,196 similar lines. There are several such files in MySQL. If CMake generates similar Makefiles to this for Inkscape, I think it could make tracking down subtle problems such as the one experienced by mathog much harder than it needs to be. I have no idea whether CMake for Inkscape does generate files like this, but I can see why this version or a future version of CMake might if CMake's authors wanted to support primitive or incompatible make's like nmake.

If CMake produced simple makefiles with familiar rules like:

%.o: %.c $(CC) $(CFLAGS) -c $< -o $@

I doubt it would cause anyone tracking down bugs much concern, but if that is the case, why bother with CMake at all? The Makefile I produced consists of about 75 rules to build, install and source package the whole system, two-thirds of which are convenience rules which can be removed at a stroke if you are unhappy with them. By far the largest content of the Makefile is the list of files distributed in the source tar file. If there was a way to generate a clean list of these files, I could drop the size of the Makefile by 70%. If I remember correctly, my first cut of the Makefile was less than 500 lines. Despite the number of source files, Inkscape is a very straightforward package to compile.

I have explained by feelings about Waf in earlier threads, but I'll summarise them briefly here for completeness. Waf is unknown to most developers who might wish to help the Inkscape team. Although I'm sure it is straightforward to edit Waf's configuration files, if a developer encountered odd behaviour like mathog did, how might he debug it? ISTM he would need a good knowledge of python and a willingness to learn the intricacies of Waf's internals. I think most developers would become frustrated quite soon. Waf may be gaining in popularity, but it has a long way to go before ordinary developers would feel confident in their abilities. There is also the chance that Waf's popularity wanes and Inkscape is left holding the baby. I can see this being quite likely unless Waf offers a key advantages without sacrificing usability. Turning around Krzysztof's earlier remark about CMake, Waf needs to be quite a lot "better" than not just CMake/autotools but also make itself before it is adopted widely. On a technical note, from what I've read on this mailing list, building special debugging targets using Waf's command-line looks awkward compared to make's. Little things like that can be enough to stop other projects adopting it en-masse.

However, this is only the "make" aspect of building. Where CMake and Waf score is in generating compiler flags (etc.) for make from the multitude of platform variants and build options. I agree with Krzysztof that autoconf is a little too verbose and not particularly intuitive to work with and that CMake's syntax might not allow us to express these concepts clearly enough, but when CMake and chums are used merely to generate compiler flags many of these issues do not arise.

If you look at the patches I posted you will see that (with the exception of a few stragglers) I have separated "configuration" and "make"ing in such a way that you no longer need to use autoconf at all. In the next few days I am looking to tighten-up configure.ac and write a CMake file so you can directly compare autoconf and CMake's approach to setting compiler flags etc.

I hope this will help developers evaluate their options more clearly.

Regards,

Is.