2014/1/25 Josh Andler <scislac@...400...>:

Any quick instructions or tips? Are build options similar, such as special flags?

Normal build: ./waf configure ./waf build

Option list: ./waf --help

The system is rather minimal at the moment, so I didn't test anything besides vanilla building.

Regards, Krzysztof

2014-01-26 Johan Engelen <jbc.engelen@...2592...>:

Hi Krzysz, Great that you are working on unifying the build system again. There is a parallel effort using CMake, which you are aware of of course. CMake is used by many projects, so it may be easier for Inkscapers to start using that. That said, I don't feel very strongly about it.

What I think is important is that the system can target "ninja" or other build systems besides "make". CMake can create makefiles for many targets (makefiles, visual studio, ninja, ...) which is very nice and would be an extra step forward for our buildsystem. Can Waf do the same?

There is some support for generating MS Visual Studio project files, so I guess it would be entirely possible to write a generator for Ninja, but generally the premise of Waf is different than CMake. CMake is a build system generator, while Waf is the build system itself - it replaces 'make' completely.

The biggest advantages of Waf from my perspective:

1. It stores file hashes in a database which is persistent over runs, so you can use wildcards to specify the files to build, and it will figure out on its own that the wildcard includes new files which have to be compiled. This is simply impossible in a system where state information is stored only in file system attributes, such as make.

2. The build scripts use Python, which is a real programming language with very high expressive power. By comparison, programming anything even moderately complex in CMake is a bad joke.

3. As a consequence of 2, it is very easy to express complex tasks in a very small amount of code. Basically everything, from simple building, to maintenance tasks such as updating the potfiles, to release tarball generation can be integrated in the build scripts as Waf commands.

4. Since my first attempt, the documentation has improved massively - there are fairly comprehensive API docs available online.

5. I have commit access, so I can immediately fix any bugs we encounter :)

(ninja is speeding up 2geom compiles by an incredible amount, mostly because 2geom has many executable targets (all the toys, etc). Don't know what it will do for Inkscape.)

A no-change rebuild of Inkscape (~1000 tasks) takes around half a second on my machine, which is a lot faster than Autotools (I didn't measure it precisely, but it takes several seconds).

As far as I know, Waf is substantially faster than SCons, Autotools and make, but slower than lower level tools such as Ninja, Gyp and Tup. However, the difference is not very noticeable for projects the size of Inkscape, and IMO this drawback is outweighed by its ease of use.

Regards, Krzysztof

On Sun, 2014-01-26 at 21:08 +0100, Johan Engelen wrote:

• I really hope we can set all defaults correctly on Windows, so we

can build from a new checkout with the same amount of effort as we do now (almost zero).

With autotools? Even on Ubuntu, with apt-get build-deps and easy to aquire bzr, it's more difficult than I was expecting.

I wanted to test the build times between my machine Celeron toshiba vs my wife's new i7 System76 laptop. 4.5 hours vs 55 minutes. So, it's certainly faster to build than it used to be, although I guess the bulk of the time isn't make or autotools, but gcc.

But maybe if waf can compile things in a smarter way. Every time I edited document.h, I has to wait a long time for the compile because it currently compiles every included file (and I understand why it might need to) but perhaps some kind of manual command to say `make document.o only and relink`

I'd prefer waf because it's python, I wonder why there's a perl dep.

Martin,

2014-01-26 Johan Engelen <jbc.engelen@...2592...>:

Thanks for the clarification.

I'm trying to use it on Windows now.

• Extra dependency = perl (installed ActivePerl, is free)

Perl is a dependency of intltool-merge. AFAIK we don't need to use intltool-merge on Windows, because we don't build a desktop file there, so it's a question of further tweaking the build script. (Note that I didn't do any testing on Windows yet, so I don't expect it to work.)

• It finds MSVC first, but I can tell it not to do that (how to set the

default? CXX env var does not work, as it still thinks that is MSVC...)

We can set the variable waflib.compiler_cxx.cxx_compiler to set the compiler priority, e.g. add this before the functions: from Tools import compiler_cxx compiler_cxx.cxx_compiler['win32'] = ['g++', 'clang++']

I also thought about adding some magic code that detects what compiler did you specify in the CXX variable and picks the appropriate Waf tool, but I'm not sure whether this is generic enough to warrant

• It creates /build. Does it do out-of-source build automatically? That'd be

great.

Yes, you can tweak the output directory by using --out at config time or changing the 'out' variable in the wscript.

• Now it errors because it cannot find 'glib-mkenums' (indeed, the

executable is not in our devlibs)

This is loaded by the tool "glib2" by default, but we don't use it, so it's the same as with Perl - some tweaking is necessary.

• I really hope we can set all defaults correctly on Windows, so we can

build from a new checkout with the same amount of effort as we do now (almost zero).

In the earlier system, I did get to the point where saying "./waf dist" created the installation directory exactly like btool. I think this time around I'll be able to achieve that state much quicker.

2014-01-26 Vinícius dos Santos Oliveira <vini.ipsmaker@...400...>

Does it track include files as well? I did a simple test seconds ago and CMake was able to track include files too.

Use case: You change a header file included by some two implementation files. The implementation files are not changed, but the header file is changed. The implementation files including this header file should be recompiled.

Yes, it does track include files. I used Waf for some other projects before and never had an issue on that front.

2014-01-26 Martin Owens <doctormo@...400...>:

But maybe if waf can compile things in a smarter way. Every time I edited document.h, I has to wait a long time for the compile because it currently compiles every included file (and I understand why it might need to) but perhaps some kind of manual command to say `make document.o only and relink`

If you edit document.h, you must rebuild every file that includes document.h - there is no way around that, in any build system in existence. It is theoretically possible to check whether the edits change API/ABI and only recompile then, but this would be incredibly complex to implement; I don't even know whether this is possible at all.

Regards, Krzysztof

Em Dom, 2014-01-26 às 20:44 +0100, Krzysztof Kosiński escreveu:

1. It stores file hashes in a database which is persistent over runs,

so you can use wildcards to specify the files to build, and it will figure out on its own that the wildcard includes new files which have to be compiled. This is simply impossible in a system where state information is stored only in file system attributes, such as make.

Does it track include files as well? I did a simple test seconds ago and CMake was able to track include files too.

Use case: You change a header file included by some two implementation files. The implementation files are not changed, but the header file is changed. The implementation files including this header file should be recompiled.

2014-01-28 Johan Engelen <jbc.engelen@...2592...>:

Random idea: what would be superawesome if we could build a single file, something like:

waf compile --file=johan.cpp --cxxflags="..."

that would only compile (!) johan.cpp with _extra_ cxxflags added. For example to look at assembler/LLVM IR output.

You can say ./waf build --files=src/johan.cpp to compile only that file. (It also works with globs.) The more problematic part is convincing Waf to produce assembly output, because it would require changing the global definition of compilation tasks and disabling the link tasks. However, I think implementing something like this would be possible:

./waf build --assembly-output --files=src/johan.cpp

If you wanted to specify custom flags for some specific files at configure time, that would be easier, but the wscript would have to be written with this functionality in mind. E.g. you would have to define a separate taskgen for each group of files that receives custom flags.

Regards, Krzysztof

2014-01-30 Alexandre Prokoudine <alexandre.prokoudine@...400...>:

On Thu, Jan 30, 2014 at 2:36 PM, Isobel Knowles wrote:

...

I've only just come across this thread, so pardon me if I ask you what is waf and what problem does it seek to solve?

http://inkscape.13.x6.nabble.com/Waf-build-branch-td2844228.html

By the way, downsides 1, 2, 3 and 6 from that thread have been already addressed.

Regards, Krzysztof

2014-01-30 Martin Owens <doctormo@...400...>:

On Thu, 2014-01-30 at 13:14 +0100, Krzysztof Kosiński wrote:

...

By the way, downsides 1, 2, 3 and 6 from that thread have been already addressed.

Did the original developer acquire sense or was it forked?

As far as I can see, mostly the former.

Regards, Krzysztof

2014-01-30 Joel Holdsworth <joel@...1709...>:

From my experience CMake is a lot easier to use and more widely used than waf. We use waf at work, and it's ok - but not easy to use unless you really know python.

CMake requires you to learn a frustratingly weak and idiosyncratic scripting language which is only useful in CMake (and even there barely so), whereas learning Python is generally useful, especially to a programmer coming from a Java / C / C++ background.

Also CMake has a much larger community around it - hence support for Ninja, MSVC etc., wheras we find ourselves mostly talking directly to the Waf guy.

That's true; on the other hand, it's much easier to add something to Waf.

Also Debian refused to package waf because it wants to be a packaging system which clashes with their own packaging system.

Debian refused to package Waf for different reasons: 1. Originally the Waf tarball contained non-free docs (CC-BY-ND or something similar); it no longer contains them 2. The maintainer insisted that Waf should not be installed system-wide due to potential incompatibilities between versions 3. Most projects included the "Waf binary", a compressed version of Waf codw with stripped comments, which Debian says is incompatible with DFSG; this can be avoided by including the source distribution, which is only a little bigger.

The downside of CMake is the rather weird language of CMake files.

That's a pretty big downside IMO, see above.

All that being said, if there's a consensus we should use CMake, I'll go along with that

Regards, Krzysztof

On Fri, 31 Jan 2014 00:06:38 +1100 Nathan Hurst <njh@...1927...> wrote:

On Thu, Jan 30, 2014 at 01:45:12PM +0100, Krzysztof Kosi??ski wrote:

...

2014-01-30 Joel Holdsworth <joel@...1709...>:

...

From my experience CMake is a lot easier to use and more widely used than waf. We use waf at work, and it's ok - but not easy to use unless you really know python.

CMake requires you to learn a frustratingly weak and idiosyncratic scripting language which is only useful in CMake (and even there barely so), whereas learning Python is generally useful, especially to a programmer coming from a Java / C / C++ background.

I'm going to second this, as a language cmake is plain ugly (up there with php and apache config in ugliness :). For my own projects I've ended up with a hand rolled build system in preference to cmake (or worse, auto*). When a build tool is so hard to use for most programmers that they write their own in preference, you know it's not really working out.

If you want a programming language, use one :) We already have a dependency on python.

...

...

Also CMake has a much larger community around it - hence support for Ninja, MSVC etc., wheras we find ourselves mostly talking directly to the Waf guy.

That's true; on the other hand, it's much easier to add something to Waf.

Yes, a simple program that can be understood with general programming knowledge is always a better choice in the long run. I'm also not clear what benefits cmake really brings. My original attractions to cmake were: Firstly, the efficient parallelisation, but since then I've discovered I can write pretty much the same thing using python in a few tens of lines. The second attraction was the idea that cmake would have all the system integration stuff (find where the cairo headers and libraries are, set the appropriate g++ flags); but we soon found that support was spotty, and adding even trivial new rules was an expedition into the land of pain. The third attraction to me was the spiffy coloured build progress...

...

All that being said, if there's a consensus we should use CMake, I'll go along with that

I hope not. cmake was great for 2geom up to the point where we wanted it to do something more than make. Then it was just a pain. I still end up getting scislac or acspike to make changes to the cmake config rather than fight its arcane and inconsistent programming language.

What exactly did you want to do during a build that make couldn't accomplish, or are you talking about cmake's deficiencies?

I for one vote for something which keeps configuration separate from building. It is easy to write and maintain multi-platform makefiles for a project of inkscape's size and complexity, but only if the configuration system is properly separated from the build system. Once you start combining them like automake, cmake and waf do, it rapidly evolves into a programming project all of its own. Although I make no comment on its ability to configure a build, from what I've seen of waf's command-line syntax, as a "make" system it looks like a step backwards.

I know I'm only a user (and experienced make person), but the reason this matters to me is my platform is sufficiently different to that of the run-of-the-mill developer that I usually need to hack^H^H^H^Hport UNIX packages before they'll compile on my machines. If inkscape starts using waf or anything else that confuses configuration with building, it's just makes it "a land of pain".

Yours,

Is.

PS. My offer to write some 'nice' makefiles is still open.

On Thu, 30 Jan 2014 20:11:14 +0100 Johan Engelen <jbc.engelen@...2592...> wrote:

On 30-1-2014 20:02, Isobel Knowles wrote:

...

PS. My offer to write some 'nice' makefiles is still open.

No-one is stopping you :-) Make us dazzle...

:) Johan

Dear Johan,

OK. Before I do, do you think you could tell me what it is you want from the build? What are the essentials and what are the nice-to-haves? Where do you see the problems in the current architecture and what are its strengths?

I'm only after some pointers so I don't set off in the wrong direction.

I'll start the ball rolling with these:

- Documentation. People who are new to Inkscape should find it easy to edit the makefiles.

- Separate configuration and build cleanly so when the configuration phase doesn't do what you want for one reason or another, it is straight-forward to manually configure the build options in one or two files.

- Separate compilation and installation/packaging. Installation is generally performed as root, and all too often in GNU software libtool and the like get their teeth into a 'make install', turning it into compilation job. This is wrong.

- An ordinary build job should do the minimum work in the minimum time. For this I propose a flat makefile structure, perhaps composed of included makefiles. Parallel makes like this perform correctly and have no scalability issues whereas recursive makefiles are beset with problems and complication.

- It should be simple to modify build flags and make alternative targets for debugging purposes.

I wouldn't describe it as dazzling, but it's a start.

Yours,

Is.

On Thu, 2014-01-30 at 13:45 +0100, Krzysztof Kosiński wrote:

All that being said, if there's a consensus we should use CMake, I'll go along with that

Python on the design of CMake and autotools: https://www.youtube.com/watch?v=e2PyeXRwhCE

I vote for waf, if there was ever a vote. ;-)

Martin,