on Windows 10, current trunk, I normally execute the following two Cmake commands: mingw32-make -j 2 mingw32-make install
The time required for this is 300 seconds, 150 seconds for each step. This time is the same no matter how small the change, even if I just insert one single blank character somewhere. Previously, using btool, the total time required for this would be about 10 seconds or so, for a complete rebuild of Inkscape. So CMake is about 30 times slower than btool, which makes it almost impossible to do diagnostic work with Inkscape, any way of speeding it up?
Alvin
-- View this message in context: http://inkscape.13.x6.nabble.com/is-it-possible-to-speed-up-CMake-tp4979104.... Sent from the Inkscape - Dev mailing list archive at Nabble.com.
On 1 Mar 2017 21:20, "alvinpenner" <penner@...1856...> wrote:
on Windows 10, current trunk, I normally execute the following two Cmake commands: mingw32-make -j 2 mingw32-make install
The time required for this is 300 seconds, 150 seconds for each step. This time is the same no matter how small the change, even if I just insert one single blank character somewhere. Previously, using btool, the total time required for this would be about 10 seconds or so, for a complete rebuild of Inkscape.
A complete *rebuild* of Inkscape in 10 seconds using one computer only? What are the specs of that machine?
So CMake is about 30 times slower than btool, which makes it almost impossible to do diagnostic work with Inkscape, any way of speeding it up?
Alvin
-- View this message in context: http://inkscape.13.x6.nabble. com/is-it-possible-to-speed-up-CMake-tp4979104.html Sent from the Inkscape - Dev mailing list archive at Nabble.com.
------------------------------------------------------------ ------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________ Inkscape-devel mailing list Inkscape-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/inkscape-devel
On 01-Mar-2017 12:25, Olof Bjarnason wrote:
A complete *rebuild* of Inkscape in 10 seconds using one computer only? What are the specs of that machine?
That isn't what he said. Change a space in some source file and rebuild and the work which must be done is:
recompile module rebuild library relink executable
cmake does that, eventually, but along the way it scans through a lot of other stuff which is why it takes so long. AP's point is that btool was much more efficient about it, not wasting time getting this job done, so when working on one module it was quick to rebuild, but with cmake, it isn't.
On a similar note, I still can't get cmake to build Inkscape on Ubuntu 14.04 LTS 32 bit, and have to manually enter a different link command (changing cms library linkage) every time.
From where I sit, in terms of performance and functionality, cmake was a large step backwards from what we had previously on both Windows and Linux.
Regards,
David Mathog mathog@...1176... Manager, Sequence Analysis Facility, Biology Division, Caltech
Am 01.03.2017 um 21:19 schrieb alvinpenner:
on Windows 10, current trunk, I normally execute the following two Cmake commands: mingw32-make -j 2 mingw32-make install
The time required for this is 300 seconds, 150 seconds for each step. This time is the same no matter how small the change, even if I just insert one single blank character somewhere. Previously, using btool, the total time required for this would be about 10 seconds or so, for a complete rebuild of Inkscape. So CMake is about 30 times slower than btool, which makes it almost impossible to do diagnostic work with Inkscape, any way of speeding it up?
Alvin
Hi Alvin,
I can confirm this behavior. What takes that long is scanning dependencies (i.e. CMake checks which dependencies every file has and if one of those changed and a rebuild of that file is required). This step seems to be especially slow on Windows, probably due to slower file IO compared to *nix.Therefore every build will take some minimum time, even if no files changed. On my machine it was considerably faster though (~45 s if I remember correctly), but still annoying when working on something.
To solve this issue I'd recommend you to use ninja to build instead of ming32-make. I have that successfully running on Windows 10 now and incremental rebuilds only take ~10s. I documented the necessary steps in our Wiki not long ago: http://wiki.inkscape.org/wiki/index.php/CMake#Using_CMake_with_Ninja_to_buil...
Regards, Eduard
To solve this issue I'd recommend you to use ninja to build instead of
ming32-make. I have that successfully running on Windows 10 now and incremental rebuilds only take ~10s.
Thanks, I'll definitely check that out. I am currently looking at only two bugs, so if I can solve them before I lose my patience, then I'll stay with CMake, otherwise I'll try ninja.
Alvin
-- View this message in context: http://inkscape.13.x6.nabble.com/is-it-possible-to-speed-up-CMake-tp4979104p... Sent from the Inkscape - Dev mailing list archive at Nabble.com.
Am 01.03.2017 um 22:09 schrieb alvinpenner:
To solve this issue I'd recommend you to use ninja to build instead of
ming32-make. I have that successfully running on Windows 10 now and incremental rebuilds only take ~10s.
Thanks, I'll definitely check that out. I am currently looking at only two bugs, so if I can solve them before I lose my patience, then I'll stay with CMake, otherwise I'll try ninja.
Alvin
Definitely try it! I was in exactly the same position some weeks ago, and afterwards I really regretted not having tried ninja earlier (It could have saved me *a lot* of time!). You really only have to download one tiny executable (391 kB) and that's it! I'm exclusively using ninja for new builds now.
What does ninja do more technically? Magic? :D
On 1 Mar 2017 22:18, "Eduard Braun" <eduard.braun2@...173...> wrote:
Am 01.03.2017 um 22:09 schrieb alvinpenner:
To solve this issue I'd recommend you to use ninja to build instead of
ming32-make. I have that successfully running on Windows 10 now and incremental rebuilds only take ~10s.
Thanks, I'll definitely check that out. I am currently looking at only
two
bugs, so if I can solve them before I lose my patience, then I'll stay
with
CMake, otherwise I'll try ninja.
Alvin
Definitely try it! I was in exactly the same position some weeks ago, and afterwards I really regretted not having tried ninja earlier (It could have saved me *a lot* of time!). You really only have to download one tiny executable (391 kB) and that's it! I'm exclusively using ninja for new builds now.
Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________ Inkscape-devel mailing list Inkscape-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/inkscape-devel
Am 01.03.2017 um 22:21 schrieb Olof Bjarnason:
What does ninja do more technically? Magic? :D
On 1 March 2017 at 22:31, Eduard Braun <eduard.braun2@...173...> wrote:
Am 01.03.2017 um 22:21 schrieb Olof Bjarnason:
What does ninja do more technically? Magic? :D
Cool, sounds like a neat philosophy.
And there's even instructions ready on the wiki!
http://wiki.inkscape.org/wiki/index.php/CMake#Using_CMake_with_Ninja_to_buil...
Here:
mkdir build cd build cmake -G Ninja /path/to/inkscape ninja ninja install
I have a feeling the reason CMake is slow is because it generates recursive makefiles.
Recursive makefiles are a sin. The dependence graph is incomplete because each instance of make only knows about a part of the dependence graph stored in the current makefile being processed. This can lead to lots of problems:
1. race conditions (for manually written makefiles - hopefuly CMake doesn't make such mistakes) 2. *inefficient* (if a bunch of sub projects depends on another common project, the dependence checking will be repeated lots of times)
http://aegis.sourceforge.net/auug97.pdf
I'm guessing Ninja is fast because it uses a whole project makefile? I wish the UNIX makefile backend for CMake can be rewritten to do the same. BTW, a whole project makefile doesn't mean putting everything in 1 file. It would be a bunch of makefiles and helper makefiles joined together using include statements.
I've had the experience of rewriting a set of recursive makefiles to a whole project makefile and 1 big pain was homographic variables overwriting each other. Often in each makefile, we want to access files several directories up, so there would be a variable like TOP_LEVEL = ../../../../
But TOP_LEVEL can be different for each project, so you have to be careful that they don't clash.
For me, a 1 line change takes 30s to build.
On Wed, Mar 1, 2017 at 1:36 PM, Olof Bjarnason <olof.bjarnason@...400...> wrote:
On 1 March 2017 at 22:31, Eduard Braun <eduard.braun2@...173...> wrote:
Am 01.03.2017 um 22:21 schrieb Olof Bjarnason:
What does ninja do more technically? Magic? :D
Cool, sounds like a neat philosophy.
And there's even instructions ready on the wiki!
http://wiki.inkscape.org/wiki/index.php/CMake#Using_CMake_with_Ninja_to_buil...
Here:
mkdir build cd build cmake -G Ninja /path/to/inkscape ninja ninja install
Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________ Inkscape-devel mailing list Inkscape-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/inkscape-devel
On Wed, 2017-03-01 at 22:31 +0100, Eduard Braun wrote:
Am 01.03.2017 um 22:21 schrieb Olof Bjarnason:
What does ninja do more technically? Magic? :D
Is this something that should be documented on the windows building READMEs, wikis etc etc?
Sounds like something that should be default for windows devs.
Best Regards, Martin Owens
It's not just the Windows builds that are slow. Any where you use the generated UNIX makefiles, it will be slow.
Has anyone benchmarked Ninja vs GNU make build times for a one line change on Linux?
My current incremental build time using GNU make is ~30s. I've never tried Ninja, but if it's a big gain, I'll switch.
On Wed, Mar 1, 2017 at 3:26 PM, Martin Owens <doctormo@...400...> wrote:
On Wed, 2017-03-01 at 22:31 +0100, Eduard Braun wrote:
Am 01.03.2017 um 22:21 schrieb Olof Bjarnason:
What does ninja do more technically? Magic? :D
Is this something that should be documented on the windows building READMEs, wikis etc etc?
Sounds like something that should be default for windows devs.
Best Regards, Martin Owens
Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________ Inkscape-devel mailing list Inkscape-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/inkscape-devel
On Wed, 2017-03-01 at 17:22 -0800, Yale Zhang wrote:
It's not just the Windows builds that are slow. Any where you use the generated UNIX makefiles, it will be slow.
Has anyone benchmarked Ninja vs GNU make build times for a one line change on Linux?
My current incremental build time using GNU make is ~30s. I've never tried Ninja, but if it's a big gain, I'll switch.
I just switched a couple of weeks ago. For me:
GNU Make ~30s Ninja ~15s
The difference is even greater if you've made a typo and the file doesn't compile.
Tav
On 03/02/2017 02:22 AM, Yale Zhang wrote:
It's not just the Windows builds that are slow. Any where you use the generated UNIX makefiles, it will be slow.
Has anyone benchmarked Ninja vs GNU make build times for a one line change on Linux?
I was curious so I tested it (Debian jessie 64 bit) and measured the times. Ninja is consistently faster but now by much in most cases.
== Not changing anything in the sources and rebuilding ==
"make -j8" without changing anything real 0m7.342s user 0m12.732s sys 0m1.384s
"ninja -j8" without changing anything real 0m0.239s user 0m0.200s sys 0m0.028s
== Changing a single file and rebuilding ==
"make -j8" after "touch src/2geom/path.cpp" which triggers re-compilation of path.cpp and relinking of lib2geom, libinkscape_base, inkscape, tests etc. real 0m20.341s user 0m35.060s sys 0m3.892s
"ninja -j8" after "touch src/2geom/path.cpp" real 0m15.573s user 0m22.352s sys 0m2.444s
== Changing a couple of files and rebuilding ==
"make -j8" after "touch src/2geom/*.cpp" real 0m34.858s user 2m21.760s sys 0m15.516s
"ninja -j8" real 0m28.501s user 2m3.020s sys 0m13.848s
== I'm working on lib2geom (outside Inkscape) and changed things in path.cpp so I measured the build times too ==
"make -j8" real 0m16.633s user 1m18.732s sys 0m9.208s
real 0m16.219s user 1m19.020s sys 0m9.496s
real 0m16.680s user 1m18.376s sys 0m9.856s
"ninja -j8" real 0m15.840s user 1m16.016s sys 0m8.580s
real 0m15.782s user 1m15.784s sys 0m8.728s
== Re-building Inkscape from scratch ==
"cmake -G Ninja -DCMAKE_BUILD_TYPE=Debug ../trunk/" real 0m9.123s user 0m6.300s sys 0m0.888s
"ninja -j8" on the freshly generated ninja files, building the whole project: real 7m8.485s user 49m15.668s sys 5m7.628s
"cmake -DCMAKE_BUILD_TYPE=Debug ../trunk/" real 0m10.208s user 0m7.100s sys 0m1.160s
"make -j8" real 7m18.929s user 49m17.140s sys 5m19.080s
Best Regards, Alexander
Am 02.03.2017 um 00:26 schrieb Martin Owens:
Is this something that should be documented on the windows building READMEs, wikis etc etc?
Sounds like something that should be default for windows devs.
Best Regards, Martin Owens
As noted before I already added a section on ninja in our Wiki [1] and linked to it from the articles on building Inkscape on Windows [2,3]. I only figured it out myself about 2 weeks ago, so I guess it did not get about yet...
Regards, Eduard
[1] http://wiki.inkscape.org/wiki/index.php/CMake#Using_CMake_with_Ninja_to_buil... [2] http://wiki.inkscape.org/wiki/index.php/Compiling_Inkscape_on_Windows [3] http://wiki.inkscape.org/wiki/index.php/Compiling_Inkscape_on_Windows_64-bit
On Wed, Mar 01, 2017 at 02:09:48PM -0700, alvinpenner wrote:
then I'll stay with CMake, otherwise I'll try ninja.
You'll still be using cmake.
cmake is a meta build system: it relies on something else to actually instruct compilers and friends. On most *nix systems it defaults to GNU make, on windows it can create msvc or mingw (which is what you use, I get?), and so on. Ninja is comparable to those latter tools, rather than cmake, and it happens to be cross platform.
A complete *rebuild* of Inkscape in 10 seconds using one computer only?
sorry, I got carried away with my own rhetoric. I meant take an existing build, make one trivial change in one file, and rebuild. I am tracking down bugs, so I typically do many recompiles with only trivial changes between them.
Alvin
-- View this message in context: http://inkscape.13.x6.nabble.com/is-it-possible-to-speed-up-CMake-tp4979104p... Sent from the Inkscape - Dev mailing list archive at Nabble.com.
Am Mittwoch, 1. März 2017, 13:19:05 CET schrieb alvinpenner:
on Windows 10, current trunk, I normally execute the following two Cmake commands: mingw32-make -j 2 mingw32-make install
The time required for this is 300 seconds, 150 seconds for each step. This time is the same no matter how small the change, even if I just insert one single blank character somewhere. Previously, using btool, the total time required for this would be about 10 seconds or so, for a complete rebuild of Inkscape. So CMake is about 30 times slower than btool, which makes it almost impossible to do diagnostic work with Inkscape, any way of speeding it up?
At least with "make" you can speed that up a bit by running
mingw32-make -j2 install
right away. It will build everything that needs to be build and you save one pass of dependency checking. Not sure about ninja, but I would guess it behaves similar.
Alvin
Tobias
Hello,
Does cmake clean the "cache" (.o, etc) ?
The make installé take a constant time no ? Maybe it s the bottle neck ?
Le 2 mars 2017 11:40:18 CET, Tobias Ellinghaus <houz@...173...> a écrit :
Am Mittwoch, 1. März 2017, 13:19:05 CET schrieb alvinpenner:
on Windows 10, current trunk, I normally execute the following two
Cmake
commands: mingw32-make -j 2 mingw32-make install
The time required for this is 300 seconds, 150 seconds for each step.
This
time is the same no matter how small the change, even if I just
insert one
single blank character somewhere. Previously, using btool, the total time required for this would be
about 10
seconds or so, for a complete rebuild of Inkscape. So CMake is about 30 times slower than btool, which makes it almost impossible to do diagnostic work with Inkscape, any way of speeding
it up?
At least with "make" you can speed that up a bit by running
mingw32-make -j2 install
right away. It will build everything that needs to be build and you save one pass of dependency checking. Not sure about ninja, but I would guess it
behaves similar.
Alvin
Tobias
Am Donnerstag, 2. März 2017, 11:55:42 CET schrieb Bardot Jérôme:
Hello,
Does cmake clean the "cache" (.o, etc) ?
I don't hope so, that would make using a tool like "make" or "ninja" pointless.
The make installé take a constant time no ? Maybe it s the bottle neck ?
Depends. When everything is build already then it's more or less constant, if some dependencies are not fulfilled yet then it takes more time as it compiles/ generates them. The bottleneck is the initial checking of the state of the dependencies, i.e., if any source files got changed and need to be reprocessed. Running "make install" directly gets rid of one such pass.
[...]
Tobias
participants (11)
-
Alexander Brock -
alvinpenner -
Bardot Jérôme -
Eduard Braun -
Martin Owens -
mathog -
Mattia Rizzolo -
Olof Bjarnason -
Tavmjong Bah -
Tobias Ellinghaus -
Yale Zhang