On Sun, Feb 15, 2015, at 03:52 AM, Tavmjong Bah wrote:

On Sat, 2015-02-14 at 17:00 +0100, Johan Engelen wrote:

...

Hi all, Whoever is interested in Jenkins / testing Inkscape: let's schedule a meeting on IRC, so I can explain what I've set up and how you can use the testing system, help improve things and maintain the testing system.

How about Wednesday at 19:00 UTC ?

I'll also try to attend but I can't guarantee it at the moment.

Tav

Me too

Not sure what we have time for this evening but would be interesting to touch on the future about (1) fixing the existing failing unit tests (2) change of test framework (google test I assume from earlier mailing threads).

I've tested Google Test the last few days for the first time and the current setup I have used with a single test.cpp which automatically (macro) loads all tests is way too time consuming to run every single time you change something. At ~400 tests the building of tests takes 20 seconds and running tests about 5 seconds. Doing TDD/BDD the total time of 25 seconds must go down to below 5 seconds (below 1 second preferably). I'm sure I'm doing something wrong in this project when it comes to the structure of tests. In the end my main question is, how do I _build_ and run just a single test suite when doing TDD with Google Test? Is it possible? -- Christoffer Holmstedt

2015-02-17 22:38 GMT+01:00 Johan Engelen <jbc.engelen@...2592...>:

Great that you are joining the discussion. See you tomorrow at 19:00 UTC

http://www.timeanddate.com/worldclock/fixedtime.html?msg=Inkscape+IRC+meetin...

in #inkscape-devel on Freenode.

cheers, Johan

On 14-2-2015 17:00, Johan Engelen wrote:

...

Hi all, Whoever is interested in Jenkins / testing Inkscape: let's schedule a meeting on IRC, so I can explain what I've set up and how you can use the testing system, help improve things and maintain the testing system.

How about Wednesday at 19:00 UTC ?

cheers, Johan

---

Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server from Actuate! Instantly Supercharge Your Business Reports and Dashboards with Interactivity, Sharing, Native Excel Exports, App Integration & more Get technology previously reserved for billion-dollar corporations, FREE http://pubads.g.doubleclick.net/gampad/clk?id=190641631&iu=/4140/ostg.cl... _______________________________________________ Inkscape-devel mailing list Inkscape-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/inkscape-devel

@Mark Exactly, but that is only the "run" part in my question concerning _building_ the tests and running them. The main problem is that the build time is too long when building all the tests into one giant test executable. So I can rephrase my question:

Has anyone used Google Test and succesfully created a structure of test suites and cases to be able to build only a single test suites or a few test cases when needed?

Of course, before committing the changes all tests must be run at least once, but during development you only want to iterate over a single test suite/case you're focusing on at the moment. -- Christoffer Holmstedt

2015-02-18 15:32 GMT+01:00 Mark Harmer <drivehappy@...400...>:

You'll want to use the --gtest_filter option: https://code.google.com/p/googletest/wiki/AdvancedGuide#Running_a_Subset_of_...

On Tue, Feb 17, 2015 at 10:29 PM, Christoffer Holmstedt <christoffer.holmstedt@...400...> wrote:

...

I've tested Google Test the last few days for the first time and the current setup I have used with a single test.cpp which automatically (macro) loads all tests is way too time consuming to run every single time you change something. At ~400 tests the building of tests takes 20 seconds and running tests about 5 seconds. Doing TDD/BDD the total time of 25 seconds must go down to below 5 seconds (below 1 second preferably). I'm sure I'm doing something wrong in this project when it comes to the structure of tests. In the end my main question is, how do I _build_ and run just a single test suite when doing TDD with Google Test? Is it possible? -- Christoffer Holmstedt

It's within another project and I'm afraid I can't share any links to it as it is not an open project.

Great, you have good experience from gtest and you nailed it now as a general compilation/linking performance problem. The main problem in our current project is that it is only one main file which runs all tests but as you suggests I assume it is possible to split the test suites into multiple files and run only a small part of them when needed...and before committing you run them all. Though when searching gtest documentation I haven't found any "best practice" suggestion on how to structure the test suites into some kind of hierarchy. If you know how to do this properly and maybe know an open project which have a good structure tell me ;)

In the upcoming weeks I will set up two test branches to try out both Google test and Boost test (in separate branches) at the moment the current tests we have are written in cpptest (I think). When I get going I will send an email to the mailing list, but as I said above any help in advance on best practice is appreciated. -- Christoffer Holmstedt

2015-02-19 6:40 GMT+01:00 Mark Harmer <drivehappy@...400...>:

Ah, I misunderstood the question. I think this is a general compilation/linking performance question; I've used gtest in other projects and by itself it's been fast when rebuilding unittests (that is, re-linking against the gtest library already built).

I don't know how you have the code setup, and I don't know what you've tried, but I would look at the following:

• Determining if the overhead is in compilation or linking
• Splitting the testsuites into separate .cpp's
• Building as Debug (-O0)

I've only done minimal development within Inkscape, I'm curious: are the tests you wrote within it or a standalone project?

On Wed, Feb 18, 2015 at 9:38 AM, Christoffer Holmstedt <christoffer.holmstedt@...400...> wrote:

...

@Mark Exactly, but that is only the "run" part in my question concerning _building_ the tests and running them. The main problem is that the build time is too long when building all the tests into one giant test executable. So I can rephrase my question:

Has anyone used Google Test and succesfully created a structure of test suites and cases to be able to build only a single test suites or a few test cases when needed?

Of course, before committing the changes all tests must be run at least once, but during development you only want to iterate over a single test suite/case you're focusing on at the moment. -- Christoffer Holmstedt

2015-02-18 15:32 GMT+01:00 Mark Harmer <drivehappy@...400...>:

...

You'll want to use the --gtest_filter option:

https://code.google.com/p/googletest/wiki/AdvancedGuide#Running_a_Subset_of_...

On Tue, Feb 17, 2015 at 10:29 PM, Christoffer Holmstedt <christoffer.holmstedt@...400...> wrote:

...

I've tested Google Test the last few days for the first time and the current setup I have used with a single test.cpp which automatically (macro) loads all tests is way too time consuming to run every single time you change something. At ~400 tests the building of tests takes 20 seconds and running tests about 5 seconds. Doing TDD/BDD the total time of 25 seconds must go down to below 5 seconds (below 1 second preferably). I'm sure I'm doing something wrong in this project when it comes to the structure of tests. In the end my main question is, how do I _build_ and run just a single test suite when doing TDD with Google Test? Is it possible? -- Christoffer Holmstedt

---

Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server from Actuate! Instantly Supercharge Your Business Reports and Dashboards with Interactivity, Sharing, Native Excel Exports, App Integration & more Get technology previously reserved for billion-dollar corporations, FREE

http://pubads.g.doubleclick.net/gampad/clk?id=190641631&iu=/4140/ostg.cl... _______________________________________________ Inkscape-devel mailing list Inkscape-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/inkscape-devel

Thanks for the information Jon, sounds promising. If time permits I will have to dig deep and split all those test cases into separate files. -- Christoffer Holmstedt

2015-02-19 21:05 GMT+01:00 Jon A. Cruz <jon@...18...>:

On Wed, Feb 18, 2015, at 10:05 PM, Christoffer Holmstedt wrote:

...

It's within another project and I'm afraid I can't share any links to it as it is not an open project.

Great, you have good experience from gtest and you nailed it now as a general compilation/linking performance problem. The main problem in our current project is that it is only one main file which runs all tests but as you suggests I assume it is possible to split the test suites into multiple files and run only a small part of them when needed...and before committing you run them all. Though when searching gtest documentation I haven't found any "best practice" suggestion on how to structure the test suites into some kind of hierarchy. If you know how to do this properly and maybe know an open project which have a good structure tell me ;)

In the upcoming weeks I will set up two test branches to try out both Google test and Boost test (in separate branches) at the moment the current tests we have are written in cpptest (I think). When I get going I will send an email to the mailing list, but as I said above any help in advance on best practice is appreciated.

Normally one would want to break test sources up to have a .cpp per test case at most. If a project has good organization you might have one test .cpp per main source .cpp (as opposed to projects that randomly clump things into .cpp source files). Additionally common fixtures and other such helpers would be broken out into their own .h & .cpp files. All those things will help reduce incremental build times for the test executable.

So you would have foo.cpp hold the source for the class Foo. Then you would have foo-test.cpp that holds the tests that are associated with that class. They could hold one or more test "cases" that collect the individual tests. When working on foo.cpp you could run GTest with Foo.* if that matched your test case name. Then run with all before committing.

GTest really doesn't focus on hierarchy. So it doesn't do the same nested groups like Boost Test does with its suites.

And as far as breaking up the tests to separate executables goes (the other way to get organization), it might make sense if your project's code created more than one executable or library. Otherwise just splitting out from a single monolithic .cpp file should really help your build times.

-- Jon A. Cruz jon@...18...

---

Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server from Actuate! Instantly Supercharge Your Business Reports and Dashboards with Interactivity, Sharing, Native Excel Exports, App Integration & more Get technology previously reserved for billion-dollar corporations, FREE http://pubads.g.doubleclick.net/gampad/clk?id=190641631&iu=/4140/ostg.cl... _______________________________________________ Inkscape-devel mailing list Inkscape-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/inkscape-devel