Hi guys,
First off, I was wondering how many people out there have been running the unit tests along with builds.
Secondly, the system we have now probably is in need of serious updating. Some of the test cases themselves aren't passing correctly, but the framework is also outdated.
So does anyone have preferences in this area? Is there some framework we should consider other than Boost Test and Google Test. Or has it not been an issue for others and I should just go ahead and update to whatever keeps things working?
Thanks.
Output from "make check" on a recent(ish) checkout (r12340) shows numerous test failures (log attached). I had to exclude it from the daily PPA build for this reason.
I'm not familiar with the structure of the tests, but can anyone confirm if any failures are caused by unimplemented features or are they things that used to work and have broken since the test suite was written? As I understand it, the "make check" (or "make test") target should always be run in our daily builds so we get automated warnings. Really, this means that it can only cover implemented features by default. I guess we could write a separate Make target (something like "make check-all") that includes unimplemented stuff,
As for the test framework itself, I have no experience... is there a compelling reason to migrate from CxxTest to another framework? We should really aim to get rid of our embedded copy of CxxTest, however, given that it is outdated and adds to our maintenance burden. See bug #1094771 for the request to use an external CxxTest implementation [1]. Note that we can't really do this until there is a Mac port of the CxxTest suite.
Best wishes,
AV
[1] https://bugs.launchpad.net/inkscape/+bug/1094771
On 16 July 2013 16:33, Jon Cruz <jon@...18...> wrote:
Hi guys,
First off, I was wondering how many people out there have been running the unit tests along with builds.
Secondly, the system we have now probably is in need of serious updating. Some of the test cases themselves aren't passing correctly, but the framework is also outdated.
So does anyone have preferences in this area? Is there some framework we should consider other than Boost Test and Google Test. Or has it not been an issue for others and I should just go ahead and update to whatever keeps things working?
Thanks.
See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clk... _______________________________________________ Inkscape-devel mailing list Inkscape-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/inkscape-devel
I've been avoiding the test suite as it has had problems any time I've run it with clean trunk.
Other projects I've worked on implement a strict trunk test passing feature that forces all branches to pass their unit testing before being allowed to merge into trunk. Our lack of trunk controls make this harder to do culturally from what I can see. Your thoughts?
Thanks for the log Alex, a lot of those verb failures are actually because the ALIGN verbs were all pushed up by 5 by scripting verbs in the enum but not in the struct. Code was fixed and the enum matches the struct somewhat better, although the test is failing as if it wanted broken code so I'm thinking that verb test needs to be improved.
I don't have any knowledge of C/C++ testing frameworks. I'd build an entire set of python tests and run via dbus and bindings if I were to make a test suite from scratch.
Best Regards, Martin Owens
On Tue, 2013-07-16 at 17:12 +0100, Alex Valavanis wrote:
Output from "make check" on a recent(ish) checkout (r12340) shows numerous test failures (log attached). I had to exclude it from the daily PPA build for this reason.
I'm not familiar with the structure of the tests, but can anyone confirm if any failures are caused by unimplemented features or are they things that used to work and have broken since the test suite was written? As I understand it, the "make check" (or "make test") target should always be run in our daily builds so we get automated warnings. Really, this means that it can only cover implemented features by default. I guess we could write a separate Make target (something like "make check-all") that includes unimplemented stuff,
As for the test framework itself, I have no experience... is there a compelling reason to migrate from CxxTest to another framework? We should really aim to get rid of our embedded copy of CxxTest, however, given that it is outdated and adds to our maintenance burden. See bug #1094771 for the request to use an external CxxTest implementation [1]. Note that we can't really do this until there is a Mac port of the CxxTest suite.
Best wishes,
AV
[1] https://bugs.launchpad.net/inkscape/+bug/1094771
On 16 July 2013 16:33, Jon Cruz <jon@...18...> wrote:
Hi guys,
First off, I was wondering how many people out there have been running the unit tests along with builds.
Secondly, the system we have now probably is in need of serious updating. Some of the test cases themselves aren't passing correctly, but the framework is also outdated.
So does anyone have preferences in this area? Is there some framework we should consider other than Boost Test and Google Test. Or has it not been an issue for others and I should just go ahead and update to whatever keeps things working?
Thanks.
See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clk... _______________________________________________ Inkscape-devel mailing list Inkscape-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/inkscape-devel
See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clk... _______________________________________________ Inkscape-devel mailing list Inkscape-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/inkscape-devel
Here's an updated log from "make check". I suspect that some of these issues are quite easy to fix. As a first step, can anyone comment on...
1. Which of these are easy/trivial to fix
2. Which are hard problems to solve
3. Which (if any) relate to unimplemented features
If we can clean out the errors ASAP, I can set up automated unit testing in the PPA.
AV
On 16 July 2013 17:42, Martin Owens <doctormo@...400...> wrote:
I've been avoiding the test suite as it has had problems any time I've run it with clean trunk.
Other projects I've worked on implement a strict trunk test passing feature that forces all branches to pass their unit testing before being allowed to merge into trunk. Our lack of trunk controls make this harder to do culturally from what I can see. Your thoughts?
Thanks for the log Alex, a lot of those verb failures are actually because the ALIGN verbs were all pushed up by 5 by scripting verbs in the enum but not in the struct. Code was fixed and the enum matches the struct somewhat better, although the test is failing as if it wanted broken code so I'm thinking that verb test needs to be improved.
I don't have any knowledge of C/C++ testing frameworks. I'd build an entire set of python tests and run via dbus and bindings if I were to make a test suite from scratch.
Best Regards, Martin Owens
On Tue, 2013-07-16 at 17:12 +0100, Alex Valavanis wrote:
Output from "make check" on a recent(ish) checkout (r12340) shows numerous test failures (log attached). I had to exclude it from the daily PPA build for this reason.
I'm not familiar with the structure of the tests, but can anyone confirm if any failures are caused by unimplemented features or are they things that used to work and have broken since the test suite was written? As I understand it, the "make check" (or "make test") target should always be run in our daily builds so we get automated warnings. Really, this means that it can only cover implemented features by default. I guess we could write a separate Make target (something like "make check-all") that includes unimplemented stuff,
As for the test framework itself, I have no experience... is there a compelling reason to migrate from CxxTest to another framework? We should really aim to get rid of our embedded copy of CxxTest, however, given that it is outdated and adds to our maintenance burden. See bug #1094771 for the request to use an external CxxTest implementation [1]. Note that we can't really do this until there is a Mac port of the CxxTest suite.
Best wishes,
AV
[1] https://bugs.launchpad.net/inkscape/+bug/1094771
On 16 July 2013 16:33, Jon Cruz <jon@...18...> wrote:
Hi guys,
First off, I was wondering how many people out there have been running the unit tests along with builds.
Secondly, the system we have now probably is in need of serious updating. Some of the test cases themselves aren't passing correctly, but the framework is also outdated.
So does anyone have preferences in this area? Is there some framework we should consider other than Boost Test and Google Test. Or has it not been an issue for others and I should just go ahead and update to whatever keeps things working?
Thanks.
See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clk... _______________________________________________ Inkscape-devel mailing list Inkscape-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/inkscape-devel
See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clk... _______________________________________________ Inkscape-devel mailing list Inkscape-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/inkscape-devel
(I'll upload a similar test log for the Gtk+ 3 build shortly)
On 17 July 2013 11:11, Alex Valavanis <valavanisalex@...400...> wrote:
Here's an updated log from "make check". I suspect that some of these issues are quite easy to fix. As a first step, can anyone comment on...
Which of these are easy/trivial to fix
Which are hard problems to solve
Which (if any) relate to unimplemented features
If we can clean out the errors ASAP, I can set up automated unit testing in the PPA.
AV
On 16 July 2013 17:42, Martin Owens <doctormo@...400...> wrote:
I've been avoiding the test suite as it has had problems any time I've run it with clean trunk.
Other projects I've worked on implement a strict trunk test passing feature that forces all branches to pass their unit testing before being allowed to merge into trunk. Our lack of trunk controls make this harder to do culturally from what I can see. Your thoughts?
Thanks for the log Alex, a lot of those verb failures are actually because the ALIGN verbs were all pushed up by 5 by scripting verbs in the enum but not in the struct. Code was fixed and the enum matches the struct somewhat better, although the test is failing as if it wanted broken code so I'm thinking that verb test needs to be improved.
I don't have any knowledge of C/C++ testing frameworks. I'd build an entire set of python tests and run via dbus and bindings if I were to make a test suite from scratch.
Best Regards, Martin Owens
On Tue, 2013-07-16 at 17:12 +0100, Alex Valavanis wrote:
Output from "make check" on a recent(ish) checkout (r12340) shows numerous test failures (log attached). I had to exclude it from the daily PPA build for this reason.
I'm not familiar with the structure of the tests, but can anyone confirm if any failures are caused by unimplemented features or are they things that used to work and have broken since the test suite was written? As I understand it, the "make check" (or "make test") target should always be run in our daily builds so we get automated warnings. Really, this means that it can only cover implemented features by default. I guess we could write a separate Make target (something like "make check-all") that includes unimplemented stuff,
As for the test framework itself, I have no experience... is there a compelling reason to migrate from CxxTest to another framework? We should really aim to get rid of our embedded copy of CxxTest, however, given that it is outdated and adds to our maintenance burden. See bug #1094771 for the request to use an external CxxTest implementation [1]. Note that we can't really do this until there is a Mac port of the CxxTest suite.
Best wishes,
AV
[1] https://bugs.launchpad.net/inkscape/+bug/1094771
On 16 July 2013 16:33, Jon Cruz <jon@...18...> wrote:
Hi guys,
First off, I was wondering how many people out there have been running the unit tests along with builds.
Secondly, the system we have now probably is in need of serious updating. Some of the test cases themselves aren't passing correctly, but the framework is also outdated.
So does anyone have preferences in this area? Is there some framework we should consider other than Boost Test and Google Test. Or has it not been an issue for others and I should just go ahead and update to whatever keeps things working?
Thanks.
See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clk... _______________________________________________ Inkscape-devel mailing list Inkscape-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/inkscape-devel
See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clk... _______________________________________________ Inkscape-devel mailing list Inkscape-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/inkscape-devel
Here's the log for the Gtk+ 3 build.
AV
On 17 July 2013 11:14, Alex Valavanis <valavanisalex@...400...> wrote:
(I'll upload a similar test log for the Gtk+ 3 build shortly)
On 17 July 2013 11:11, Alex Valavanis <valavanisalex@...400...> wrote:
Here's an updated log from "make check". I suspect that some of these issues are quite easy to fix. As a first step, can anyone comment on...
Which of these are easy/trivial to fix
Which are hard problems to solve
Which (if any) relate to unimplemented features
If we can clean out the errors ASAP, I can set up automated unit testing in the PPA.
AV
On 16 July 2013 17:42, Martin Owens <doctormo@...400...> wrote:
I've been avoiding the test suite as it has had problems any time I've run it with clean trunk.
Other projects I've worked on implement a strict trunk test passing feature that forces all branches to pass their unit testing before being allowed to merge into trunk. Our lack of trunk controls make this harder to do culturally from what I can see. Your thoughts?
Thanks for the log Alex, a lot of those verb failures are actually because the ALIGN verbs were all pushed up by 5 by scripting verbs in the enum but not in the struct. Code was fixed and the enum matches the struct somewhat better, although the test is failing as if it wanted broken code so I'm thinking that verb test needs to be improved.
I don't have any knowledge of C/C++ testing frameworks. I'd build an entire set of python tests and run via dbus and bindings if I were to make a test suite from scratch.
Best Regards, Martin Owens
On Tue, 2013-07-16 at 17:12 +0100, Alex Valavanis wrote:
Output from "make check" on a recent(ish) checkout (r12340) shows numerous test failures (log attached). I had to exclude it from the daily PPA build for this reason.
I'm not familiar with the structure of the tests, but can anyone confirm if any failures are caused by unimplemented features or are they things that used to work and have broken since the test suite was written? As I understand it, the "make check" (or "make test") target should always be run in our daily builds so we get automated warnings. Really, this means that it can only cover implemented features by default. I guess we could write a separate Make target (something like "make check-all") that includes unimplemented stuff,
As for the test framework itself, I have no experience... is there a compelling reason to migrate from CxxTest to another framework? We should really aim to get rid of our embedded copy of CxxTest, however, given that it is outdated and adds to our maintenance burden. See bug #1094771 for the request to use an external CxxTest implementation [1]. Note that we can't really do this until there is a Mac port of the CxxTest suite.
Best wishes,
AV
[1] https://bugs.launchpad.net/inkscape/+bug/1094771
On 16 July 2013 16:33, Jon Cruz <jon@...18...> wrote:
Hi guys,
First off, I was wondering how many people out there have been running the unit tests along with builds.
Secondly, the system we have now probably is in need of serious updating. Some of the test cases themselves aren't passing correctly, but the framework is also outdated.
So does anyone have preferences in this area? Is there some framework we should consider other than Boost Test and Google Test. Or has it not been an issue for others and I should just go ahead and update to whatever keeps things working?
Thanks.
See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clk... _______________________________________________ Inkscape-devel mailing list Inkscape-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/inkscape-devel
See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clk... _______________________________________________ Inkscape-devel mailing list Inkscape-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/inkscape-devel
2013/7/16 Jon Cruz <jon@...18...>:
So does anyone have preferences in this area? Is there some framework we should consider other than Boost Test and Google Test. Or has it not been an issue for others and I should just go ahead and update to whatever keeps things working?
I have no experience with Boost.Test. Google Test is already used in 2Geom and I like it - it's a lot easier to use than CxxTest.
Regards, Krzysztof
I'm used to PHPUnit and JUnit from previous projects and have no experience from C++ testing though after searching and reading up on it Google Test sounds like a good choice.
If it's already in use in 2geom and liked, I'd vote for it just because of that alone.
Cheers, Josh
On Wed, Jul 17, 2013 at 10:39 PM, Krzysztof Kosiński <tweenk.pl@...400...> wrote:
2013/7/16 Jon Cruz <jon@...18...>:
So does anyone have preferences in this area? Is there some framework we should consider other than Boost Test and Google Test. Or has it not been an issue for others and I should just go ahead and update to whatever keeps things working?
I have no experience with Boost.Test. Google Test is already used in 2Geom and I like it - it's a lot easier to use than CxxTest.
Regards, Krzysztof
See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clk... _______________________________________________ Inkscape-devel mailing list Inkscape-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/inkscape-devel
participants (6)
-
Alex Valavanis -
Christoffer Holmstedt -
Jon Cruz -
Josh Andler -
Krzysztof Kosiński -
Martin Owens