I was curious if cairo is in fact slower or faster than our current renderer. I didn't feel like converting us totally to cairo yet, but we have outline mode which is much simpler and perfect for tests like this. So in about 3 hours (of which 2 hours were spent figuring out a known bug in cairo) I converted our outline mode rendering to cairo.
The result of this experiment, after I adjusted the tolerance of cairo to match our outline mode quality, is near parity. That is, cairo is about as fast as our own renderer, at least for outline mode (i.e. lots of paths but no gradients or transparency). However this claim needs to be qualified:
- It is only true for the latest version of cairo (tested with 1.3.14). Versions 1.2.x are noticeably slower.
- This uses image backend to render to a memory buffer, and therefore cannot use any hardware acceleration. Using hardware-accelerated backends will require much more drastic changes in Inkscape's architecture.
- Even with image backend, currently cairo is used _only_ for path rendering, each path going to its own temp buffer, and these buffers are later composed together just as they were before. I suspect that if this is optimized by allowing a single cairo context draw all of the paths in the drawing into a single buffer, performance will improve noticeably.
In any case, since this does not make for any speed regression, and since it fixes at least one bug that our native renderer had in outline mode, I committed this. Please test outline mode thoroughly and report if you find any problems. If everything works fine I'll remove the old outline mode code and will look into doing some further optimizations.
On Fri, Feb 16, 2007 at 10:51:25PM -0400, bulia byak wrote:
I was curious if cairo is in fact slower or faster than our current renderer. I didn't feel like converting us totally to cairo yet, but we have outline mode which is much simpler and perfect for tests like this. So in about 3 hours (of which 2 hours were spent figuring out a known bug in cairo) I converted our outline mode rendering to cairo.
Excellent! This is good news, and I hope a first step towards moving more of our code over.
Bryce
More news on Cairo. I did some slightly deeper rearchitecturing and switched all of the drawing in outline mode to cairo (previously it was only for shapes), making all objects draw into a common cairo surface. Then I added a simple byte-reverser which converts the final buffer from RGBA to ABGR for PPC Macs and other big-endian systems. This had to be done only once per buffer render, so while Macs will render a bit slower because of that, the difference should not be significant. Jon Cruz reports that the byte-reverser works and outline mode displays correctly on again on his Mac.
Approximately the same thing needs to be done for normal mode rendering and then for all the canvas items (knots, handles, etc). This is A LOT more work, of course. Gradients, patterns, clipping, masking, filters - all of this holds innumerable surprises for an intrepid cairoifier. The canvas items are especially fun because all of them render into a 3-byte buffer (RGB without opacity, packed) which has absolutely no counterpart in cairo, and most of them are drawn by manual pixel-flipping via p += 3 pointers...
What mostly motivates me in this work is not speed gains (as I said, they are modest), but the reduction in memory consumption - and it is BIG. I loaded a complex drawing in outline mode in 0.45; top reported 222Mb total virtual memory. Then I loaded the same into current SVN - 118Mb, almost twice less! That's how bad our old renderer is. It not only uses insane amounts of memory but also constantly leaks it (see bug 1460453 for details). If we succeed in ditching the old renderer for the normal mode too, Inkscape will officially stop being a memory hog.
Update: Carl Worth has just told me that there are plans in cairo to support more buffer formats in the near future. Along with the bugfixes, this will make cairo much more attractive for us. Perhaps the switch will not be so much pain after all.
I soon realized that removing at least one intermediate buffer would be easy, so I did it. As a result, outline mode is now about 25% faster than in 0.45, which is quite noticeable.
On 2/17/07, bulia byak <buliabyak@...400...> wrote:
In any case, since this does not make for any speed regression, and since it fixes at least one bug that our native renderer had in outline mode, I committed this. Please test outline mode thoroughly and report if you find any problems. If everything works fine I'll remove the old outline mode code and will look into doing some further optimizations.
I have compiled SVN for the first time since 0.45 today and there appear to be a problem with this new outline mode: nothing is visible in outline mode, tough objects are still selectable. This is with cairo @1.2.6, gtk2 @2.10.9 on Mac OS X. There are no message to the startup terminal. Is there anyway I can get some more useful information?
JiHO
On 2/24/07, jiho <jo.irisson@...400...> wrote:
I have compiled SVN for the first time since 0.45 today and there appear to be a problem with this new outline mode: nothing is visible in outline mode, tough objects are still selectable. This is with cairo @1.2.6, gtk2 @2.10.9 on Mac OS X.
Can you try a newer cairo, ideally 1.3.14?
On 2007-February-25 , at 02:21 , bulia byak wrote:
On 2/24/07, jiho <jo.irisson@...400...> wrote:
I have compiled SVN for the first time since 0.45 today and there appear to be a problem with this new outline mode: nothing is visible in outline mode, tough objects are still selectable. This is with cairo @1.2.6, gtk2 @2.10.9 on Mac OS X.
Can you try a newer cairo, ideally 1.3.14?
Do you mean just fo testing or to use 1.3.14 in the future? If this is for a test I can probably manage to do it. If it is for real worl use I would prefer not to because: - 1.2.6 is the stable release and 1.3.x is the unstable one - it would mean not using a package distribution system to resolve Inkscape dependencies which actually means more trouble for people compiling Inkscape on OS X (unless we switch to Michael build script, but then again, package distribution systems are precisely there to avoid us some trouble in these respects). There are very few of us so it can be seen as a minor annoyance but, in my point of view, it is one more barrier to get _more_ people helping to compile and test development versions of Inkscape on OS X.
JiHO --- http://jo.irisson.free.fr/
On 2/24/07, jiho <jo.irisson@...400...> wrote:
On 2007-February-25 , at 02:21 , bulia byak wrote:
On 2/24/07, jiho <jo.irisson@...400...> wrote:
I have compiled SVN for the first time since 0.45 today and there appear to be a problem with this new outline mode: nothing is visible in outline mode, tough objects are still selectable. This is with cairo @1.2.6, gtk2 @2.10.9 on Mac OS X.
Can you try a newer cairo, ideally 1.3.14?
Do you mean just fo testing or to use 1.3.14 in the future?
Well, of course, start by testing it out. If it doesn't work with 3.14, we'll need to look elsewhere for the problem. If it does, we'll look into how we can move the Mac version to a newer cairo without pain for anyone (perhaps just by waiting until it becomes stable and supported).
Last night, with the help of Carl Worth, the issue of cairo/inkscape not working on mac was more or less clarified. Here's how I understand it.
When drawing into a memory buffer, cairo uses a fixed byte order of the R, G, B, A bytes regardless of the platform.
Inkscape and GDK (which is used by Inkscape for drawing to screen), however, both use system-dependent byte order, i.e. the order of R, G, B, A is determined by the hardware and is different on PC and Mac platforms.
And it so happens that the fixed order of cairo coincides with the hardware order on PC but not on Mac. Hence, painting by cairo directly into Inkscape/GDK buffers worked on PC but on Mac, all colors are wrong and (in case of outline) black color gets zero opacity, i.e. becomes invisible.
Essentially, this means that Inkscape and cairo are not compatible as is, and that a gradual switching of parts of our display engine to cairo is not possible. (Unless we ditch Macs, of course.)
Even if we switch our screen output from using GDK to cairo (which is relatively easy), this won't help - it will just break rendering on PCs too. Inkscape has a lot (and I do mean A LOT) of code that assumes system-dependent byte order. Composers, decomposers, premultipliers, unpremultipliers, blitters and blotters of all sizes and colors permeate the codebase. Most of them come all the way from Sodipodi.
Now, of course, it's possible to create "adapters" for cairo bits. That is, every time we use cairo, instead of drawing directly into Inkscape buffer, we can draw into a temporary buffer instead, rearrange the bytes in that buffer, and then compose that into the main buffer. But that is certain to more than negate the feeble speed gains we're seeing from cairo's image backend rendering.
Or, we can attempt a much more drastic rearchitecturing, by changing our functions so they pass around cairo contexts instead of Inkscape buffers. With this approach, byte reordering will be needed less frequently, and eventually eliminated completely. But this is not only much more work, it will also mean a lot more bugs and regressions and quite some time before Inkscape is usable again.
So, it looks like due to this stupid byte-order issue, cairo for us is practically an all-or-nothing proposition - and much more hassle than I expected. I'm not saying it's impossible but it is quite a major project that cannot be done gradually.
Unless, of course, cairo gets so in love with Inkscape as to provide us with a new image surface with system-dependent byte order...
On 2/24/07, jiho <jo.irisson@...400...> wrote:
I have compiled SVN for the first time since 0.45 today and there appear to be a problem with this new outline mode: nothing is visible in outline mode, tough objects are still selectable. This is with cairo @1.2.6, gtk2 @2.10.9 on Mac OS X.
bulia byak wrote:
So, it looks like due to this stupid byte-order issue, cairo for us is practically an all-or-nothing proposition - and much more hassle than I expected. I'm not saying it's impossible but it is quite a major project that cannot be done gradually.
Is there enough developer support to do this in a non-gradual fashion?
Aaron Spike
On 2/25/07, Aaron Spike <aaron@...749...> wrote:
bulia byak wrote:
So, it looks like due to this stupid byte-order issue, cairo for us is practically an all-or-nothing proposition - and much more hassle than I expected. I'm not saying it's impossible but it is quite a major project that cannot be done gradually.
Is there enough developer support to do this in a non-gradual fashion?
That's what I was trying to measure by sending that email :)
On Sun, Feb 25, 2007 at 01:35:09PM -0400, bulia byak wrote:
On 2/25/07, Aaron Spike <aaron@...749...> wrote:
bulia byak wrote:
So, it looks like due to this stupid byte-order issue, cairo for us is practically an all-or-nothing proposition - and much more hassle than I expected. I'm not saying it's impossible but it is quite a major project that cannot be done gradually.
Is there enough developer support to do this in a non-gradual fashion?
That's what I was trying to measure by sending that email :)
Well, how about this...
We'd been assuming previously that a cairo conversion was going to be a major project already - it was only recently that we got hope that it could be a quick switch. So we had been planning on focusing 0.46 on prepatory refactoring, to get the codebase into a better shape where all-or-nothing switches like cairo could be done a bit easier. With your work there's now a better understanding of the specifics, so maybe it'll be clearer where the refactoring is needed?
I haven't gotten much into the internals of the renderer, so don't know much about how the RGBA ordering is handled (could you point out an example?), but I do know what you mean about rendering code being scattered all over the place.
An initial thought would be - could the RGBA-specific rendering code be better compartmentalized? E.g., instead of being present in 30 different files, reducing that to 5 or 10?
Along with that, could the existing renderer calls be modified to be exact analogs to the cairo calls, to the degree that we'd be able to later just search and replace them with the corresponding cairo calls?
If this is not doable, another thought to incrementalize the mass change would be to incorporate preprocessor macros, and a WITH_CAIRO configure option, that allows us to put the new cairo code while still preserving the old system for those who need to use it. Obviously this incurs some ugliness in the short term.
Of course, a branch is another option. We've done this before, with inkboard, gtkmm, and so on, and know it works. It also has some downsides of re-integration hassles and the risk of not getting sufficient developer attention when it's outside the codebase. However, you *can* work much more swiftly on a branch where you can cause mass breakage to get things converted over. The big price to pay for this approach will be lots of new bugs (offset somewhat by other bugs that will become obsoleted).
Bryce
On 2/25/07, Bryce Harrington <bryce@...961...> wrote:
We'd been assuming previously that a cairo conversion was going to be a major project already - it was only recently that we got hope that it could be a quick switch. So we had been planning on focusing 0.46 on prepatory refactoring, to get the codebase into a better shape where all-or-nothing switches like cairo could be done a bit easier.
Some of the preparatory work I already started doing, such as planning to eliminate non-premultiplied mode (which cairo does not use, yet PNG and on-screen icons must get non-premultiplied data).
Of course some other preparatory work may be needed. But mostly, it's just grunt work. Someone needs to sit down and convert ALL the stuff. As simple as that.
And the main problem is not that it's a lot of work. The main problem is that all this must be done at once, otherwise it won't work. And that means that the number of bugs after this conversion will be bordering on unmanageable. Not only Inkscape bugs; cairo has its own bugs, several of which I discovered already by the outline mode exercise. They are all reported to cairo but I have no idea when they will be fixed.
This cannot be done on the trunk because SVN must be usable for work every day. And it does not seem to me that branches are a successful strategy in general. Work in branches seems rather prone to dying out soon after branching.
I'm also of the opinion that cairo could make this switch much easier for us if they chose to. If only they could provide an image surface which is not premultiplied and another which uses a byte order compatible with ours on all platforms, most of our problems would be eliminated and the switch could be done very gradually, with proper community testing of every step.
If this is not doable, another thought to incrementalize the mass change would be to incorporate preprocessor macros, and a WITH_CAIRO configure option, that allows us to put the new cairo code while still preserving the old system for those who need to use it. Obviously this incurs some ugliness in the short term.
I guess this is what will be done in the short term, at least for the outline mode to be usable on Macs.
bulia byak wrote:
On 2/25/07, Bryce Harrington <bryce@...961...> wrote:
We'd been assuming previously that a cairo conversion was going to be a major project already - it was only recently that we got hope that it could be a quick switch. So we had been planning on focusing 0.46 on prepatory refactoring, to get the codebase into a better shape where all-or-nothing switches like cairo could be done a bit easier.
Some of the preparatory work I already started doing, such as planning to eliminate non-premultiplied mode (which cairo does not use, yet PNG and on-screen icons must get non-premultiplied data).
I would hate to see this go, as it can give nicer results in some cases (it's at least one reason why export often looks better). Of course there would be less need for it if it was possible to use 16bits per color. And if it would be the only thing against Cairo, well, don't let it stop you, but if it would be possible to keep this possibility with Cairo, that would be fabulous.
bulia byak wrote:
This cannot be done on the trunk because SVN must be usable for work every day. And it does not seem to me that branches are a successful strategy in general. Work in branches seems rather prone to dying out soon after branching.
I don't think this is strictly true. Filters were done on a branch, and that branch didn't die until it merged back to trunk. Branches can work, but they need the support and energy of two or more coders and lots of organization.
I'm also of the opinion that cairo could make this switch much easier for us if they chose to. If only they could provide an image surface which is not premultiplied and another which uses a byte order compatible with ours on all platforms, most of our problems would be eliminated and the switch could be done very gradually, with proper community testing of every step.
Asking Cairo to dirty their API to help us along seems like quite a large request. What's stopping these things from being done on our side?
Aaron Spike
On Sun, Feb 25, 2007 at 06:15:35PM -0600, Aaron Spike wrote:
bulia byak wrote:
This cannot be done on the trunk because SVN must be usable for work every day. And it does not seem to me that branches are a successful strategy in general. Work in branches seems rather prone to dying out soon after branching.
I don't think this is strictly true. Filters were done on a branch, and that branch didn't die until it merged back to trunk. Branches can work, but they need the support and energy of two or more coders and lots of organization.
Same with the gtkmm branch - it also got to the point of reintegration with the main codebase. In fact, interest died down only after switching to a refactoring approach.
You could argue that if the work is prone to die out if it's done in a branch, then perhaps there wasn't really sufficient interest in it to begin with. Who's to say that same work wouldn't also die if done in the main codebase. And with a branch if the work fails to reach the point where it can be merged, then at least it isn't going to clutter up the main codebase.
From my own experience, the risk with branches is trying to do too much
in a branch. The longer the branch exists unsynced with main, the harder it gets to merge later on.
Not that I think this work *should* be done as a branch; I bet it can be done with #ifdefs. However, I don't think branches should be dismissed outright. Like any tool they're not a silver bullet but can be very useful in some situations.
Bryce
On 2/25/07, Aaron Spike <aaron@...749...> wrote:
Asking Cairo to dirty their API to help us along seems like quite a large request.
As I understand it, GDK uses the same fixed order as Inkscape. Providing a mode compatible with this very widely used lib, to facilitate migration from it, does not sound like "dirtying" to me.
What's stopping these things from being done on our side?
Nothing, except all the hassle with all-or-nothing conversion which I tried to explain in this thread.
first: wow I did not expect such a deep problem when reporting this. I am glad you found it (I was about to repart that my new tests, with Cairo 1.3.14 as you advised, presented the same problem, obviously...)
On 2007-February-25 , at 17:10 , bulia byak wrote:
Last night, with the help of Carl Worth, the issue of cairo/inkscape not working on mac was more or less clarified. Here's how I understand it.
When drawing into a memory buffer, cairo uses a fixed byte order of the R, G, B, A bytes regardless of the platform.
Inkscape and GDK (which is used by Inkscape for drawing to screen), however, both use system-dependent byte order, i.e. the order of R, G, B, A is determined by the hardware and is different on PC and Mac platforms.
And it so happens that the fixed order of cairo coincides with the hardware order on PC but not on Mac. Hence, painting by cairo directly into Inkscape/GDK buffers worked on PC but on Mac, all colors are wrong and (in case of outline) black color gets zero opacity, i.e. becomes invisible.
Is this a problem in the hardware in terms of the physical pieces in the computer or in the way the hardware is handled by the system, deeply inside? I don't know if this is relevant but then, is there a difference between PowerPC and Intel based Macs (I am on a PPC)? Indeed those new Macs are really more like PCs inside and the difference with PCs is more and more at the system level. In which case the future looks brighter because PPC macs will progresively disappear. Meanwhile the old code would be conserved and wraped under a configure option (like --with-old-rendering) we would use only on PPC macs while Cairoification continues on the rest.
Essentially, this means that Inkscape and cairo are not compatible as is, and that a gradual switching of parts of our display engine to cairo is not possible. (Unless we ditch Macs, of course.)
I hope this won't be the case! Notify me, I'll buy a PC then ;-)
Not sure I really understand the details of the rest so:
[...]
Thank you again for your efficient investigation. Too bad it yielded such a result. I think it deserves a news item on the website though.
JiHO --- http://jo.irisson.free.fr/
On Sun, 25 Feb 2007, jiho wrote:
On 2007-February-25 , at 17:10 , bulia byak wrote:
And it so happens that the fixed order of cairo coincides with the hardware order on PC but not on Mac. Hence, painting by cairo directly into Inkscape/GDK buffers worked on PC but on Mac, all colors are wrong and (in case of outline) black color gets zero opacity, i.e. becomes invisible.
Is this a problem in the hardware in terms of the physical pieces in the computer or in the way the hardware is handled by the system, deeply inside? I don't know if this is relevant but then, is there a difference between PowerPC and Intel based Macs (I am on a PPC)?
Hardware. PPC is big-endian, i386 is little-endian. So yes, the Intel based Macs are little-endian, just like normal PCs and the new cairo outline code works fine.
Cheers, Michael
On 2007-February-26 , at 04:26 , Michael Wybrow wrote:
On Sun, 25 Feb 2007, jiho wrote:
On 2007-February-25 , at 17:10 , bulia byak wrote:
And it so happens that the fixed order of cairo coincides with the hardware order on PC but not on Mac. Hence, painting by cairo directly into Inkscape/GDK buffers worked on PC but on Mac, all colors are wrong and (in case of outline) black color gets zero opacity, i.e. becomes invisible.
Is this a problem in the hardware in terms of the physical pieces in the computer or in the way the hardware is handled by the system, deeply inside? I don't know if this is relevant but then, is there a difference between PowerPC and Intel based Macs (I am on a PPC)?
Hardware. PPC is big-endian, i386 is little-endian. So yes, the Intel based Macs are little-endian, just like normal PCs and the new cairo outline code works fine.
So the future looks indeed brighter! Are there other big-endian machines out there which could be problematic? Unless there is a concern with the fact that hardware order and fixed order are coinciding *by chance alone* on little-endian machines, work may continue as planned while keeping the old code inside ifdefs and having a configure option for big-endian machines for a little while (or even an automatic switch turned on when configure detects that the machine is big-endian). Indeed, PPC machines will probably disappear quite fast, the Mac community being usually a early- adopting one. For example, many software makers already announce that the next version of their software will only run on version 10.5 of the OS (the next version to come) which is a paid upgrade from 10.4 and will be out only one year after 10.4... and no one seems to have concerns with this (well, appart from me that is ;-) ). Therefore Inkscape won't be too much of a bad guy if versions 0.45 or 0.46 were the last to be PPC compatible. Plus, I think I am the only one here left with a PPC machine and will probably ditch it soon for a new i386 one. Whatever I do, I will have less and less time over the next 6 months (because of the end of my PhD). Hence, unless someone else compiles svn snapshots with a PPC machine, any new code probably won't have much testing on this platform (while keeping old code there will be less of a problem).
Just my humble opinion on a matter that overwhelms me...
JiHO --- http://jo.irisson.free.fr/
On Feb 26, 2007, at 12:17 AM, jiho wrote:
Indeed, PPC machines will probably disappear quite fast, the Mac community being usually a early- adopting one. For example, many software makers already announce that the next version of their software will only run on version 10.5 of the OS (the next version to come) which is a paid upgrade from 10.4 and will be out only one year after 10.4... and no one seems to have concerns with this (well, appart from me that is ;-) )
Well, I wouldn't go that far.
In fact, the Mac community also tends to *really* hang onto older hardware. Resale value of older macs seems to be higher than one could account for on pure technical abilities. And the only reason I've left my old G4 behind for this G5 is that the G4 got killed in our house fire. :-)
Another factor countering the 10.5 point is that a much higher percentage of mac owners do buy the OS upgrades. So unlike MS systems, a vendor supporting 10.5 can expect to get a lot of owners with older hardware. I forget off-hand where the last details were, but I just read an article about that in the last week or so.
So the next questions is, do we have an ifdef that filters out only PPC-based Macs? If not, can the autoconfig gurus please help with creating one?
On Feb 26, 2007, at 7:56 AM, bulia byak wrote:
So the next questions is, do we have an ifdef that filters out only PPC-based Macs? If not, can the autoconfig gurus please help with creating one?
We might want to try to hit all big-endian machines, since it sounds like it's purely an endianess problem.
On Mon, Feb 26, 2007 at 09:04:27AM -0800, Jon A. Cruz wrote:
On Feb 26, 2007, at 7:56 AM, bulia byak wrote:
So the next questions is, do we have an ifdef that filters out only PPC-based Macs? If not, can the autoconfig gurus please help with creating one?
We might want to try to hit all big-endian machines, since it sounds like it's purely an endianess problem.
Curiously, there's a third endianness listed in endian.h:
#define __LITTLE_ENDIAN 1234 #define __BIG_ENDIAN 4321 #define __PDP_ENDIAN 3412
Bryce
On Mon, 26 Feb 2007 10:20:59 -0800, Bryce Harrington <bryce@...961...> wrote:
Curiously, there's a third endianness listed in endian.h:
#define __LITTLE_ENDIAN 1234 #define __BIG_ENDIAN 4321 #define __PDP_ENDIAN 3412
Thankfully, PDP-endianness is not an issue on any platforms we target.
-mental
On Mon, 26 Feb 2007, Jon A. Cruz wrote:
On Feb 26, 2007, at 7:56 AM, bulia byak wrote:
So the next questions is, do we have an ifdef that filters out only PPC-based Macs? If not, can the autoconfig gurus please help with creating one?
We might want to try to hit all big-endian machines, since it sounds like it's purely an endianess problem.
Glib gives you this information: you can just test for G_BYTE_ORDER == G_BIG_ENDIAN
Cheers, Michael
On Mon, 26 Feb 2007, bulia byak wrote:
So the next questions is, do we have an ifdef that filters out only PPC-based Macs? If not, can the autoconfig gurus please help with creating one?
But, I guess the real question is: what does GDK do? If GDK doesn't switch based on endian-ness, I think we'll be doing some pretty crazy adjustments to work within the GDK framework.
--Ted
On Mon, Feb 26, 2007 at 11:56:11AM -0400, bulia byak wrote:
So the next questions is, do we have an ifdef that filters out only PPC-based Macs? If not, can the autoconfig gurus please help with creating one?
Is endian.h available on all platforms? This works on Ubuntu:
#include <endian.h> #include <stdio.h>
int main() { #ifdef BIG_ENDIAN printf("This is a big endian machine\n"); #elif LITTLE_ENDIAN printf("This is a little endian machine\n"); #else printf("Neither BIG_ENDIAN nor LITTLE_ENDIAN are defined\n"); #endif return 0; }
Bryce
On Mon, Feb 26, 2007 at 11:56:11AM -0400, bulia byak wrote:
So the next questions is, do we have an ifdef that filters out only PPC-based Macs? If not, can the autoconfig gurus please help with creating one?
Whoops, I misunderstood the defines in endian.h. Here's a correction:
#include <endian.h> #include <stdio.h>
int main() { #if __BYTE_ORDER == __BIG_ENDIAN printf("This is a big endian machine\n"); #elif __BYTE_ORDER == __LITTLE_ENDIAN printf("This is a little endian machine\n"); #else printf("Don't know the endianness of this machine\n"); #endif return 0; }
bulia byak wrote:
So the next questions is, do we have an ifdef that filters out only PPC-based Macs? If not, can the autoconfig gurus please help with creating one?
How about a live test? Have an test like endian_type() that returns BIG_ENDIAN or LITTLE_ENDIAN.
If this is too much of a burden at runtime, then add it as a test during config time.
A good way to check is to look at the header of a Java .class file in a hex editor. Check out the magic number 0xcafebabe as bytes, then words.
Simple test: 1. Set a 32-bit int to some literal like 0x11223344; 2: Test byte[0]. If it is 0x11, it is BIG_ENDIAN, if 0x44, it is LITTLE_ENDIAN.
But is value-shifting really that expensive? Assume a canonical format for ARGB or RGBA, always in the same byte order. Shift and mask to get the fields. A bit more work, but infinitely more portable.
A = (pixel >> 24) & 0xff; R = (pixel >> 16) & 0xff; G = (pixel >> 8) & 0xff; B = (pixel ) & 0xff;
bob
On Mon, 26 Feb 2007 16:29:18 -0600, Bob Jamison <rwjj@...127...> wrote:
How about a live test? Have an test like endian_type() that returns BIG_ENDIAN or LITTLE_ENDIAN.
Um, no.
This is something that can and should be entirely accomodated at compile-time, as it's a feature of the compiler's target architecture.
-mental
On Monday 26 February 2007 23:39:46 MenTaLguY wrote:
On Mon, 26 Feb 2007 16:29:18 -0600, Bob Jamison <rwjj@...127...> wrote:
How about a live test? Have an test like endian_type() that returns BIG_ENDIAN or LITTLE_ENDIAN.
Um, no.
This is something that can and should be entirely accomodated at compile-time, as it's a feature of the compiler's target architecture.
For the cmakers... INCLUDE (${CMAKE_ROOT}/Modules/TestBigEndian.cmake) TEST_BIG_ENDIAN(WORDS_BIGENDIAN)
Craig
On Mon, 26 Feb 2007, jiho wrote:
So the future looks indeed brighter! Are there other big-endian machines out there which could be problematic?
Yes. x86 is the only little-endian architecture that I know of. Well, there's MIPS, but everyone I know of runs it in big-endian mode.
--Ted
Sun, 25 Feb 2007 11:10:16 -0500 "bulia byak" <buliabyak@...400...> kirjoitti:
When drawing into a memory buffer, cairo uses a fixed byte order of the R, G, B, A bytes regardless of the platform.
Inkscape and GDK (which is used by Inkscape for drawing to screen), however, both use system-dependent byte order, i.e. the order of R, G, B, A is determined by the hardware and is different on PC and Mac platforms.
Umh, ok... So this means, that NRPixBlocks have different byte orders depending on system?
Is it just that, that colours are somewhere handled as 32-bit integers and as such, the platform byte order (high endian / low endian) makes a difference? Or is there some still deeper problem here?
And further, how do I properly determine the byte order for rgba images? I am now coding the support for the feBlend filter, and for that it is really important to know, from which byte to read the opacity value ;)
(Also, this really should be documented. As the mode for rgba images is called NR_PIXBLOCK_MODE_R8G8B8A8P, I've always assumed, that the format is R8 G8 B8 A8)
On 2/25/07, Niko Kiirala <niko@...1267...> wrote:
Umh, ok... So this means, that NRPixBlocks have different byte orders depending on system?
Yes, looks like it.
Is it just that, that colours are somewhere handled as 32-bit integers and as such, the platform byte order (high endian / low endian) makes a difference? Or is there some still deeper problem here?
As I understand it, the difference is that Inkscape code reads/writes the 4-byte value by individual components, i.e. as char values to char* pointers. Cairo always does it as uint32 values to uint32* pointers. And these two methods give different results on a Mac.
So, perhaps a more correct description is that, on the contrary, it's Inkscape that always uses the fixed order of r, g, b, and a bytes. Cairo on the other hand relies on uint32 values, and these get written with different byte order depending on endianness of the plarform. Perhaps some cairo expert can chime in to confirm or deny this.
Sorry for the confused explanations. I've never dreamt of being an expert on these ugly topics :)
And further, how do I properly determine the byte order for rgba images? I am now coding the support for the feBlend filter, and for that it is really important to know, from which byte to read the opacity value ;)
I think that so long as you do this by getting individual chars and so long as it works on one platform, it will work everywhere, as our entire codebase confirms. Problems start only when you try to mix in cairo with its uint32s.
(Also, this really should be documented. As the mode for rgba images is called NR_PIXBLOCK_MODE_R8G8B8A8P, I've always assumed, that the format is R8 G8 B8 A8)
Yes, I think that's a correct assumption.
bulia byak wrote:
... So, perhaps a more correct description is that, on the contrary, it's Inkscape that always uses the fixed order of r, g, b, and a bytes. Cairo on the other hand relies on uint32 values, and these get written with different byte order depending on endianness of the plarform. Perhaps some cairo expert can chime in to confirm or deny this.
If that is the case you can ignore my earlier remark about my patch, as it would then be correct in its assumption that the alpha channel is the last byte in a pixel (if there is an alpha channel).
(Also, this really should be documented. As the mode for rgba images is called NR_PIXBLOCK_MODE_R8G8B8A8P, I've always assumed, that the format is R8 G8 B8 A8)
Yes, I think that's a correct assumption.
Now you mention it, that is indeed how nr-compose.cpp handles the components, so I guess that more or less settles it (otherwise there would be rendering bugs pretty much all over the place).
On Feb 25, 2007, at 12:22 PM, bulia byak wrote:
Is it just that, that colours are somewhere handled as 32-bit integers and as such, the platform byte order (high endian / low endian) makes a difference? Or is there some still deeper problem here?
As I understand it, the difference is that Inkscape code reads/writes the 4-byte value by individual components, i.e. as char values to char* pointers. Cairo always does it as uint32 values to uint32* pointers. And these two methods give different results on a Mac.
There is another *very* helpful tool for managing conversions and such.
LittleCMS does buffer conversion in its transform code. A list can be seen in the tutorial:
http://www.littlecms.com/tutorial.txt
The include file "lcms.h" has more.
Anyway, an lcms transformation could be used to achieve the buffer conversions. I would also assume that such transformations that are commonplace would be fairly optimized already. Calling cmsCreateTransform() with the stock sRGB profile for both input and output and the proper two buffer types.
Of course... I would *love* to see this go in at the final step before image data hits the screen. That is the best place for a color management transform, and thus the call could serve two functions at once.
On 2/25/07, Jon A. Cruz <jon@...18...> wrote:
Of course... I would *love* to see this go in at the final step before image data hits the screen. That is the best place for a color management transform, and thus the call could serve two functions at once.
Sounds like a good idea, although it will only solve a small part of our problems. The main problem, summarizing, is that we have a lot of pieces of code that render to buffers, and they all assume Inkscape-style buffers. Changing them to cairo piece-by-piece, as I initially hoped, is therefore impossible.
bulia byak wrote:
Last night, with the help of Carl Worth, the issue of cairo/inkscape not working on mac was more or less clarified. Here's how I understand it.
When drawing into a memory buffer, cairo uses a fixed byte order of the R, G, B, A bytes regardless of the platform.
Inkscape and GDK (which is used by Inkscape for drawing to screen), however, both use system-dependent byte order, i.e. the order of R, G, B, A is determined by the hardware and is different on PC and Mac platforms.
This might cause a problem with a patch I just wrote, as it assumes the alpha channel is the last byte (when using premultiplied alpha, otherwise it doesn't matter), and I imagine there might other places too which assume something about this order. So it might be a good idea to do a thorough overhaul of the code anyway, to make sure everyone is on the same page.
BTW, I would appreciate any pointers on how to properly test for this case without having to do something like cast an array of four bytes to a 32bit integer.
bulia byak wrote:
...
Or, we can attempt a much more drastic rearchitecturing, by changing our functions so they pass around cairo contexts instead of Inkscape buffers. With this approach, byte reordering will be needed less frequently, and eventually eliminated completely. But this is not only much more work, it will also mean a lot more bugs and regressions and quite some time before Inkscape is usable again.
BTW, if you're looking for manpower to help with this, I wouldn't mind helping out, but I'm pretty swamped the next two months (until the end of april). After that I'd be willing to put a reasonable amount of time into it.
bulia byak lamentò:
So, it looks like due to this stupid byte-order issue, cairo for us is practically an all-or-nothing proposition - and much more hassle than I expected. I'm not saying it's impossible but it is quite a major project that cannot be done gradually.
Unless, of course, cairo gets so in love with Inkscape as to provide us with a new image surface with system-dependent byte order...
It should be possible to write a custom image surface which does this in the inkscape codebase without putting it in the cairo library, thus allowing the gradual shift to cairo.
In the worst case we can copy the image backend changing the byte order of the surfaces and use this until the transition is completed, then switching to the official image backend.
jiho wrote:
use I would prefer not to because:
- 1.2.6 is the stable release and 1.3.x is the unstable one
- it would mean not using a package distribution system to resolve
Inkscape dependencies which actually means more trouble for people compiling Inkscape on OS X (unless we switch to Michael build script, but then again, package distribution systems are precisely there to avoid us some trouble in these respects). There are very few of us so it can be seen as a minor annoyance but, in my point of view, it is one more barrier to get _more_ people helping to compile and test development versions of Inkscape on OS X
Inkscape is still <1.0, so trying new things is something that we must do. We really don't even have a version yet.
We only need to think about stability after we have a 1.0+ baseline. The next release will be extra special, because we are going to start worrying about an Inkscape API. How about that? Talk about scary! :-)
Our synchronicity with Cairo is a wonderful thing. We can weed out all of the minuses and share the pluses and help to make each other the best agents in our domains.
bob
participants (13)
-
Aaron Spike -
Bob Jamison -
Bryce Harrington -
bulia byak -
Craig Bradney -
Emanuele Aina -
Jasper van de Gronde -
jiho -
Jon A. Cruz -
MenTaLguY -
Michael Wybrow -
Niko Kiirala -
Ted Gould