Hello
Here's my Cairo rendering proposal. I made it public so that all people can comment. http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/tweenk...
I also have some doubts whether cairo-gl surfaces can be created on all platforms. It looks like only EGL and GLX is supported. An additional problem is that GtkGLExt, which I'll need to use to create cairo-gl surfaces for onscreen drawing, doesn't work on OSX. However, there are Quartz surfaces in Cairo, which we might be able to use in the Aqua port.
Regards, Krzysztof
2010/4/7 Alexandre Prokoudine <alexandre.prokoudine@...400...>:
On 4/7/10, Krzysztof Kosiński wrote:
Hello
Here's my Cairo rendering proposal. I made it public so that all people can comment.
Alas, no. Signing in is still required :)
AFAIK you can sign in using your Gmail account. In any case, here's a copy-pasted version
Title: Cairo rendering for Inkscape Student: Krzysztof Kosinski
Inkscape uses its own rendering library, called libnr, when rendering graphics on the screen and exporting bitmaps. This library is slow, single-threaded and does not have any hardware acceleration.
This proposal is about converting Inkscape's rendering subsystem to Cairo. Using the hardware-accelerated cairo-gl backend and OpenCL-based filters will be investigated. The goal is to substantially improve rendering performance, so that very complex drawings can be edited easily.
Description
Rendering in Inkscape is done on two levels. The canvas widget, SPCanvas, is a regular GTK widget, which appears to be derived from the old Gnome Canvas. It is used to render visual editing aids, like the page border and control points. One of its items, NRArena, renders the content of the document. NRArena shows a tree of NRArenaItems by rendering them into garbage collected memory buffers called NRPixBlocks and then painting them onto the Cairo context of the SPCanvas widget.
This project will aim at refactoring the NRArenaItem hierarchy and porting it from NRPixBlock-based drawing to Cairo operations. Shapes will be drawn directly on the target context, or composited using the minimum possible number of intermediate surfaces. Implementation Details
* Rendering tree pruning. Currently there are two levels of pruning done in NRArena. The first is tiling - the canvas is split into several smaller rectangles that are painted independently. This improves interactivity at the cost of some drawing artifacts and slower overall redraw. The second is simple bounding box based pruning, where objects that don't intersect with the drawing area are not drawn. In my project, I will drop tiling (it introduces artifacts, and is likely to degrade performance in hardware accelerated scenarios), and will instead introduce a pruning system based on dirty rectangles. After each change to the document, the dirty rectangle (the minimum area of the screen that needs redrawing) will be computed. The computed rectangle (bounding box) will be passed to the rendering functions, and objects whose bounding boxes don't intersect with the redraw area will be skipped. * Memory management. NRArena uses the Boehm garbage collector. This is sometimes poorly compatible with Cairo. I will introduce an explicit memory management strategy based on smart pointers and/or rigorously specified ownership semantics. * Caching. Initially, no caching of rasterized graphics will be done, but the new NRArena classes will be written in a way that doesn't preclude introducing it later. Caching rasterized data is unlikely to give any performance boost when using hardware acceleration. * Intermediate results. Some SVG features, like masks and filters, require rendering intermediate results for compositing. Image surfaces (kept in main memory) or offscreen native surfaces will be used for this purpose. * Filters. Filter implementations will be converted to use Cairo image surfaces via direct data manipulation. I do not expect to encounter too many problems with this conversion, since the only thing that will change is the way the memory is accessed and possibly the order of channels in image data. * Text. Text rendering will use Pango-Cairo. An effort will be made to access fonts only through Pango. This will enable us to remove the dependency on fontconfig in the Windows port, and use Pango's native Windows fonts support in the future. * Refactoring. Inkscape's 0.49 release cycle will be focused on removing deprecated code and rewriting exisint code for better understandability and correctness. As part of that effort, in addition to NRArena C++ conversion, I will remove as much legacy libnr geometry code as possible by converting it to 2Geom and introducing additional 2Geom functions or classes when necessary. * Hardware acceleration strategy. Cairo provides support for OpenGL-based harware acceleration using the experimental (as of 1.9.6) cairo-gl backend [1]. This appears to be the best option for hardware acceleration. After I finish the Cairo renderer, I will attempt to make it use cairo-gl surfaces when available. My initial research suggests that the best way of achieving this is by using GtkGLExt combined with GLX. * Once cairo-gl rendering is accomplished, I will try to implement one of the filters using OpenCL and its OpenGL interoperability feature [2]. Filter sources will be rendered into OpenGL memory objects, and the output will be computed into another OpenGL memory object using an OpenCL kernel. I have very little knowledge of OpenGL and OpenCL, so I estimate that my chances of completing this particular goal are rather low; therefore I do not include it in the list of deliverables.
Schedule
Before the program begins:
* Read Inkscape's source code. (I'm already doing this.) * Read the available Cairo documentation and run some example programs to get a good grasp of the API. * Read OpenCL and OpenGL documentation to determine whether I have some chances of implementing an OpenCL-based filter. Use the information to influence filter rendering design.
First half of the program. I will be completing my BSc thesis in chemistry, so some of these might spill over into the second half:
* Refactor 2Geom's Path and PathVector classes: Path should use boost::ptr_vector, and PathVector should be renamed to PathSequence and be implicitly shared. Make any necessary changes to the SVGPathSink API. Use them to reimplement 2Geom->Cairo path conversion. * Convert NRArena hierarchy to C++ and implement the memory management strategy. * Convert shapes to Cairo drawing operations. Introduce caching of Cairo resources where feasible. * Implement the backwards compatibility solution for filters.
Second half of the program:
* Complete the conversion to Cairo. * Benchmark the performance of the new renderer and introduce optimizations / additional caching when necessary. * Prototype cairo-gl support. * Prototype an OpenCL filter working either on OpenGL memory objects or Cairo image surfaces. * Write documentation. * Commit to trunk. * Bugfix.
Deliverables
* On-screen rendering uses Cairo. * Rendering performance is improved. * NRArena object hierarchy uses C++. * Documentation of new NRArena APIs. * (Optional) Legacy libnr geometry code is removed. * (Optional) cairo-gl backend is used when available.
References [1] http://www.cairographics.org/news/cairo-1.9.6/ [2] http://developer.amd.com/support/KnowledgeBase/Lists/KnowledgeBase/DispForm....
Regards, Krzysztof
2010/4/8 Krzysztof Kosiński <tweenk.pl@...400...>:
On 4/7/10, Krzysztof Kosiński wrote:
Here's my Cairo rendering proposal. I made it public so that all people can comment.
(linking to archived mail instead of full quoting original message) http://article.gmane.org/gmane.comp.graphics.inkscape.devel/33871
To me this looks like a good approach for a cairo based renderer for inkscape, since I maintain GEGL which could possibly considered an alternative I'll post some thoughts on whether GEGLs rendering model could possibly fit into inkscape. (Note, until the last paragraph I list thing that are similar to what is needed, but at the moment probably would be a much worse option than what you have outlined). GEGL deals with many or most of the concerns of an interactive SVG canvas. And at least the long term it should be an eligible candidate for such SVG rendering (I've probably deleted an old naive SVG -> GEGL graph compiler I had lying around, as well as experiments with stroking SVG paths with soft brushes and variable line widths).
GEGL already does various caching of intermediate rendered surfaces and propagation of dirty rectangles in the compositing graph based on graph re-arrangements/property changes. Rendering is at the moment split into spatial regions that are processed sequentially (work is slowly under way to paralellize this processing of rectangular subregions with threads).
The biggest disadvantages I see with doing something primarily vector focused using GEGL at the moment is that GEGL uses sparse tiled buffers, which are not currently supported cairo, thus additional copies are needed for vector rendering. The GEGL model is also currently avoiding any possible in-place compositing of vectors and thus ends up using more copies. The naive approach to convert an SVG document to a GEGL graph also yielded various resampling artifacts as shapes were rasterized prior to transforms being applied, this is an issue that might be possible to work around by making all nodes in the GEGL compositing graph have/be aware of transformation matrices that apply, I have also entertained the idea of passing bezier paths between nodes, allowing nodes in the graph to simply be 2geom "filters" manipulating vector data similar to how raster data is manipulated.
Replacing the full inkscape rendering with GEGL would probably not be a good idea (long term, _I_ probably want to experiment with some such hybrid non-SVG raster/vector authoring UI, with features such as stroking with soft brushes, variable line widths, non SVG 1.2 filters and more, so perhaps it will more sense at some point in the future). Where GEGL and SVG overlap most is SVG 1.2 Filters, it would make sense to use GeglBuffers ability to wrap other in-memory linear buffers (like cairo image surfaces) and do such raster ops using GEGL.
/Øyvind K.
W dniu 9 kwietnia 2010 16:52 użytkownik Øyvind Kolås <islewind@...400...> napisał:
Replacing the full inkscape rendering with GEGL would probably not be a good idea (long term, _I_ probably want to experiment with some such hybrid non-SVG raster/vector authoring UI, with features such as stroking with soft brushes, variable line widths, non SVG 1.2 filters and more, so perhaps it will more sense at some point in the future). Where GEGL and SVG overlap most is SVG 1.2 Filters, it would make sense to use GeglBuffers ability to wrap other in-memory linear buffers (like cairo image surfaces) and do such raster ops using GEGL.
Thanks for a great comment!
I didn't consider GEGL for filters because I thought it was primarily for raster graphics compositing rather than rasterization, though it could make sense if it implements the necessary operations. However, can GEGL operate on OpenGL memory objects? In the mid-to-long term I want to use the OpenGL based cairo-gl backend on platforms where it's available, and being able to run the filters on the GPU without moving the data to the main memory could give a decent performance boost.
Regards, Krzysztof
Hi Krzysztof,
Let's get it going :) Below are my comments on your plan.
First of all, something you likely already know: in outline mode, all reandering is already by cairo image surfaces. Obviously it ignores the more difficult points of stroking, opacity, transparency, patterns, filters, etc. etc., but at least the basic infrastructure is there, and you can build from it. Please try to contain your refactoring urge, and don't break things that already work, unless you have good reasons :)
* Rendering tree pruning. Currently there are two levels of pruning done in NRArena. The first is tiling - the canvas is split into several smaller rectangles that are painted independently. This improves interactivity at the cost of some drawing artifacts and slower overall redraw. The second is simple bounding box based pruning, where objects that don't intersect with the drawing area are not drawn. In my project, I will drop tiling (it introduces artifacts, and is likely to degrade performance in hardware accelerated scenarios), and will instead introduce a pruning system based on dirty rectangles. After each change to the document, the dirty rectangle (the minimum area of the screen that needs redrawing) will be computed. The computed rectangle (bounding box) will be passed to the rendering functions, and objects whose bounding boxes don't intersect with the redraw area will be skipped.
I think there's confusion here. First, all the tiling is in Canvas layer, not Arena layer, so if you limit your work to arena items, you will not affect tiled rendering at all.
Second, I would like to explain some details on the canvas tiling, so as to prevent confusion of terms. There are two mechanisms that can be called "tiling".
One is that the visible screen is divided into square 16x16 tiles, but it does not mean that each such tile is painted separately. Instead, these tiles are quanta of dirtying the canvas: if you dirty a single pixel, the entire 16x16 tile it belongs to is dirtied. The goal of this is to prevent too small dirtied areas and to accumulate many small dirtying events into a few larger events (because obviously, if you request to dirty a pixel in an already dirtied tile, nothing happens).
The other mechanism divides the area-to-paint into buffers, and paints each buffer completely separately, returning to the main event loop after each buffer so that redraw can be interrupted. The size of these buffers depends on mode: 256K bytes in normal mode and 1M in outline mode (these are the values I found optimal after testing; divide by 4 to get the number of pixels in a buffer). These buffers divide the area to paint into strips along the longer side of the area to paint (i.e. horizontal if the area is mode wide than tall), and the width of these strips is calculated from the size of the area and the size of the buffer; these strips are painted starting from the one in which the mouse cursor is, and spreading outwards from it. Buffer boundaries have no coordination with 16x16 tiles, but they just go until interrupted or until covering all dirtied area.
All this is on top of the simple "don't render this object if it's outside the currently painted buffer" logic which is indeed in arena items that you will have to work with in your project. Both these mechanisms - canvas tiles and canvas buffers - have their purposes in life, and both have been tweaked for performance extensively. Please understand very well how they work and don't try to change or abandon them unless you have a _very_ clear idea of what and why you're doing :)
* Memory management. NRArena uses the Boehm garbage collector. This is sometimes poorly compatible with Cairo. I will introduce an explicit memory management strategy based on smart pointers and/or rigorously specified ownership semantics.
I think I would much prefer a rigorous ownership semantics option. Arena items live easily definable lives and I don't really see why anything more complex might have been necessary. One issue I recall is that arena items hold pointers to the same SPStyle objects as their parent SPObjects, which I think was a problem resulting in memory leaks and other bugs (please contact Mentalguy for more on that, he's the best person to know about all that).
* Caching. Initially, no caching of rasterized graphics will be done, but the new NRArena classes will be written in a way that doesn't preclude introducing it later. Caching rasterized data is unlikely to give any performance boost when using hardware acceleration.
I think caching, if done properly, will give a huge interactivity gain. Right now, when you move a canvas item such as a handle or node, objects underneath it have to be redrawn (within the touched 16x16 tiles) starting from scratch - tesselating, stroking, filling, etc., even if they didn't change, which is one of the major contributors to the "feeling of slow". If these redraws can be pulled from a cache, it will feel much more responsive I think.
* Intermediate results. Some SVG features, like masks and filters, require rendering intermediate results for compositing. Image surfaces (kept in main memory) or offscreen native surfaces will be used for this purpose.
The problem with image surface is that it's never going to be really fast - it's just software rendering. And a problem with offscreen GL surface, as you said, is that it may not be available everywhere. Please weigh these alternatives carefully before committing to one of them. It might help to contact Cairo people for their insights and future plans.
* Filters. Filter implementations will be converted to use Cairo image surfaces via direct data manipulation. I do not expect to encounter too many problems with this conversion, since the only thing that will change is the way the memory is accessed and possibly the order of channels in image data.
Yes, I think if you properly redesign NRPixBuffers as adaptors for cairo surface buffers, there will be very little, if anything at all, to change in the filter code.
* Text. Text rendering will use Pango-Cairo. An effort will be made to access fonts only through Pango. This will enable us to remove the dependency on fontconfig in the Windows port, and use Pango's native Windows fonts support in the future.
Text rendering in outline mode is already done by cairo - it gets pathvector for the glyphs and renders it as path. PLEASE keep this approach and try to change as little as possible in text and font code: this is outside the scope of the project and is very sensitive area where a lot of testing is needed. Also, this approach should make it easier to deal with paths and text uniformly when it comes to stroking, gradients, etc., which is what SVG requires.
* Refactoring. Inkscape's 0.49 release cycle will be focused on removing deprecated code and rewriting exisint code for better understandability and correctness. As part of that effort, in addition to NRArena C++ conversion, I will remove as much legacy libnr geometry code as possible by converting it to 2Geom and introducing additional 2Geom functions or classes when necessary.
Changing linbr to 2geom should be OK, but please to be as careful as possible with other code :)
* Hardware acceleration strategy. Cairo provides support for OpenGL-based harware acceleration using the experimental (as of 1.9.6) cairo-gl backend [1]. This appears to be the best option for hardware acceleration. After I finish the Cairo renderer, I will attempt to make it use cairo-gl surfaces when available. My initial research suggests that the best way of achieving this is by using GtkGLExt combined with GLX.
Here I'm not knowledgeable to advise anything, but please try to contact cairo guys for deeper perspectives on this.
One final note: please try to keep both new and old rendering functioning together for as long as possible, with an easy way to switch between them (ideally without restart), so that the new renderer can be quickly checked for correctness and speed. This is a critical point in ensuring a smooth transition! Also, this will be another reason to limit refactoring to the necessary minimum, so as to not disable the old rendering code unless absolutely unavoidable.
On Apr 16, 2010, at 8:12 AM, bulia byak wrote:
- Caching. Initially, no caching of rasterized graphics will be
done, but the new NRArena classes will be written in a way that doesn't preclude introducing it later. Caching rasterized data is unlikely to give any performance boost when using hardware acceleration.
I think caching, if done properly, will give a huge interactivity gain. Right now, when you move a canvas item such as a handle or node, objects underneath it have to be redrawn (within the touched 16x16 tiles) starting from scratch - tesselating, stroking, filling, etc., even if they didn't change, which is one of the major contributors to the "feeling of slow". If these redraws can be pulled from a cache, it will feel much more responsive I think.
This is one area where a different implementation can be included to help significantly with the responsiveness of the program. That is, many applications will switch to some form of proxy representation during user interaction. For example, they might render a vector item into a bitmap cache and switch to drawing that bitmap during drags, then switch back to normal rendering once the user releases an item.
So we probably should keep a look out for opportunities to leverage caching and alternative rendering for interactivity.
On Apr 16, 2010, at 8:12 AM, bulia byak wrote:
- Memory management. NRArena uses the Boehm garbage collector.
This is sometimes poorly compatible with Cairo. I will introduce an explicit memory management strategy based on smart pointers and/or rigorously specified ownership semantics.
I think I would much prefer a rigorous ownership semantics option. Arena items live easily definable lives and I don't really see why anything more complex might have been necessary. One issue I recall is that arena items hold pointers to the same SPStyle objects as their parent SPObjects, which I think was a problem resulting in memory leaks and other bugs (please contact Mentalguy for more on that, he's the best person to know about all that).
I would definitely second this point on the memory management.
Going with smart pointers really tends to lead to almost the same complications we have now with Boehm and its use. Since care is going to be required anyway in order to get all things working, and well, a little extra care in specifying ownership semantics will give us better safety while also helping improve performance. And specifically in the Inkscape codebase, switching from "magic" memory handling to consistent and explicit ownership has been seen to be a bigger gain.
On Fri, 2010-04-16 at 08:47 -0700, Jon Cruz wrote:
Going with smart pointers really tends to lead to almost the same complications we have now with Boehm and its use. Since care is going to be required anyway in order to get all things working, and well, a little extra care in specifying ownership semantics will give us better safety while also helping improve performance. And specifically in the Inkscape codebase, switching from "magic" memory handling to consistent and explicit ownership has been seen to be a bigger gain.
I'm a bit disillusioned with boehm at this point myself. If we could have used it consistently it still might have been a win, but there turned out to be an awful lot of work to get there from here and I don't realistically expect it to get done at this point.
Arena items have pretty straightforward lifecycles, also, so there's probably no reason to make things more complicated than they need to be.
But we do need to get rid of the references from arena items to SPStyle, badly. Those make everything tremendously hard. I think I removed many of them, but the main area where they are still present (last I looked ... it's been a while) is for text/font properties. Since there are so many of them, simply copying attribute values over probably isn't a solution by itself.
-mental
W dniu 16 kwietnia 2010 17:12 użytkownik bulia byak <buliabyak@...400...> napisał:
One is that the visible screen is divided into square 16x16 tiles, but it does not mean that each such tile is painted separately. Instead, these tiles are quanta of dirtying the canvas: if you dirty a single pixel, the entire 16x16 tile it belongs to is dirtied. The goal of this is to prevent too small dirtied areas and to accumulate many small dirtying events into a few larger events (because obviously, if you request to dirty a pixel in an already dirtied tile, nothing happens).
Is there any speed advantage over just accumulating the dirty events into one Geom::Rect?
The other mechanism divides the area-to-paint into buffers, and paints each buffer completely separately, returning to the main event loop after each buffer so that redraw can be interrupted. The size of these buffers depends on mode: 256K bytes in normal mode and 1M in outline mode (these are the values I found optimal after testing; divide by 4 to get the number of pixels in a buffer). These buffers divide the area to paint into strips along the longer side of the area to paint (i.e. horizontal if the area is mode wide than tall), and the width of these strips is calculated from the size of the area and the size of the buffer; these strips are painted starting from the one in which the mouse cursor is, and spreading outwards from it. Buffer boundaries have no coordination with 16x16 tiles, but they just go until interrupted or until covering all dirtied area.
This kind of tiling will not speed up HW accelerated rendering, because in practice the speed of HW accelerated 2D drawing does not depend on the area redrawn but on the number of commands issued. Tiling the image into 16 parts and redrawing all of them will be, in the worst case (when all objects intersect all tiles), 16x slower than drawing everything in one go. For software rendering, the current aggresive tiling strategy is probably better, as the time taken is more dependent on the number of pixels rendered. So I think the best idea is to leave the tiling system intact but provide for opportunities to bypass it.
Another thing, instead of allocating 256K buffers on demand, it might be better to create one big Cairo surface, and then split it into chunks using cairo_surface_create_for_region. This could simplify multithreaded software rendering, as threads won't need to allocate any memory for the output, but I don't see this function anywhere in the public API of Cairo.
I think caching, if done properly, will give a huge interactivity gain. Right now, when you move a canvas item such as a handle or node, objects underneath it have to be redrawn (within the touched 16x16 tiles) starting from scratch - tesselating, stroking, filling, etc., even if they didn't change, which is one of the major contributors to the "feeling of slow". If these redraws can be pulled from a cache, it will feel much more responsive I think.
For dragging, the first optimization that comes to my mind is as follows 1. We determine whether all the dragged or modified objects are adjacent in the z-order 2. If they aren't we skip this optimization as it's going to be too complex 3. We render the canvas into 3 layers: below the dragged object(s), above it and a layer with the object(s) itself 4. On every drag we redraw only the portion with the affected object and composite the output from those three layers
I think this might not work when filters come into the picture though.
The problem with image surface is that it's never going to be really fast - it's just software rendering. And a problem with offscreen GL surface, as you said, is that it may not be available everywhere.
Obviously we need both. :)
One final note: please try to keep both new and old rendering functioning together for as long as possible, with an easy way to switch between them (ideally without restart), so that the new renderer can be quickly checked for correctness and speed. This is a critical point in ensuring a smooth transition! Also, this will be another reason to limit refactoring to the necessary minimum, so as to not disable the old rendering code unless absolutely unavoidable.
I am afraid this might go the way of the failed gtkmm rewrite attempt, and could limit my options. I would need to duplicate the display and libnr directories to do anything meaningful. Isn't it enough to work in a branch?
Regards, Krzysztof
On Fri, 2010-04-23 at 00:29 +0200, Krzysztof Kosiński wrote:
Another thing, instead of allocating 256K buffers on demand, it might be better to create one big Cairo surface, and then split it into chunks using cairo_surface_create_for_region. This could simplify multithreaded software rendering, as threads won't need to allocate any memory for the output, but I don't see this function anywhere in the public API of Cairo.
Don't hesitate to ask on the Cairo list about this. They tend to be pretty responsive and they may also have recommendations to eek out the most performance.
Cheers, Josh
On 22 Apr 2010, at 23:29, Krzysztof Kosiński <tweenk.pl@...400...> wrote:
W dniu 16 kwietnia 2010 17:12 użytkownik bulia byak <buliabyak@...400...> napisał:
I think caching, if done properly, will give a huge interactivity gain. Right now, when you move a canvas item such as a handle or node, objects underneath it have to be redrawn (within the touched 16x16 tiles) starting from scratch - tesselating, stroking, filling, etc., even if they didn't change, which is one of the major contributors to the "feeling of slow". If these redraws can be pulled from a cache, it will feel much more responsive I think.
For dragging, the first optimization that comes to my mind is as follows
- We determine whether all the dragged or modified objects are
adjacent in the z-order 2. If they aren't we skip this optimization as it's going to be too complex 3. We render the canvas into 3 layers: below the dragged object(s), above it and a layer with the object(s) itself 4. On every drag we redraw only the portion with the affected object and composite the output from those three layers
I think this might not work when filters come into the picture though.
I know of several cad/cae programs that when your in a graphics intensive shaded view or similar render simpler versions whilst dragging, so having the Option to ignore filters whilst dragging may be of benefit to us. maybe render something that gives a feel of boundaries so you can position, but don't render them properly. (or just use a much cruder filter representation)
The problem with image surface is that it's never going to be really fast - it's just software rendering. And a problem with offscreen GL surface, as you said, is that it may not be available everywhere.
Obviously we need both. :)
One final note: please try to keep both new and old rendering functioning together for as long as possible, with an easy way to switch between them (ideally without restart), so that the new renderer can be quickly checked for correctness and speed. This is a critical point in ensuring a smooth transition! Also, this will be another reason to limit refactoring to the necessary minimum, so as to not disable the old rendering code unless absolutely unavoidable.
I am afraid this might go the way of the failed gtkmm rewrite attempt, and could limit my options. I would need to duplicate the display and libnr directories to do anything meaningful. Isn't it enough to work in a branch?
Regards, Krzysztof
Inkscape-devel mailing list Inkscape-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/inkscape-devel
2010/4/22 Krzysztof Kosiński <tweenk.pl@...400...>:
Is there any speed advantage over just accumulating the dirty events into one Geom::Rect?
There may be none. We need to test that, but let's not assume automatically that it's better :)
This kind of tiling will not speed up HW accelerated rendering, because in practice the speed of HW accelerated 2D drawing does not depend on the area redrawn but on the number of commands issued. Tiling the image into 16 parts and redrawing all of them will be, in the worst case (when all objects intersect all tiles), 16x slower than drawing everything in one go. For software rendering, the current aggresive tiling strategy is probably better, as the time taken is more dependent on the number of pixels rendered. So I think the best idea is to leave the tiling system intact but provide for opportunities to bypass it.
As I explained, the goal of this is not overall speed increase, but finding a way to insert interruption points during rendering, even if the overall render time grows. If, as you say, accelerated rendering depends on the number of command and not on the area to paint (which may be true to an extent, but I doubt it is literally true), then we'll need to break into chunks the stream of commands, not the area to paint, so that interrupting would be possible with all of the area but part of the objects painted, not part of the area with all of the objects as now.
Another thing, instead of allocating 256K buffers on demand, it might be better to create one big Cairo surface, and then split it into chunks using cairo_surface_create_for_region. This could simplify multithreaded software rendering, as threads won't need to allocate any memory for the output, but I don't see this function anywhere in the public API of Cairo.
I second the suggestion to contact Cairo developers on this matter.
For dragging, the first optimization that comes to my mind is as follows
- We determine whether all the dragged or modified objects are
adjacent in the z-order 2. If they aren't we skip this optimization as it's going to be too complex 3. We render the canvas into 3 layers: below the dragged object(s), above it and a layer with the object(s) itself 4. On every drag we redraw only the portion with the affected object and composite the output from those three layers
I think this might not work when filters come into the picture though.
You seem to refer to dragging SVG objects, but I was referring to dragging canvas items over drawing, such as nodes or handles. They are always on top of the drawing, and for them a much simpler solution is possible where each canvas item remembers what was under it the last time it was repainted, and restores that when it is moved.
One final note: please try to keep both new and old rendering functioning together for as long as possible, with an easy way to switch between them (ideally without restart), so that the new renderer can be quickly checked for correctness and speed. This is a critical point in ensuring a smooth transition! Also, this will be another reason to limit refactoring to the necessary minimum, so as to not disable the old rendering code unless absolutely unavoidable.
I am afraid this might go the way of the failed gtkmm rewrite attempt, and could limit my options. I would need to duplicate the display and libnr directories to do anything meaningful. Isn't it enough to work in a branch?
I would not insist on that, but I know from experience that rendering artefacts and glitches may be very hard to reproduce if you have to start a new copy of the program for that, as opposed to hitting a single key to redraw with a different renderer.
W dniu 23 kwietnia 2010 19:41 użytkownik bulia byak <buliabyak@...400...> napisał:
I would not insist on that, but I know from experience that rendering artefacts and glitches may be very hard to reproduce if you have to start a new copy of the program for that, as opposed to hitting a single key to redraw with a different renderer.
If rendering glitches are the problem, I would rather look towards better integrating the rendering test suite and expanding it as needed, instead of providing the ability to switch the renderer.
Regards, Krzysztof
2010/4/24 Krzysztof Kosiński <tweenk.pl@...400...>:
If rendering glitches are the problem, I would rather look towards better integrating the rendering test suite and expanding it as needed, instead of providing the ability to switch the renderer.
Tests are good, but they will never catch problems with rendering interruption and other interactive aspects.
W dniu 24 kwietnia 2010 14:51 użytkownik bulia byak <buliabyak@...400...> napisał:
Tests are good, but they will never catch problems with rendering interruption and other interactive aspects.
If interruption is a problem, maybe we need to design the renderer in a way that allows it to throw away the results at any point in time and start working on something else?
For me an important goal would be to to never show a partially redrawn screen. The kind of artifacts that tiled redraw introduces aren't nice. By the way, when dragging, we force the redraw to complete every 5 motion events; instead of doing that we could interrupt the drawing at most 5 times during a drag and then force a redraw.
Regards, Krzysztof
2010/4/24 Krzysztof Kosiński <tweenk.pl@...400...>:
W dniu 24 kwietnia 2010 14:51 użytkownik bulia byak <buliabyak@...400...> napisał:
Tests are good, but they will never catch problems with rendering interruption and other interactive aspects.
If interruption is a problem, maybe we need to design the renderer in a way that allows it to throw away the results at any point in time and start working on something else?
That's the ideal. In practice, "any point in time" will in any case be limited to a limited number of code junctions, and if we space them too densely, we will lose too much time on the constant checks "do we have an interrupting event?" So we need to find a compromise between granularity of interruption points and responsiveness to events.
For me an important goal would be to to never show a partially redrawn screen. The kind of artifacts that tiled redraw introduces aren't nice.
I don't think this goal is realistic. There _always_ will be documents that render for minutes, and failure to interrupt this rendering is much worse, usability-wise, than temporary artefacts that the next redraw will remove anyway.
So, most vector editors render in parts, and allow you to interrupt rendering between the parts. There are two main approaches here. Illustrator renders by objects, and so you can never see a partially rendered object, but you can see a drawing that's missing part of its objects. The other approach is that we use: we never show an incomplete stack of objects at any point, but we show a partially repainted area.
To me, our approach is preferable, because it allows me to more easily perceive artifacts as artifacts - it sends a clear visual signal that what you see is partial rendering, and you have no problems to guess which part of it is rendered and which is not. With Illustrator on a slow machine, on the other hand, it is rather disconcerting to watch as more and more objects pop up in unpredictable places (depending on z-order) and to think, is this all? is it finished now? or will one more object jump one me and cover what I'm seeing?
Also, in Illustrator's approach, the maximum time between interruption points depends on the complexity of a single object and so can more easily run out of control, especially where blur is involved - in which zooming stretches the blur radius, and speed of blur sharply depends on radius. For example in Xara, which has a very fast renderer but Illustrator's per-object strategy, viewing an object with large blurred shadow in zoom-in may make it irresponsible for long long seconds. In Inkscape, it is never as bad as that, because rendering blur in a narrow strip is much faster than in the entire screen (and screen resolutions keep growing, making this problem worse all the time). Also, Xara considers a single object a stroke which uses a "brush" composed of multiple objects scattered along a path, and strives to render it all at once - as a result, when my kids tried to make a not very complex drawing with such strokes, it quickly grew unusably slow.
Definitely, both methods are wasteful. With our method, we have to tesselate, stroke etc. each touched path for each buffer. With Illustrator, entire screen may have to be repainted multiple times even if these repaints will be covered by a top opaque object. In both cases, various smart strategies can be invented to reduce this waste.
So, it may be that something - like the hardware-accelerated rendering - forces us to switch to Illustrator repaint method. But in any case please don't do this switch without careful consideration and testing. Our method has served us very well, and what you call "artefacts" are in fact, after you get used to them, useful hints that enable you to work much faster than what our slow renderer would alow otherwise.
By the way, when dragging, we force the redraw to complete every 5 motion events; instead of doing that we could interrupt the drawing at most 5 times during a drag and then force a redraw.
I think this is exactly what it does now. The method is called "force_redraw_after_interruptions"; it counts the number of times render was interrupted, and does not directly depend on motion events.
bulia byak wrote:
... So, most vector editors render in parts, and allow you to interrupt rendering between the parts. There are two main approaches here. Illustrator renders by objects, and so you can never see a partially rendered object, but you can see a drawing that's missing part of its objects. The other approach is that we use: we never show an incomplete stack of objects at any point, but we show a partially repainted area.
To me, our approach is preferable, because it allows me to more easily perceive artifacts as artifacts - it sends a clear visual signal that what you see is partial rendering, and you have no problems to guess which part of it is rendered and which is not. With Illustrator on a slow machine, on the other hand, it is rather disconcerting to watch as more and more objects pop up in unpredictable places (depending on z-order) and to think, is this all? is it finished now? or will one more object jump one me and cover what I'm seeing?
Illustrator's method indeed doesn't sound nice, but perhaps we could consider showing objects that still need to rendered as outlines? (For example.) Also, currently we draw from bottom to top (z-order), but in almost all cases you can just as easily draw top to bottom, which could help in giving people a more useful image much sooner, as well as lessen the sensation of shapes "popping up", as any object that is drawn will not be covered afterwards.
2010/4/26 Jasper van de Gronde <th.v.d.gronde@...528...>:
Illustrator's method indeed doesn't sound nice, but perhaps we could consider showing objects that still need to rendered as outlines? (For example.)
But then, when it's time to render it normally, you will have to delete the outlines first (remember, all objects may have transparency, so leaving the outline and just "covering" it with normal rendering will not work), which means partial rerendering of the objects below where they are touched by the outline. I'm afraid this will slow it all down considerably and add confusion.
Also, currently we draw from bottom to top (z-order), but in almost all cases you can just as easily draw top to bottom, which could help in giving people a more useful image much sooner, as well as lessen the sensation of shapes "popping up", as any object that is drawn will not be covered afterwards.
You cannot do this when transparent objects are stacked, because adding something to the bottom means you have to re-compose (not necessarily re-render if you have them cached, but even then, composing of transparencies also takes time) everything on top of it. Since transparencies are not quite rare in vector graphics, I think this approach will slow it down too much, not to mention it is somewhat counter-intuitive.
participants (9)
-
Alexandre Prokoudine -
bulia byak -
Jasper van de Gronde -
John Cliff -
Jon Cruz -
Joshua A. Andler -
Krzysztof Kosiński -
MenTaLguY -
Øyvind Kolås