Hi all,
This is my first post to the Inkscape developer list, so - hello! I've been using Inkscape for various server-side operations as part of a project I'm currently working on, and I believe I've run into the limits of the existing scripting / extensions capability. However, I'll describe my requirements, problem, current solution, and proposed improvements, and see what you think.
The requirements: I wish to perform "deep subtraction" of one pre-defined path (let's call it P) from an SVG - that is, to perform a Boolean difference operation against each path within the SVG (let's call them S1, S2, ... Sn) such that a 'hole' (in the shape of my path) is punched through all of the SVG layers.
The problem: Inkscape Boolean difference operations operate on pairs of paths, such that the uppermost of the two paths is subtracted from the lower path. The operation may only be performed on selections of two paths (i.e. you can't select 10 paths and have the uppermost path be subtracted from all of the lower 9 simultaneously).
My current solution: The obvious approach is to simply perform the Boolean operation once for each path in the SVG, cloning the path P to allow it to be subtracted again each time. That is:
Create P Perform S1 - P Create P Perform S2 - P ... Create P Perform Sn - P
This results in every path in the SVG having P subtracted, which is exactly what I want. To implement the "Create P" operation, I've written a Python extension that inserts a <path> element into the document (i.e. I can create P by simply executing this extension script within Inkscape). To perform a difference operation, I can use the (somewhat maligned?) command line interface with
--select <id of S1> --select <id of P> --verb SelectionDiff
and I can automatically call my "create P" extension with --verb <my.extension.name.noprefs>.
I therefore just have to ensure all paths in the source SVG have IDs, extract these IDs, and then chain a big long command line together to perform all the creation, selection, and Boolean subtractions. I accomplish the ID extraction with my own Python script (running independently from Inkscape, simply using normal XML parsing code) and get that script to concoct a long string of command-line arguments.
The obvious problem here is that for files with more than a few IDs, I run into the command-line argument length limit. I have hacked around this by editing Inkscape's main.cpp file and allowing it to take --select / --verb arguments from a text file instead (note that --shell does not do the same thing - it still requires all commands to be on one shell-line, and the line length has a limit). Clearly this is not particularly clever, although it does work - it means I have my own special build of Inkscape and it uses an extremely clunky "scripting" mechanism.
The proposed improvements: I have looked at alternative solutions and have come up with the following: - I could edit the C++ code for Boolean operations such that the difference operation works on more than one path. However this is very specific to my problem, and it's likely that I'll want to perform other Inkscape automation tasks in future that don't use the Boolean ops. - There appeared to be some DBus API at one point but it seems to have fizzled out ( http://comments.gmane.org/gmane.comp.graphics.inkscape.devel/40384 was a useful discussion) - and DBus isn't really ideal for this kind of scripting anyway. - Allow further scripting of Inkscape through a Python API - this is my preferred solution, and also, it seems, several other peoples'. The major drawback is that this is likely to be more coding work. However, I have a suggestion that will solve my immediate problem and potentially be useful to others. If I wrote a Python API that allowed Inkscape extension scripts to execute Inkscape verbs and selections, then I could perform all of the necessary operations inside a single Python extension. I could then run the extension from the command line in the usual way (--verb my.extension.name.noprefs). Initially this might take the form of an extra couple of functions on the Python inkex.Effect object: - do_verb(verb_name) - select_ids(id_list)
Then any Inkscape Effect extensions can execute verbs and modify selections as they see fit.
Does this sound like a good plan? Any other input? I'm happy to write the code for this if you think it would be generally useful, it seems a bit limiting at the moment that effects essentially have to modify the raw XML document themselves to get any useful work done. If the "do_verb" function was modified to take an argument list (along with the appropriate C++ functions), then it would allow for even richer GUI-free interaction (e.g. automation of "Save As" operations and so on).
Thanks for reading such a long message,
Eric
It's good to know it will be useful to more than just me!
As far as writing the code goes - I've discovered that there are some potential hurdles to implementing this "properly" due to the way Inkscape extensions currently work, which I believe is as follows: 1) The working document is written to a file 2) The extension script (typically a Python file, but perl, ruby and sh appear to be supported) is executed by calling the appropriate interpreter as a separate process, with selected element IDs and current document provided as input 3) The extension script transforms the SVG and writes it to stdout 4) The extension script reloads the file that the extension wrote to stdout, and replaces its existing document with the newly written one.
If we are to run verbs / change selections on the document, in a vaguely efficient manner, it means that the script would have to either: a) send its results back to Inkscape before each verb execution, get Inkscape to run the verb, and then reload the new version immediately afterwards before the script can continue (slow, but fits the existing async communication model), or b) operate on the actual Inkscape in-memory document itself, meaning that tighter script integration would be needed (e.g. using boost::python) and non-Python scripts would have to be supported differently (e.g. using a Python wrapper that execs the scripts in the existing way, so that all extensions are executed as Python scripts, or by having a different path for Python script execution in script.cpp)
I note that option b) is probably quite a lot of work, and might be some effort to maintain backward compatibility, and adds boost as a dependency (I don't believe Inkscape currently uses boost), while option a) is a bit more inefficient and requires inter-process communication (possibly using the existing file-based mechanism used for communication). The inefficient option a) seems like the one to go for initially, but doesn't provide much scope for future improvements to scripting - embedding boost::python looks like the best way to provide a proper integrated scripting experience. Hence...
c) I could take an "easy option" for proper scripting and implement the boost::python integrated scripting approach, but outside of the existing extension mechanism - this would give the benefits of fast, integrated scripting and a base for future scripting work to develop from, but leaves the existing extension mechanism as a bit of an evolutionary "dead end". Also, it would require adding Boost.Python as an Inkscape dependency. This type of scripting interface would be able to execute existing extensions as verbs, of course.
Thoughts from those who might be able to approve any future merge requests I make? My preference is for option c), but if a Boost dependency would be frowned upon for any reason, then option a) would be next on my list.
Cheers,
Eric
On 26 June 2013 00:07, Mark Schafer <mschafer@...2596...> wrote:
That's a pretty exciting tool for me. I have almost exactly the same problem for one of my extensions - jigsaw - that needs to make hundreds of boolean ops to pairs of paths. The python is great too. So while it's a halfway solution and full python integration would be preferred - IMHO its worth adding this to get the functionality sooner. Cheers...
On Wed, 2013-06-26 at 13:20 +0100, Eric Greveson wrote:
Thoughts from those who might be able to approve any future merge requests I make? My preference is for option c), but if a Boost dependency would be frowned upon for any reason, then option a) would be next on my list.
Option e) have an idx specify it uses dbus and pass the desktop's dbus address and the addresses/ids of selected objects only. Disable script if dbus support not compiled in.
Certainly what I would go for considering the advantages of not having to care too much about the script's language. It could even be compiled C, C++, Vala or Java. (for those that want to make things hard ;-))
Some improvements to the DBus api, good signals etc and it might be worth quite a bit to allowing our extensions more latitude.
Martin,
Hi Martin,
Thanks for the suggestion. This does sound easier to implement, and should integrate nicely with the existing extensions. This approach could solve my particular problem, however, my concerns are:
1) Changes made to the document in the extension are not reflected in Inkscape, so for example, if in my extension I add a path, then attempt to select it and perform some verb on it, it won't work. Inkscape has no knowledge of the path that the extension has added until the extension terminates. Getting this to work properly would require synchronization of the documents before and after each DBus message (i.e. send the document from extension to Inkscape, get Inkscape to replace its document with the one that was sent, perform the DBus action, send the new document from Inkscape back to the extension, resume execution of the extension script). Can this synchronization be performed via DBus as well? Wouldn't it slow down the script considerably? 2) What is the status of DBus on Windows or Mac? Would the scripting improvements be available in standard installs on all operating systems? 3) If it's conditional on DBus being compiled in (which I guess answers my question 2, as DBus might not be available by default on all platforms), this would presumably result in the creation of "second-class citizen" scripts which only run in certain environments - and perhaps limit the appeal of using these features.
I can see the obvious benefits (potentially less coding for me, no Boost dependency) but the fundamental mode of operation of existing extensions (dump Inkscape document SVG, run script to modify document outside of Inkscape, re-import document) makes it difficult to integrate the mixing of existing script functionality (direct DOM changes/transforms) with verb and select operations (which operate on the Inkscape document).
Perhaps I should consider writing my Python script outside of Inkscape, and sending the commands via DBus from this external Python script, as this would seemingly solve my current problem. However, could I get this to work in non-GUI mode? How much overhead is involved (say, if I plan to do ~10000 Boolean operations on a typical SVG containing ~10000 paths)?
Eric
On 26 June 2013 14:58, Martin Owens <doctormo@...400...> wrote:
On Wed, 2013-06-26 at 13:20 +0100, Eric Greveson wrote:
Thoughts from those who might be able to approve any future merge requests I make? My preference is for option c), but if a Boost dependency would be frowned upon for any reason, then option a) would be next on my list.
Option e) have an idx specify it uses dbus and pass the desktop's dbus address and the addresses/ids of selected objects only. Disable script if dbus support not compiled in.
Certainly what I would go for considering the advantages of not having to care too much about the script's language. It could even be compiled C, C++, Vala or Java. (for those that want to make things hard ;-))
Some improvements to the DBus api, good signals etc and it might be worth quite a bit to allowing our extensions more latitude.
Martin,
On Wed, 2013-06-26 at 15:28 +0100, Eric Greveson wrote:
- Changes made to the document in the extension are not reflected in
Inkscape.
Dbus is asyncronous; you wouldn't be working on a local copy of the document, but instead you would be pulling the strings on inkscape's API effecting the document directly. No passing back and forth of xml, no syncing (as such).
- What is the status of DBus on Windows or Mac? Would the scripting
improvements be available in standard installs on all operating systems?
From 2010
http://inkscape.13.x6.nabble.com/Inkscape-amp-Dbus-td2856124.html I share your concern. Although I feel getting solid functionality done to proof it's utility is more important than waiting for platform compatibility. When the scripts demand the support will come IMO.
Perhaps I should consider writing my Python script outside of Inkscape, and sending the commands via DBus from this external Python script, as this would seemingly solve my current problem.
I don't know if Inkscape runs headless, it might (for web-server renderings?). Regardless, it should be a simple process to enable and experiment. I'd love to hear your experiences if you do try the api.
Martin,
Thanks for the quick reply! Thoughts inline:
On 26 June 2013 15:37, Martin Owens <doctormo@...400...> wrote:
On Wed, 2013-06-26 at 15:28 +0100, Eric Greveson wrote:
- Changes made to the document in the extension are not reflected in
Inkscape.
Dbus is asyncronous; you wouldn't be working on a local copy of the document, but instead you would be pulling the strings on inkscape's API effecting the document directly. No passing back and forth of xml, no syncing (as such).
I understand that DBus is asynchronous (which is great), but the existing extensions aren't! My point is that if you decide to make a DBus call half-way through an effect script which also modifies the document XML in the traditional inkex.Effect way, then all of the "changes" that have been made to the XML are still only local changes within the script's in-memory representation of the SVG. The DBus call will then happily be made on the main Inkscape document as it looked before the script started to execute (i.e. the script's changes are not yet known to Inkscape).
- What is the status of DBus on Windows or Mac? Would the scripting
improvements be available in standard installs on all operating systems?
From 2010 http://inkscape.13.x6.nabble.com/Inkscape-amp-Dbus-td2856124.html I share your concern. Although I feel getting solid functionality done to proof it's utility is more important than waiting for platform compatibility. When the scripts demand the support will come IMO.
Interesting. Will bear that in mind.
Perhaps I should consider writing my Python script outside of Inkscape, and sending the commands via DBus from this external Python script, as this would seemingly solve my current problem.
I don't know if Inkscape runs headless, it might (for web-server renderings?). Regardless, it should be a simple process to enable and experiment. I'd love to hear your experiences if you do try the api.
There's a --without-gui (or -z) switch that takes a special codepath in main.cpp (sp_main_console) (this path is also invoked if certain other command-line args are passed to Inkscape). However I'm not sure if it's working properly with verbs: I need to investigate this further (it's possible that I'm doing something wrong). Web-server renderings is exactly my use case! :)
NP, see below.
On Wed, 2013-06-26 at 15:55 +0100, Eric Greveson wrote:
I understand that DBus is asynchronous (which is great), but the existing extensions aren't! My point is that if you decide to make a DBus call half-way through an effect script which also modifies the document XML in the traditional inkex.Effect way.
Yeah. If you're going to do a dbus extension, don't actually put it in as an extension (inx file) until the extensions-system has been told how to deal with dbus based scripts. Otherwise you may end up with the document getting told two different things.
Idea: Perhaps we could develop a similar module to inkey but for the dbus side of it too. More data is needed from experimenters to see if this is worth spending time on.
Martin,
OK, thanks. I think I'll avoid trying to mix the DBus usage with the current extensions system at the moment, for that reason. I'm currently looking at why the verb-execution doesn't work in headless (non-GUI) mode, and it turns out that verbs and selections are tightly-bound to the UI classes (Inkscape::UI::View::View and SPDesktop). This seems unfortunate - I was hoping that those actions that only need to operate on SPDocument instances (with Inkscape::Selection objects) might be decoupled somewhat from the GUI. This appears not to be the case, meaning that my objective of automating Inkscape functionality without a GUI might be harder than I first thought. The current headless operations (e.g. exporting PNG, PDF, query dimensions etc) are all specially hard-coded in main.cpp. Worse, the GUI functionality (upon which the verb actions depend) is all wrapped up in a giant concrete class called SPDesktop, rather than an interface class with a separate implementation. Running verbs and selection operations without a GUI would therefore seem to require the following:
- Changing the action verbs to require some new interface class rather than SPDesktop, which SPDesktop implements. The actions that do genuinely require the full GUI functionality could cast to SPDesktop and return an error (or silently fail, which seems to be the existing policy if null pointers are provided) if a full SPDesktop is not available. - Writing of a GUI-less implementation of the new interface class which is passed to the actions - Doing a similar thing for Inkscape::Selection (so that it doesn't require a full working SPDesktop in the constructor, and will behave nicely in the non-GUI case when one is not available). This looks slightly more straightforward, as the desktop is only used for layer-based information in the Selection class methods, as far as I can see.
There may well be good reasons for the current design where all the UI classes are required to perform any actions on an SVG document - please let me know if this is the case! However, if you think it would be acceptable for me to attempt some changes along these lines, so that actions operate on a document model where possible, and only request a GUI interface when absolutely necessary (e.g. when a dialog box needs to be shown to get parameters from the user), I think it would be a worthwhile change to those wanting to script Inkscape from the command line. I appreciate that this is probably not one of the original goals of Inkscape - however, it appears to be the best freely available software for performing certain vector graphics operations, and the current situation of requiring a GUI for anything other than the simplest format conversion operations seems to be a shame!
On 26 June 2013 16:29, Martin Owens <doctormo@...400...> wrote:
NP, see below.
On Wed, 2013-06-26 at 15:55 +0100, Eric Greveson wrote:
I understand that DBus is asynchronous (which is great), but the existing extensions aren't! My point is that if you decide to make a DBus call half-way through an effect script which also modifies the document XML in the traditional inkex.Effect way.
Yeah. If you're going to do a dbus extension, don't actually put it in as an extension (inx file) until the extensions-system has been told how to deal with dbus based scripts. Otherwise you may end up with the document getting told two different things.
Idea: Perhaps we could develop a similar module to inkey but for the dbus side of it too. More data is needed from experimenters to see if this is worth spending time on.
Martin,
I've made some concrete progress now for improving command-line scriptability of Inkscape. I've made some fairly significant changes to the verb interface, so that now it's possible to execute certain verbs (and make selections via --select) in command-line mode (-z or --without-gui flag). This work involved: - factoring out the layer model from SPDesktop into a new class (Inkscape::LayerModel) - allowing an Inkscape::Selection to take one of these new LayerModels in its constructor, and removing the need for an SPDesktop in Inkscape::Selection (although it's still in the class when in GUI mode due to so many parts of the app using it to get access to the desktop) - editing main.cpp to run the selections / verbs when taking the command-line code path - changing the interface to Verb and SPAction so that a new ActionContext object is provided, rather than just an Inkscape::UI::View::View, providing access to document, selection model and (only in GUI mode) the Inkscape::UI::View::View of old - changing the Boolean operations to work without a SPDesktop, i.e. in command-line mode, although if a GUI is available they will still show error messages in dialog boxes (in console mode, they will print to stderr). - printing error messages on stderr when all other verbs are called from the command line (old behaviour was silent failure)
This now means that you can run Inkscape on the command line with e.g. "inkscape file.svg --select Path1 --select Path2 --verb SelectionDiff -z", and Inkscape will subtract the uppermost path from the lowermost path, and overwrite file.svg with the result, all without ever creating any GUI. This is obviously much faster than before (where the verbs didn't work with -z) and opens the way for other suitable verbs to be modified for console-mode (e.g. server-side) usage, as required.
I'd like to integrate my work with trunk, if you think that my changes are valuable - I'm currently on a branch on my Launchpad account. I was up to date with trunk a few days ago - is it good practice to trunk-track (i.e. merge all changes from trunk to my branch since my branch started) before clicking "propose for merging", or is this not necessary?
Next step: trying to use the DBus interface when running Inkscape in console mode...
On Mon, 2013-07-01 at 22:13 +0100, Eric Greveson wrote:
This now means that you can run Inkscape on the command line with e.g. "inkscape file.svg --select Path1 --select Path2 --verb SelectionDiff -z", and Inkscape will subtract the uppermost path from the lowermost path, and overwrite file.svg with the result, all without ever creating any GUI. This is obviously much faster than before (where the verbs didn't work with -z) and opens the way for other suitable verbs to be modified for console-mode (e.g. server-side) usage, as required.
Eric, I'd like to celebrate your work. This is a very useful feature and I can't wait to see what might be possible with it in the future. *high five*
is it good practice to trunk-track (i.e. merge all changes from trunk to my branch since my branch started) before clicking "propose for merging", or is this not necessary?
It's a requirement. All good merge proposals should be as up to date as possible and conflicts are the responsibility of the proposer if any crop up during review.
Next step: trying to use the DBus interface when running Inkscape in console mode...
That would certainly be interesting, being able to modify svg documents via DBus without an inkscape GUI running.
Warm Regards, Martin Owens
Hi again,
I've done some DBus work now, so that it's possible to run a command-line only instance of Inkscape, connect to it with DBus, and perform operations on the document (e.g. selection of paths, Boolean operations on selected paths, and saving of the document). Also fixed a couple of minor issues with the DBus interface as I came across them (e.g. setting error messages when certain methods fail). I've not written an example app to augment the existing "pytester.py" DBus script, but can do so if it's useful. The only real difference is that you should start Inkscape with "inkscape <file.svg> -z --dbus-listen" and then use the "get_active_document()" DBus method on the "/org/inkscape/application" interface.
I've trunk-tracked, as suggested, and checked that there are no conflicts, so I've created a merge request (with Martin as the reviewer - hope you don't mind!). I tried to follow the various Bazaar workflow email threads, and I think I've understood how to do it properly, but I'd appreciate if you could double-check as this is my first time using Bazaar.
It would be great if this makes it into trunk, if there are problems then please let me know and I'll attempt to rectify them!
On another note, I wonder if the --enable-dbusapi flag might be enabled in future release builds of Inkscape (especially packages on Ubuntu)? While the API is obviously in a state of flux, it's possible to do useful scripting work with it already (and it's a whole lot better than the --verb --select command-line args way, even if that no longer requires a GUI).
Best, Eric
On 1 July 2013 23:37, Martin Owens <doctormo@...400...> wrote:
On Mon, 2013-07-01 at 22:13 +0100, Eric Greveson wrote:
This now means that you can run Inkscape on the command line with e.g. "inkscape file.svg --select Path1 --select Path2 --verb SelectionDiff -z", and Inkscape will subtract the uppermost path from the lowermost path, and overwrite file.svg with the result, all without ever creating any GUI. This is obviously much faster than before (where the verbs didn't work with -z) and opens the way for other suitable verbs to be modified for console-mode (e.g. server-side) usage, as required.
Eric, I'd like to celebrate your work. This is a very useful feature and I can't wait to see what might be possible with it in the future. *high five*
is it good practice to trunk-track (i.e. merge all changes from trunk to my branch since my branch started) before clicking "propose for merging", or is this not necessary?
It's a requirement. All good merge proposals should be as up to date as possible and conflicts are the responsibility of the proposer if any crop up during review.
Next step: trying to use the DBus interface when running Inkscape in console mode...
That would certainly be interesting, being able to modify svg documents via DBus without an inkscape GUI running.
Warm Regards, Martin Owens
On Thu, 2013-07-04 at 00:23 +0100, Eric Greveson wrote:
The only real difference is that you should start Inkscape with "inkscape <file.svg> -z --dbus-listen" and then use the "get_active_document()" DBus method on the "/org/inkscape/application" interface.
Does this mean that a normal gui inkscape won't have dbus enabled?
with Martin as the reviewer - hope you don't mind!
NP, Review done.
On another note, I wonder if the --enable-dbusapi flag might be enabled in future release builds of Inkscape
I would certainly support a move to enable it for Linux/Mac users, or just linux users if that's the easiest for now. With more use, getting it working and packaged correctly for windows would be easier to argue. Where as it's languishing at the moment.
Thanks again Eric for this work. :-)
Best Regards, Martin Owens
On 4 July 2013 03:46, Martin Owens <doctormo@...400...> wrote:
On Thu, 2013-07-04 at 00:23 +0100, Eric Greveson wrote:
The only real difference is that you should start Inkscape with "inkscape <file.svg> -z --dbus-listen" and then use the "get_active_document()" DBus method on the "/org/inkscape/application" interface.
Does this mean that a normal gui inkscape won't have dbus enabled?
A normal GUI inkscape will still have DBus enabled without using the --dbus-listen flag. The flag is only required in console mode because it the app needs to start a main-loop to listen for messages: in normal console mode, the app processes all commands and exits immediately.
NP, Review done.
Thanks! Making changes to the code to address the issues you've highlighted now.
On another note, I wonder if the --enable-dbusapi flag might be enabled in future release builds of Inkscape
I would certainly support a move to enable it for Linux/Mac users, or just linux users if that's the easiest for now. With more use, getting it working and packaged correctly for windows would be easier to argue. Where as it's languishing at the moment.
I still need to test the new code on Windows. Will see if I can get round to doing that today, it would be nice if all platforms supported the same feature set!
Eric
participants (3)
-
Eric Greveson -
Mark Schafer -
Martin Owens