Hi Martin,

thanks for your reply, I realised I was a bit scant on details. First, a basic technical detail: the device is a bluetooth LE device and needs to be explicitly synced, it doesn't stay connected. Any data that is synced off the device is deleted by the firmware, thus local buffering is needed anyway. This buffering is currently handled by the Tuhi daemon, but it's not a permanent storage solution (we automatically delete old drawings).

On Tue, Feb 27, 2018 at 08:28:06AM -0500, Martin Owens wrote:

Hi Peter,

This is an exciting addition.

The way I would think about this problem is "what operating systems do you want to target", if the target is predominately Linux, then the most super-amazing intergration would be a fuse-fs user space mount.

only Linux, because on other platforms Wacom provides applications to extract the data anyway. There is no need for a daemon on other platforms.

IMO a fuse-fs would be overkill. The data is already synced and buffered, simply saving this to a preconfigured folder seems sufficient and gets you 99% of the way there. The fuse FS would only get you the look of a USB stick, at the cost of wrapping everything into a fake fs.

Also, to get the device to sync, you usually have to press the button on the device and that would require some prompting.

You plug the device in and it pops up just like a regular usb stick, but with it's own wacom icon and relivant name. Inside you can offer up raw or converted file formats, it's up to you how complex you want to go. Although offering svg files directly would be double rainbow amazing tbh.

Offering SVG files is trivial, we already do so in our debugging tools (minus things like pressure handling, not sure how do do that). But I just offering SVG means we're losing some of the capabilities: the data we get from the device however can also include stroke timestamps and per-point pressure. The Wacom Android app has a slider where you can replay the drawing and export a partial one. This would be possible with the raw data but not with an SVG unless we shove it into magic attributes somewhere (how??).

That way Inkscape would be able to open up the files as is, no special arrangements. The best part is that the files would work with many-many programs and you get a many-to-many win-win. I highly recommend this. (some good references for fuse mounts can be found in the google docs fs, the android adb fs, the media-fs for phones etc)

We're planning to have an application that provides SVG export etc. as standalone anyway. This can take over the user prompting and (background?) sync anyway, so the files will be available somewhere, ready for the taking :)

Cheers, Peter

It doesn't sound like you're targeting MacOSX or Windows considering you're using DBus. So I hope this is the way you're going to go with it.

Best Regards, Martin Owens

On Wed, 2018-02-21 at 11:38 +1000, Peter Hutterer wrote:

...

Benjamin Tissoires and I have been working on a project called Tuhi that communicates with the wacom smartpad devices and pulls the data off them.

https://github.com/tuhiproject/tuhi/

Devices working so far are the Bamboo Spark, Slate and the Intuos Pro Paper. Smartpad devices aren't traditional graphics tablets [1] but instead have a paper notepad on top of a sensor with a special pen. You draw on the paper, the sensor registers the drawings and stores it in a custom vector- based format in the device. Later, you connect the device over bluetooth to the  host and fetch the drawings from the firmware.

This is the bit that tuhi does, and it exposes the drawings as a simple json-based vector format overa session DBus API. To interact, you basically connect to the dbus daemon, tell it to listen to the device and then get notified about new drawings. Then you can fetch the drawings one-by- one as needed.

Tuhi only has a commandline client for debugging and we don't intend to ship that client, it's for debugging only. Eventually, we may have a standalone application similar to Wacom's Inkspace (yeah, confusing, isn't it?). This app would be a gallery-style application that displays the drawings and exports them to the desired formats. That's a fairly straightforward process.

But right now what we're interested in is whether inkscape has interest in a plugin-style implementation that talks to the tuhi daemon. Or whether there are use-cases where the standalone application is insufficient and some tighter integration is needed. Or other use-case you can think of that we haven't yet.

Any comments appreciated!

Cheers,    Peter

[1] the intuos pro paper is when you plug it in, let's ignore that for now

---

---

Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ Inkscape-devel mailing list Inkscape-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/inkscape-devel

On Tue, Feb 27, 2018 at 06:51:19PM -0500, Martin Owens wrote:

Hi Peter,

...

thanks for your reply, I realised I was a bit scant on details. First, a basic technical detail: the device is a bluetooth LE device and needs to be explicitly synced, it doesn't stay connected. Any data that is synced off the device is deleted by the firmware, thus local buffering is needed anyway. This buffering is currently handled by the Tuhi daemon, but it's not a permanent storage solution (we automatically delete old drawings).

Yes I read the bluetooth part of it after the fact. tbh the whole exercise would probably need to be ux tested anyway, since a lot of these types of things can be quite cumbersom if not.

...

IMO a fuse-fs would be overkill. The data is already synced and buffered, simply saving this to a preconfigured folder seems sufficient and gets you 99% of the way there. The fuse FS would only get you the look of a USB stick, at the cost of wrapping everything into a fake fs.

Not much of a cost considering the UX gains you get to have. i.e. not having a seperate program you have to run in order to do things.

following up from that, I started with the fuse code this morning, just to get a grasp on how things could work. And I almost immediately ran into a bunch of UX issues that make me question the fuse FS approach again.

First, for a device to sync with a host, it needs to be 'registered' first. This includes holding the button down for 6+ seconds until the LED starts blinking, then pressing the button to confirm once the daemon has connected. The tuhi daemon has the required DBus signals so a client can time this correctly and show the required notifications. But in an FS we're pretty much limited to having a README.txt that spells out what you need to do. Less than ideal...

Second, a device may only sync when the button is pressed. While we can dbus-activate the tuhi daemon on the first FS access, we cannot communicate this to the user beyond having a README.txt.

Third, the registration process is a bit whacky and if a user syncs the device with a different application (like the Android app), the device needs to be re-registered because its UUID will change. Again, something we have signals for in the DBus API but not something easy to handle in an FS.

Fourth, this obviously only works when bluetooth is on. If it's off, we need to ask the user what they want to do.

You can get around these by using the org.freedesktop.Notifications API but I'm not sure how useful that's going to be.

Then there is the subject of permanent storage: the device deletes drawings immediately after sync. the tuhi daemon will delete old drawings on restart, so the only ones guaranteed to be available are the ones drawing in the last session. This make background sync risky, the files need to be fetched, converted and saved somewhere in a normal file system for permanence.

Now the fuse fs is little more than a passthrough from the real directory where everything sits, except that it has a terrible UI :)

so yeah, not convinced the fuse approach is a good one for this particular device. as it looks like, we'll need a fair bit of UI around it anyway.

Cheers, Peter

...

Also, to get the device to sync, you usually have to press the button on the device and that would require some prompting.

I see the button as the jumping off point. You press the button, the demon code is already running with bluez or similar and starting syncing up and boots up the fusefs as the results page. Would be cool if I could just open up my laptop, log in and press the physical device button to get access to my files.

...

Offering SVG files is trivial, we already do so in our debugging tools (minus things like pressure handling, not sure how do do that). But I just offering SVG means we're losing some of the capabilities: the data we get from the device however can also include stroke timestamps and per-point pressure. The Wacom Android app has a slider where you can replay the drawing and export a partial one. This would be possible with the raw data but not with an SVG unless we shove it into magic attributes somewhere (how??).

Yeah, stick the timestamp into stroke-time attribute on the path, not ideal since it's out of spec for svg, but with svg on it's knees, it's time to add some custom specs anyway. Might be a fun addition to inkscape later on to add the whole draw forwards/backwards thing and an interesting saved-undo.

The part that concerns me is the pressure information, since we'd just have strokes and have no real format for saving powerstrokes as svg directly. We could convert things to powerstrokes, but there'd have to be a bit of code that could detect that they're not 'filled out' and do that work on load.

...

We're planning to have an application that provides SVG export etc. as standalone anyway. This can take over the user prompting and (background?) sync anyway, so the files will be available somewhere, ready for the taking :)

That's the ticket. Just less interaction and more 'just pop the stuff up' and also less UI work on your side. I suppose you could just make a folder stick some files in it and pop up a nautilus window, but it's not as cool as the device recognision you'd get otherwise.

Thanks for taking the time to fill in a few details.

Best Regards, Martin Owens

On Wed, Feb 28, 2018 at 08:03:03PM -0500, Martin Owens wrote:

On Thu, 2018-03-01 at 09:29 +1000, Peter Hutterer wrote:

...

So yeah, not convinced the fuse approach is a good one for this particular device. as it looks like, we'll need a fair bit of UI around it anyway.

Thank you for trying the idea, that's more effort than I deserved or expected. :-)

no worries, the FS idea sounded interesting enough to knock out a quick client. But given the UI needed, it won't be a very useful client without a lot more work so we might as well go the full length there.

I'm sorely tempted to buy one of these things to hack on it to be honest.

if you do, buy a folio. we don't know if that works yet :)

If you're producing svgs as the end of your process, then you can have inkscape open them. Accommodations can be made if we want Inkscape to be aware of any new attributes you add in to retain useful data.

our file format atm is just a json file with the raw data (x/y/pressure in arrays that are the strokes). we can produce whatever we want from that. I hear the mp3 patent has expired ;)

Cheers, Peter