Hello Android developers!

We have some updates to the InkSDK that we would like to share with you, as well as some general guidance on persisting ink and open ink standards.

Overview

While Android provides different canvases for rendering ink, there is little guidance on how to store that ink or what format it should be in when communicating with other apps. Let’s dive into some topics surrounding this concept.

Persisting Ink

Rendering ink on a canvas is important, but how that data is pulled off the canvas and saved is equally as important.

In the previous InkSDK versions, developers have access to a bitmap representation of the canvas, which is useful for sharing drawings with other apps. But what if you want to save the drawing and be able to edit it later? A bitmap does not provide this functionality.

To save and load notes without data or functionality loss, there are a couple of approaches to enable this. These approaches convert a canvas’s ink to some other data type that can be saved locally.

1. Serializing the data into a string using Kotlin or Java serializers (often formatted as JSON)
2. Creating a custom parser that converts the data into a differently structured format
   1. Converting to an open standard type like InkML is more complex, but may have some advantages that will be discussed below

Regardless of the developer’s approach to saving the ink (e.g. saving in a file, saving in a Room database, etc.), oftentimes only saving simple data types is supported. Anything with references to a Context, Activity, or Application are difficult to save without causing memory leaks.

The Brush class introduced in the InkSDK contains all the data needed to persist drawings without any data loss.

Some of the topics covered here, as well as some more info on persisting ink can be found in a discussion in our SDK repo.

Open Ink Standards

Assuming that ink data is available within the application, can we now send it to other apps to be rendered? Not necessarily. To send data between any two apps, they need to both understand the data format. The same holds true with ink.

Just as there are open standards for transferring commonly encountered data like .png for images, there are open standards for ink.

To send ink between apps, both apps need to agree on a specific format. Since copy/pasting ink between apps is still relatively uncommon in Android, many inking apps today opt not to accept ink formats, choosing to only accept bitmaps or screenshots of the ink instead. This approach has some key disadvantages, like the ones discussed on persisting ink using bitmaps.

While a bitmap may preserve what the ink looked like at a given point in time, a lot of data is lost. Data cannot be easily added to a drawing, so editing it becomes impossible. In a basic sense, exporting a drawing to another app as a bitmap declares the drawing as finished, with no room for additional edits.

A different approach is to use an open ink standard like InkML (Ink Markup Language) for sending ink data. InkML is growing in popularity and has the advantage of being accepted by a growing number of web pages and other apps. It has room for additions and deletions, unlike bitmap representations.

By no means is InkML or any other open standard a complete replacement for sending bitmap representations of drawings. Implementing it in your app, however, can help enhance a user’s overall inking experience.

Updates to the InkSDK

Now that we have some context, let’s take a look at some changes that have been made to the InkSDK.

Developers can access ink data from the canvas now, allowing them to save and load data as they see fit. Ink data is formatted as a Brush data class, which packages the necessary information that a single brush stroke may contain.

data class Brush(, 
    val color: Int,
    val strokeWidth: Float,
    val strokeWidthMax: Float,
    val paintHandler: DynamicPaintHandler?,
    val stroke: InputManager.ExtendedStroke
)

All of the fields in the Brush class are common things we’ve seen in the inking canvas (InkView.kt) in the past, but let’s quickly cover their uses.

• color – base color of the brush stroke
• strokeWidth – typical width of the stroke if no additional manipulations are made (like thinning or thickening the line based on pen pressure)
• strokeWidthMax – the maximum width constraint the stroke can be after additional manipulations (like thinning or thickening the line based on pen pressure)
• paintHandler – an optional class that describes custom changes to the brush stroke (e.g. highlighter, rainbow, patterned, etc)
• stroke – the (x,y) coordinates that define the brush stroke’s path across the canvas

Packaging all this information together, we can accurately recreate a line drawn on a canvas. Given an array of Brush objects, we can recreate an entire drawing.

Buttons have also been added to the InkSDK sample app to demonstrate how to save/load Brush data from/to a canvas.

You can import these InkSDK changes into your project:

implementation "com.microsoft.device:ink:1.0.0-alpha5"

How does these changes relate to persistence and open standards?

While the new InkSDK changes do not directly reference InkML in any way, we added the getter and setter methods for Brush data in the canvas with these concepts in mind.

Unlike the bitmaps, the Brush data available from the InkSDK can be converted to InkML or any other open standard (and back) without any data loss.

Circling back to persistence, InkML can be formatted as a string, which can easily be saved in any persistence implementation.

Overall, giving developers access to the Brush data contained in the canvas opens a lot of opportunities for developers to create a rich inking experience in their apps.

Contributing and joining the conversation

As mentioned above, this InkSDK update came directly from a discussion in our SDK GitHub repo. Our team is always happy to work with you on feedback, suggestions, and potential contributions to our resources!

If you have any questions or would like to tell us about your dual-screen applications, use the feedback forum, message us on Twitter @surfaceduodev, or start a discussion in one of our repos.

Finally, please join us every Friday on Twitch at 11am Pacific time to chat about Surface Duo developer topics!