Pen Events on the Surface Duo
Hello Android developers!
The Surface family of devices is known for great touch support, including pen input, and the Microsoft Surface Duo will be no different. By default, pen events are treated the same as a finger touch or mouse move, but you can detect and handle pen input differently, with a wider range of data such as pressure and orientation. Some pens also have a button and a virtual eraser, which you can also respond to. In this post I will explain how to read all that data and enable pen support for your Surface Duo apps.
Surface Duo and Input
Due to the nature of Surface Duo, we had to enhance the Android input stack to enable both touch and pen to seamlessly work between the screens. This means that if the user drags an image in one screen, this action is detected and the drag operation will be continued onto the second screen without requiring the user to lift their finger to drag the image to the other screen.
The same goes for pen. When an app is spanned across the two screens, the user can use the pen and draw, and the pen stroke will work between the screens.
Pen events in Android
Android pen events are handled by implementing the onTouchEvent method in your View class. The key to making your app touch interactive is to expand your implementation of View to override the onTouchEvent to listen for touch events. The MotionEvent as an argument reports input details from the touch screen to let users interact with the screen using a pen.
More details on how to respond to the touch event are available in this document.
We can identify that the user is using a pen by looking at the value returned from the MotionEvent.getToolType method. A value of TOOL_TYPE_STYLUS indicates a pen and TOOL_TYPE_ERASER indicates a pen held inverted with the eraser down.
In addition to the usual data we get for input devices, pen devices also set pressure, orientation, and tilt values.
This value represents the pressure the user is using the pen with. It is usually between 0 to 1 (but it is not limited by the system); 0 indicates light pressure and 1 indicates strong pressure.
The getOrientation value is defined in the documentation as follows:
“Returns the orientation of the touch area and tool area in radians clockwise from vertical for the given pointer index (use getPointerId(int) to find the pointer identifier for this index). An angle of 0 radians indicates that the major axis of contact is oriented upwards, is perfectly circular or is of unknown orientation. A positive angle indicates that the major axis of contact is oriented to the right. A negative angle indicates that the major axis of contact is oriented to the left. The full range is from -PI/2 radians (finger pointing fully left) to PI/2 radians (finger pointing fully right).”
Let’s explore further what this means.
Illustration: A pen held in this direction will return an orientation value of –PI/4
The orientation value starts at 0 when the pen is held pointing towards the top of the screen with the eraser part pointing at the user.
When rotating the pen, the values will be negative if the pen points to the left (tip of pen left to the eraser) and positive when pointing to the right.
For example, in the sample app below, I use the orientation to draw a red arc in the direction the pen points with this code:
float orientation = ((event.getOrientation() * 57.2958f) + 90 ) %360 ; canvas.drawArc(oval,orientation,5.0f,true,paint);
Tilt represents the angle between the pen and the screen as described below in the documentation:
“For a stylus, reports the tilt angle of the stylus in radians where 0 radians indicates that the stylus is being held perpendicular to the surface, and PI/2 radians indicates that the stylus is being held flat against the surface.”
Note: the Linux kernel in the core of Android lacks the support for tilt in its multi-touch protocol. For this reason, the Surface Duo does not support this value.
Pen Events Sample
In the Pen Events sample available on GitHub, I collected all the data contained in the sample and visualized it below:
- The green circle represents the point under the pen tip; its size is controlled by the pressure
- The blue circle size represents the pressure; it will turn pink if the button is pressed
- The red triangle represents the orientation
- If an eraser is used, the shapes will become outlines
|Pen with low pressure
|Pen with high pressure
|Pen with the barrel button pressed
|Pen held upside down with the eraser on the screen
Try out the Pen Events sample on GitHub on the Surface Duo emulator and start applying these ideas to your own Android apps! We look forward to seeing your app on the Surface Duo.
We would LOVE to hear from you about your experiences using the SDK, emulator, and your first thoughts on how you can use these in YOUR app.
In case you have not seen these already, we started a series of 1 minute videos to answer the most frequent questions we have received.