Hi Android developers!

While the Surface Duo runs Android apps just like any other Android device, there are some differences in the hardware:

• There is one camera, which can behave as either front-facing or rear-facing depending on how the device is folded. Tapping the “swap cameras” button will change how the image is mirrored to correctly render “selfies”.
• There are two screens, so your camera app might be running at the same time as another app. Because the two apps might have different orientation requirements, your camera view might be rotated and letterboxed by the operating system.

In this post, we will review some dual-screen-specific camera behaviors and present some different options for enhancing camera capture in your apps to work even better on the Surface Duo. The complete source code is available for a Camera sample app, shown here:

Figure 1: Camera sample app

Working with portrait orientation lock

Many camera apps lock orientation – especially social media apps, which often lock the orientation to portrait mode. When your app is running on a single screen and is portrait locked, users will typically have the same experience as any other Android device.

There is one new scenario for orientation-locked apps on the Surface Duo – remaining readable when the device is rotated. For example, when an app is portrait-locked, on a single screen, and the device is in double-landscape mode, the other screen will likely have a landscape-oriented app (or launcher) showing. Orientation may be locked by an entry in the AndroidManifest.xml file:

android:screenOrientation="portrait"

or in code:

setRequestedOrientation(Configuration.ORIENTATION_PORTRAIT); // Java

or

requestedOrientation = ActivityInfo.SCREEN_ORIENTATION_PORTRAIT // Kotlin

For the orientation-locked app to remain in portrait mode, Surface Duo keeps the app’s orientation and creates a “letterbox effect” to preserve the correct aspect ratio.

This is a new situation for many apps, and because the app is now oriented differently to the device, the camera view will be rotated. As a developer, you may want to tweak your code for a better dual-screen experience. There are a few options to consider:

1. Detect the letterboxed state, show a message to the user asking them to rotate or flip the device for a better experience.
2. Detect the letterboxed state and rotate the camera feed to match the device posture.
3. Remove the portrait orientation lock. This would allow the app to rotate freely, but if your UX depends on the portrait orientation, it might not be an option for you.

The sample app includes code for each of these options, explained in more detail below.

1. Show a message.

To recognize when an orientation-locked app has been “letterboxed”, we have created a helper class PortraitLockHelper. This is configured in the activity’s OnCreate method by wiring up a listener and checking for a letterboxed state being detected:

portraitHelper = new PortraitLockHelper(this);
portraitHelper.StateListener = new PortraitLockHelper.PortraitStateListener() {
    @Override
    public void PortraitStateChanged(int state) {
//...
    if((state & PortraitLockHelper.PORTRAIT_STATE_LETTERBOXED_90) > 0 ){
        if(showRotationMessage){
            rotationMessageView.setVisibility(View.VISIBLE);
        }
//...
    }
}

In the Camera sample we use this information to show a message advising the user to rotate the device. This is the simplest solution that preserves the app’s portrait-locked behavior without requiring any other significant code changes.

2. Rotate the camera feed

If you’re interested in better supporting the “letterboxed” state by correcting the orientation of the camera feed, you’d use the same PortraitLockHelper class to recognize that the app has been “letterboxed”. However, instead of showing a message to the user, rotate the camera data stream so it matches the app’s perceived orientation.

For this to happen, you need to make sure you use a TextureView and not a SurfaceView as your preview area.

TextureView allows the developer to transform it, thus enabling the ability to rotate and stretch the content. For example, textureView.setRotation(90) will rotate the feed 90 degrees. And to stretch the feed (to compensate for the change is aspect ratio), the setScaleX and setScaleY methods are used.

In the sample we use the following values:

The sample app also includes buttons to enable or disable the camera feed rotation, which helps demonstrate the rendering difference.

The code to change the rotation setting is in the PortraitStateListener declaration. First it determines the device rotation, and whether the camera feed should be rotated. It then calls the transformText method which calculates and applies the scale and rotation values to suit the device and application settings.

3. Support all orientations with Camera2

If your app supports both portrait and landscape orientation, the Camera2 API can adapt to both. Rather than lock the orientation, your app should allow the view to adapt to both portrait and landscape and adapt the feed accordingly.

The CameraSample contains the code to support orientation changes with a camera view. The code path is similar to rotating the camera feed, except that first it will change the requested orientation (e.g. to allow any orientation):

Java

setRequestedOrientation(ActivityInfo.SCREEN_ORIENTATION_UNSPECIFIED);

It then calls PortraitStateChanged which does the same determination of device state before applying scale and rotation values. Apps that choose not to lock to specific orientations can use this code to create a camera view that works in all orientations.

CameraX

If you’re building a new app or planning to upgrade your camera code, adopting the CameraX API will give you a modern base to build on. Google’s CameraXBasic sample on GitHub runs great on the Surface Duo – but we have two small code tweaks to improve the experience:

AndroidManifest.xml

android:configChanges="orientation|screenLayout|screenSize|smallestScreenSize"

MainActivity.kt

    override fun onWindowFocusChanged(hasFocus: Boolean) {
        super.onWindowFocusChanged(hasFocus)
        if(hasFocus){
            container.postDelayed({
                container.systemUiVisibility = FLAGS_FULLSCREEN
            }, IMMERSIVE_FLAG_TIMEOUT)
        }
    }

These minor updates provide a smoother experience when spanning and unspanning the app, and when focus changes between the camera app and whatever is on the other screen.

Feedback

For existing apps with camera functionality, this post has two different options for how you might enhance the user interface for the Surface Duo, with working sample code. If you’re building a new app, you can base it on the sample using the Camera2 API, or use the Google sampled based on CameraX.

We would love to hear from you about your experiences using the camera in your Surface Duo apps.

Please reach out using the feedback forum or direct message me on Twitter.