{"id":1677,"date":"2021-07-01T11:06:17","date_gmt":"2021-07-01T18:06:17","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/surface-duo\/?p=1677"},"modified":"2021-07-01T11:06:17","modified_gmt":"2021-07-01T18:06:17","slug":"exoplayer-video-foldable-dual-screen","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/surface-duo\/exoplayer-video-foldable-dual-screen\/","title":{"rendered":"ExoPlayer video on dual-screen and foldable devices"},"content":{"rendered":"

\n Hello Android developers!\n<\/p>\n

\n The Microsoft Surface Duo is a great device for consuming multimedia content, and ExoPlayer is a simple, powerful widget for rendering video. Raymond and I are interns on the Surface Duo Developer Experience team, and we’re excited to share two dual-screen enhanced samples<\/a> that showcase video experiences on foldable devices. \n<\/p>\n

\n Our samples were inspired by this Google I\/O video Foldables – WindowManager with MotionLayout<\/a> and the ConstraintLayout FoldableExperiments sample<\/a>. The goal for each sample is to demonstrate a video player user experience that works great on single-screens, the dual-screen Surface Duo, and other foldable devices.\n<\/p>\n

\n \"FoldableFigure 1: Adapting to device posture on a foldable emulator<\/em>\n<\/p>\n

Dual-screen video player<\/h2>\n

\n On single-screen devices, video controls are normally placed on top of the video. On foldable and dual-screen devices this could introduce some issues. Hinges can cut off large parts of the video and controls in certain configurations, and hinges can undesirably distort the video. Therefore, it makes sense to sometimes separate the video and the controls, so they are displayed in separate areas on the device.\n<\/p>\n

\n On devices like Surface Duo, there is a permanent hinge that blocks out a portion of the video. This isn\u2019t too disruptive when the app is spanned in dual-portrait mode, but we can still separate the controls to make them accessible. When the app is spanned in dual-landscape mode, we always want to place the video on the top screen and the controls on the bottom screen to avoid having the video blocked by the hinge. On foldable single-screen devices, there is no \u201chinge,\u201d but we still may want to separate the controls when the device is partially folded.\n<\/p>\n

Technical details<\/h3>\n

\n MotionLayout is used to animate the movement of the PlayerView and external ControlViews. The PlayerView and the ControlViews are constrained to a ReactiveGuide, which changes depending on the presence and location of a hinge\/fold.\n<\/p>\n

\n The app uses Jetpack Window Manager to detect the presence of a fold\/hinge when the device\u2019s layout configuration changes. When there is a hinge and the controls need to be separated from the video, the app calculates the position of the fold using the FoldingFeature provided by Jetpack WM\u2019s WindowLayoutInfo and the MotionLayout view. The app pushes that fold position to the ReactiveGuide. The PlayerView and the ControlView are both constrained to the ReactiveGuide position, so they move accordingly to separate the video and the controls. The overlaid controls on the PlayerView are disabled, leaving only the video on one half, and the external controls on the other half. On Surface Duo, this is implemented twice \u2013 once for dual-portrait, and once for dual-landscape.\n<\/p>\n

\"Surface<\/a>Figure 2: User has control over the viewing experience on Surface Duo \u2013 full screen to maximize viewing area or side-by-side with the controls.<\/em>\n<\/p>\n

\n\"Surface<\/a>Figure 3: Surface Duo \u201claptop\u201d viewing posture with video on the top screen and controls on the bottom.<\/em>\n<\/p>\n

inner class StateContainer : Consumer {\r\n\r\n  override fun accept(newLayoutInfo: WindowLayoutInfo) {\r\n\r\n    fab.hide()\r\n\r\n    \/\/ Add views that represent display features\r\n    for (displayFeature in newLayoutInfo.displayFeatures) {\r\n      val foldFeature = displayFeature as? FoldingFeature\r\n      if (foldFeature != null) {\r\n\r\n        lastFoldingFeature = foldFeature\r\n\r\n        \/\/isSeparating is true if there is a hinge, or a fold is apparent in the device\r\n        if (foldFeature.isSeparating) {\r\n\r\n          if (foldFeature.orientation == FoldingFeature.ORIENTATION_HORIZONTAL) {\r\n            var fold = horizontalFoldPosition(motionLayout, foldFeature)\r\n            ConstraintLayout.getSharedValues().fireNewValue(R.id.horiz_fold, fold)\r\n            playerView.useController = false\r\n\r\n          } else {\r\n            \/\/move over on-video controls so it's not cut off by hinge\r\n            setMargins(connectedControls, connectedControls.width \/ 2, 0, 0, 0)\r\n\r\n            \/\/make split control fab visible\r\n            fab.show()\r\n            \/\/update vertical reactive guide with verticalFoldPosition()\r\n            updateSplitControl(lastFoldingFeature)\r\n          }\r\n\r\n        } else {\r\n          \/\/reset fold values\r\n          ConstraintLayout.getSharedValues().fireNewValue(R.id.horiz_fold, 0)\r\n          ConstraintLayout.getSharedValues().fireNewValue(R.id.vert_fold, 0)\r\n          playerView.useController = true \/\/ use on-video controls\r\n\r\n        }\r\n      }\r\n    }\r\n  }\r\n}<\/pre>\n
Video player with chat<\/h2>\n
\n  Video watching experiences often include user participation. This sample focuses on a streaming experience where users often simultaneously watch the livestream and interact with a live chat. The dividing of screen space between two panels, such as a video panel and a chat panel, can work especially well on dual-screen devices. \n<\/p>\n
Layout options<\/h3>\n
\n  In single-screen situations, the device orientation is enough information to arrange the panels. In landscape mode, the panels should be arranged left-to-right. In portrait mode, the panels should be arranged top-to-bottom. The relative size of the panels is somewhat arbitrary \u2013 the animations below show a ratio of 70\/30 for landscape and 50\/50 for portrait. If the user only wants the video panel, it should fill the screen.\n<\/p>\n
\n\"Surface<\/a>Figure 4: showing and hiding the chat on single-screen <\/em>\n<\/p>\n
\n  On foldable devices, the orientation of the fold is more important than the orientation of the device. If the fold is vertical, the panels should be arranged left-to-right, and if the fold is horizontal, the panels should be arranged top-to-bottom. The panels should be on opposite sides of the fold and take the width of the fold into account. If the user isn\u2019t using the chat panel, the video panel should only span across the fold if the video experience can be improved (the video gets larger).\n<\/p>\n
\n\"Surface<\/a>Figure 5: showing and hiding the chat on a spanned app<\/em>\n<\/p>\n
Layout technology<\/h3>\n
\n  In this sample, the video and chat panels are arranged in a MotionLayout. The layout uses two constraint sets, one for \u201cfull-screen\u201d mode, and one for \u201cchat-enabled\u201d mode.  Since both top-bottom and left-right arrangements are possible, the chat fragment is duplicated; one lives under the video panel, and another to the right of the video panel. The three panels are constrained to two guidelines, one horizontal and one vertical.\n<\/p>\n
\n  Guidelines can be positioned programmatically, making them perfect for executing the discussed layout ideas. With enough layout information, the program can dynamically control the guideline positions in the \u201cchat-enabled\u201d constraint set. (The \u201cfull-screen\u201d constraint set can lock the guidelines to keep the chat panel invisible.)\n<\/p>\n
\n  Like the previous sample, the app uses Jetpack WM to collect layout information each time the device\u2019s layout changes. These updates, along with the user toggling \u201cfull-screen\u201d or \u201cchat-enabled\u201d mode, trigger the layout updating functions. MotionLayout automatically animates the transition between constraint sets, giving the app a smooth feel. The code snippet below shows how the root MotionLayout is controlled.\n<\/p>\n
fun setFullscreen() {\r\n    bottomChatView.setPadding(0,0,0,0)\r\n    endChatView.setPadding(0,0,0,0)\r\n    rootView.transitionToState(R.id.fullscreen_constraints, 500)\r\n}\r\n\r\nfun setGuides(vertical_position : Int, vertical_padding : Int, horizontal_position: Int, horizontal_padding: Int) {\r\n    bottomChatView.setPadding(0,vertical_padding,0,0)\r\n    endChatView.setPadding(horizontal_padding,0,0,0)\r\n\r\n    var constraintSet = rootView.getConstraintSet(R.id.shrunk_constraints)\r\n    constraintSet.setGuidelineEnd(R.id.horizontal_guide, vertical_position)\r\n    constraintSet.setGuidelineEnd(R.id.vertical_guide, horizontal_position)\r\n\r\n    if (rootView.currentState == R.id.shrunk_constraints) {\r\n        rootView.updateStateAnimate(R.id.shrunk_constraints, constraintSet, 500)\r\n    }\r\n    else {\r\n        rootView.transitionToState(R.id.shrunk_constraints, 500)\r\n    }\r\n}<\/pre>\n
\n  Just like the example at the start of the post, the chat can detect and use the fold on single-screen devices as an anchor when showing video and chat together:\n<\/p>\n
\n  \"FoldableFigure 6: Chat viewing experience on a foldable device<\/em>\n<\/p>\n
Feedback and resources<\/h2>\n
The code for the FoldingVideo sample<\/a> and the FoldingVideoPlusChat sample<\/a> is available on GitHub<\/a>.\n<\/p>\n
\n  If you have any questions, or would like to tell us about your apps, use the feedback forum<\/a> or message us on Twitter @surfaceduodev<\/a>.\n<\/p>\n
\n  Finally, please join us for our dual screen developer livestream<\/a> at 11am (Pacific time) each Friday \u2013 mark it in your calendar and check out the archives on YouTube<\/a>. <\/p>\n","protected":false},"excerpt":{"rendered":"
Hello Android developers! The Microsoft Surface Duo is a great device for consuming multimedia content, and ExoPlayer is a simple, powerful widget for rendering video. Raymond and I are interns on the Surface Duo Developer Experience team, and we’re excited to share two dual-screen enhanced samples that showcase video experiences on foldable devices. Our samples […]<\/p>\n","protected":false},"author":65067,"featured_media":1683,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[718,706,473,46],"class_list":["post-1677","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-surface-duo-sdk","tag-exoplayer","tag-jetpack-window-manager","tag-kotlin","tag-surface-duo"],"acf":[],"blog_post_summary":"
Hello Android developers! The Microsoft Surface Duo is a great device for consuming multimedia content, and ExoPlayer is a simple, powerful widget for rendering video. Raymond and I are interns on the Surface Duo Developer Experience team, and we’re excited to share two dual-screen enhanced samples that showcase video experiences on foldable devices. Our samples […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/posts\/1677","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/users\/65067"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/comments?post=1677"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/posts\/1677\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/media\/1683"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/media?parent=1677"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/categories?post=1677"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/surface-duo\/wp-json\/wp\/v2\/tags?post=1677"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}