Gaming on dual-screens, from a Flutter perspective

John Wiese

So, you’ve built a game and it works great on all those single screen devices out there but what about these new foldables or the dual-screen Microsoft Surface Duo? Does your game work on it? It should, in single screen mode, but what happens when the user spans your game across two screens? Does that work? Should it work the same or is there a better way for your game to lay itself out when spanned across two screens? These are the questions I hope to help you answer in this article.

To help with this post, I built a game using Flutter and the amazing Flame game development engine. If you are interested in learning more, please check out the Flame engine docs and their very active Discord. Let me know if you’d like me to do a write-up of the process I went through to build the game as I won’t be covering that here; just ping me @johnwiese on twitter.

The Reference Game

I picked a relatively simple game to develop. This is only the second game I’ve ever written and the first was about 25 years ago in college…in C…on a Unix system, so it was fun to learn the ins and outs of game development for this article. Having this game as a reference will allow us to discuss some of the points one needs to consider when looking at supporting dual-screen devices in their game. These are the points we will cover:

1. Should you support your game spanned on a dual-screen device?
2. What are the different ways to support a game spanned on dual-screen devices?
3. What things should you consider as you build a game, or are modifying your game to support #2?
4. How was all this accomplished in the reference game?

Should you support your game spanned on a dual-screen device?

The very first thing you should be asking yourself is should I support running across two screens. Not all games will work well when spanned across multiple screens, and that is ok. If you have a game in mind, or already created one that you do not think would work well when spanned across two screens, then there are a few things to think about. At the present time, the Surface Duo does not provide a way programmatically or in the OS to stop an application from being spanned. Given this, how do you want your game to react when it is spanned? The default is to just continue to run as if on one big screen; is that ok with you? Will that user experience cause negative reviews? Your best option is to implement code that sees the spanning happen and then displays a notice to the user telling them that the game is designed for a single screen and may not work as expected when spanned across two screens. This at least let’s your users know that you’ve thought about it and decided that it doesn’t really make sense to invest the time to modify the game to run on dual-screen devices. While it will entail a small amount of work, it is relatively simple, and we have a lot of documentation to help you get the code implemented.

Before making that decision, ask yourself: Could my game be refactored to make it work (maybe even be better) on a dual-screen device?

What are the different ways to support a game spanned on dual-screen devices?

There are a few ways you could modify your game to support running on a dual-screen device. Perhaps your game is in the very small minority of games that will continue to work just fine when spanned across two screens. If so, you are very lucky.

Most game developers will need to consider if their game could be broken into two screens to enable supporting running spanned across two screens. This could mean that you modify your game to account for the space where the hinge is. This will require that you redraw the playing area to include “blank” space where the hinge is. For simple games like Backgammon, Checkers, Chess, or other games that have a fixed layout, this may be the best option. You can account for the hinge in your layout and then simply insert that space when the game is spanned and ignore that code when it is not spanned.

Image 2: Taking single screen layout to two screens

The second way to modify the game to support running spanned across two screens is to move pieces of the application around. For this scenario, let’s take the example of any of the popular first-person shooter games on the market today. In all of these games, they overlay the controls on top of the game play screen. This works well on a single screen, but what happens when the user spans the game? What would happen with no change to the game code is that the game will simply span across both screens…no problem, right? Nope, if you think about it, in all these types of games the focal point is the very center of the screen. This is where you are aiming your gun, where you are focusing on your opponent, etc. With no changes to the code, this area of the game will fall under the hinge and you won’t be able to see where you are aiming!

A better way to handle things in these types of games, and many others, would be to split the current screen into two distinct screens, and force the screens into the landscape view when the game is spanned across two screens. This gives the player a great experience. The main view of the game, the primary playing area, is on the top screen and can be their primary focal point. The controls they need to interact with can be placed on the bottom screen. Things like inventory, changing weapons, movement control, etc., can all be placed on the second screen and no longer interfere with the game play view. Wouldn’t that be a great experience for the game player? Of course, this will require more work to adapt an existing game and there is much to consider when beginning this work.

What things should you consider as you build a game, or are modifying your game to support running spanned across two screens?

When you begin to dig into your code, or plan your game, there are a few things that you’ll need to consider. The biggest thing to consider is how you layout the screen.

Think in two screens

What does this mean? If you are building a game from scratch, think about how you will split things between two screens if you have them. What would go on one screen versus the other? If you have a good picture of this when you start, you can ensure the rest of your work lends itself to easily adapting to a two-screen device.

Do you use relative or absolute positioning?

Which positioning method you use will determine how much work you must do to make your game work spanned across two screens. For instance, if you are placing items on the screen based on absolute positions (e.g. “x pixels from the top and left sides”) then you may require minimal work, but if you are placing items based on pixels off the bottom of the screen you could run into issues. When the app is spanned, the reported screen size will change and you may need to account for that in your placement code. Part of this could be fixed if you replace using the Left, Top, Right, and Bottom with specified pixels you query from the device. For instance, if you instead first determined the “usable” screen area, then did your placement off those dimensions, you would have less work to do and your game would work on a single screen or spanned across two screens.

Image 3: Placing things using the screen dimensions instead of the edge

Should the orientation change? How does that affect layout?

When the device is rotated from portrait to landscape, does your game adapt? How does it adapt? Maybe you don’t adapt because “no one would do that for my game a single screen device”, that could be possible too. This is also something you will need to think about when you consider supporting a dual-screen device. When a user spans your game on a dual-screen device like the Surface Duo, you may want to force the game into a landscape orientation to account for the two screen layout, or perhaps you don’t. If your game makes sense to split the screen into a left/right vs. a top/bottom orientation, then keep it in portrait; but if not, you will need to consider re-laying out your game.

Do you layout your controls independent of the game play?

Most of you are probably doing this already as it is normal to do these things independently, but you should be handling game play layout separate from game control layout. If you’ve done this, then you should be able to easily layout the game controls on the second screen and keep game play on the primary screen.

I’m sure as you go through your game and begin to think about adding support for spanning on a dual-screen device you will encounter other things I haven’t listed. Each game is different, and some will have situations to handle that others do not.

How was all this accomplished in the reference game?

So, how did I accomplish taking the single screen concept and enabling it to support spanning on a dual-screen device like the Surface Duo? The first thing I did was create the game play. After having the general layout, graphics, and requirements figured out, I created the game to run on a single screen device. When doing this, I used some of the guidance from the Flame community for adding a splashscreen and other screens using Routes. This gave me a main widget that looks like this:

@override
Widget build(BuildContext context) {
  return MaterialApp(
    debugShowCheckedModeBanner: false,
    routes: {
      '/splash_screen': (ctx) => FlameSplashScreen(...),
      '/title': (ctx) => TitleScreen(),
      '/game_over': (ctx) => GameOverScreen(),
      '/game': (ctx) {...},
    },
    initialRoute: '/splash_screen',
  );
}

With this in place, I built out the game and ended up with a great single screen experience.

Image 4: Original single screen view of the reference game

Notice how the controls are overlayed on the screen. This works great on a single screen, but when you span the game across the two screens of a Surface Duo, you end up with an “unplayable” area in the middle of the screen no matter if you play it in portrait or landscape mode. To get around this, I decided that when the game is spanned, the first thing I would do is force the game into a landscape orientation. With that decided, I then decided that I would also keep the game area (the area from the top down to the players ship) on the top screen and the controls would go on the bottom screen.

To accomplish this, the first thing I had to do was “know” when the game was spanned versus not spanned. I was able to do this using the multiple_screens flutter package. This package provided me with a stream I could monitor to get the current spanned status.

MultipleScreensMethods.isAppSpannedStream().listen((data) =>
    setState(() => _isAppSpannedStream = data));

With this info I could adjust my layout code to layout the game area and the controls on one screen in the un-spanned state, or on different screens when spanned.

if (_isAppSpannedStream == null || !_isAppSpannedStream) { 
  Flame.util.setPortrait(); 
  return Directionality( 
    textDirection: TextDirection.ltr, 
     child: Stack( 
        children: [ 
          _game.widget, 
           Column( 
                children: [ 
                 Spacer(), 
                 Gamepad(game: _game), 
                  SizedBox(height: 64), 
               ], 
            ),  
         ], 
      ), 
   ); 
} else { 
  Flame.util.setPortrait(); 
  return Column( 
    children: [ 
       Expanded( 
           flex: 1, 
           child: _game.widget, 
        ), 
        SizedBox(width: 34), 
        Expanded( 
           flex: 1, 
            child: Container( 
              color: Colors.black, 
              child: Gamepad(game: _game), 
           ), 
        ), 
     ], 
  ); 
}

As you can see in the code, I am also able to force the orientation. In this case, the orientation for Flame is always Portrait as it lays out based on the dimensions, which will always have a greater height that width due to the device reporting both screens as one large screen when spanned.

In my case, my layout when spanned uses a simple column to place the game play area in half the screen and the controls area in the other half of the screen, then inserts a small empty box to account for the hinge area. In a future update to the code, I would not hardcode this, but instead pull the value using the APIs to ensure it works if a new device has a different width for the hinge.

The rest of the game “just works” like it should. Of course, that wasn’t the case out of the gate, which is how I came up with some of the things you need to think about when building or modifying your game to support dual-screen devices like the Surface Duo.

What’s Next

For the game itself, it’s there for anyone to contribute PRs to and play on their devices, both single and dual-screen, iOS, or Android. There are a lot of things that could be done in the game. Whatever you can think of, go for it! Fork the repo or create a branch and make a Pull Request.

For you, the next step is to think about how this applies to your game, or the one you are going to create. Take advantage of the dual-screen possibilities and come up with a new and great way for game players to experience your game!

You can find the source for the reference game here: https://github.com/johncwiese/tech_invaders

Feedback

We would love to hear from you about your experiences using the Surface Duo SDK, emulator, and your thoughts on how you can utilize these in your Flutter applications and games.

Please reach out using our feedback forum or message me on Twitter or GitHub.

John Wiese Principal Technical Architect, Business Applications

Follow