Guest Blog | A Mobile DevOps Retrospective Part II: Automation
As I wrote in my last post, DevOps is (1) more than tooling, (2) crucial to mobile app development, (3) rarely done, and (4) worth it.
Today, I’ll dig deeper into one of DevOps’ core concepts and something all developers are intimately familiar with: automation. I’ll walk through how we’ve automated our process at Olo (where we build restaurant ordering apps for 150+ brands), give you some real-world implementation examples and advice, and share how automation has impacted our development process, our team, and our mobile quality.
DevOps inherently understands developers’ goals. We want to automate anything that can be automated, because automating everything frees us to spend our valuable and scarce development cycles solving real problems.
For my team at Olo, mobile DevOps means automating much of the development process, from app builds to unit and UI tests, and from beta testing to app store deployment. With Visual Studio Mobile Center, you can easily automate each step of your process—or choose which services work best for you*—and stay focused on shipping great apps to your users.
*Visual Studio Mobile Center includes cloud services for Build, Test, Distribute, Push, Crash, and Analytics, but integrates with several 3rd party dev tools and services, so we pick and choose what works for us (e.g. we’ve been using another service for CI, but we use Xamarin Test Cloud (Mobile Center Test) and HockeyApp (Mobile Center Distribute) for automated testing and beta distribution.
Having a predictable, deterministic build process is an absolute must-have. If you’ve ever set up a developer machine for app development, you’re well aware of how many different tools mobile apps require: the version of Windows or macOS, Xcode, Android Studio, Java, the Android SDKs, Android’s build tools, the list goes on and on. Whatever tech stack you choose, there are a lot of moving parts.
If your development machine is your build environment, any changes on your machine can derail your build process, which is incredibly risky. Offloading your builds to a separate process frees up your developer machine to actually be a developer machine. You can try out new versions, work on bug fixes, or experiment with features safely, without fear that you’re impacting your ability to ship and publish your apps.
Many years ago, my laptop was the only machine that could build our apps. Not only was it time-consuming, but things suddenly (and often) stopped building for all sorts of reasons: sometimes it was an operating system or Android / iOS SDK update, sometimes a bug we’d introduced. Without a consistent build environment, identifying the root cause was like playing terrible game of whack-a-mole.
We now have dedicated build machines, and comparing our productivity and confidence level before and after is like night and day. Every commit to every branch kicks off builds for all projects in the repository, and we expose those builds in our pull requests. Since all team members have visibility, it’s much easier to surface errors and ensure we don’t merge broken features.
I also strongly suggest making as much of your build process as possible reproducible on your development machines. If you can locally run the same process that your build process runs, you can quickly test your process, debug any issues, and add new features. It’s no fun trying to determine why something doesn’t work on the build machine when it runs like a charm on your own. FAKE and Cake are great options for setting up your build scripts and allowing you to define them in F# or C#, respectively.
Check out my “Using Continuous Integration” session from Xamarin Evolve 2014 for more details about how we’ve implemented build automation at Olo and tips on how to incorporate in your development pipeline.
After you’ve gotten your predictable builds up and running, it’s time to start automating your testing processes.
Unit and Integration Testing
A good starting point is building unit or integration tests in your favorite testing framework. If you don’t have a favorite yet, I’d recommend exploring xUnit, a great framework with a top-notch cross-platform story, allowing you run your tests across Android, iOS, UWP, and more.
Covering large portions of your app with functional tests means you maintain high confidence that changes aren’t negatively affecting any part of your app, especially in unexpected, seemingly unrelated ways.
At Olo, a vast majority of our app logic is in a core platform-agnostic project, including: the behavior of every screen, the navigation patterns and behavior, database access, network access, logging, and more.
This architecture allows us to cover most of any app’s behavior in a comprehensive set of unit tests. We run all of the tests on every single build, helping us quickly and easily identify any regressions, anywhere in our apps, as we develop new features. This has saved us from mistakes on many occasions. Stability across releases is an absolute top priority, and our tests give us the confidence we need without requiring us to manually test everything, every time.
In addition to making our core library platform-agnostic, we’ve done the same for the tests around it. We take that same suite of unit tests and run those tests on real Android and iOS devices, courtesy of xUnit’s cross-platform support, so we’re sure our code behaves as expected in real-world scenarios before our apps get into users’ hands.
Ultimately what matters is how your assembled app actually behaves, and these techniques are ways to help you ensure the quality of that behavior. Visual Studio Mobile Center allows you to automatically test your apps on real devices in the cloud to catch issues before they get out into the wild, regardless of whether your app is written in Objective-C, Swift, Java, React Native, C#, or F#.
For more information on how to architect apps for testability and why it matters, check out my “Keeping Your Users Happy with Testable Apps” session from Xamarin Evolve 2013.
Unit tests are great, but they’re not enough. Ultimately, real people are the ones using your apps, not a build process, so you need a simple way to get your test builds into the hands of real users.
Going back some years, distributing our apps to our internal QA and external beta users was painful. Provisioning and installing apps onto test devices required plugging them into my laptop, since that was also the build machine, which isn’t scalable. We were a growing company, becoming more and more remote.
Today anyone on the team, whether developers, QA, or product managers, can push a single button to build, package, and deploy any one of our apps to our internal distribution system. How? We’ve integrated distribution into our build system, allowing our QA engineers and stakeholders across the entire company to easily access, try out builds, and provide valuable feedback.
Seeing product specs and screenshots are great, but there’s no substitute for having the app in your hands and trying it out. Team members can create builds from any release point, from the current version available in the app store to the newest in-progress bits with all the latest features.
Pushing releases to our internal distribution system also sends a Slack notification, so everyone stays up to date, without having to dig.
To start sharing with your team members, internal testers, and external testers, check out Visual Studio Mobile Center’s Distribute service. You can quickly create and add members to distribution groups, and easily notify those groups when new releases are available to download.
In addition to automating how you distribute your test builds, it’s critical to automate how you package and prepare your production-ready apps for publishing.
Just like with your build pipeline, you want your packaging process to be consistent, predictable, and tied to the same processes you use to create your test builds.
Without consistency between your testing and packaging builds, you can’t be sure you’re about to deploy the same app you tested. Once you release an app into the store, you can’t pull it back. The only option is to roll forward, submit an update, and hope your users get the latest version.
At Olo, our internal distribution processes are fundamentally the same as our public deployment processes, including: creating an app store-ready package with the click of a button in our build system. As I mentioned earlier, building a mobile app requires a lot of tooling, so it’s also important to know precisely which versions of those tools went into any app we’re testing or releasing.
Each time we package an app for testing or distribution, we capture all of the tooling versions used to create that package (such as the versions of Xamarin, Java, the Android SDK, and so on). If we discover a problem with a particular version of something, we can easily see exactly which apps were built with that particular configuration and make any necessary changes.
If you’re building Xamarin apps, check out our Visual Studio for Mac add-in called Olo.BuildTools to capture this information as part of your build process.
Automation for the Win
Automation has obvious benefits: saving time, improving quality, and freeing you to focus on what matters. When done right, though, it also creates a better team dynamic (throughout the organization, not just the development team). DevOps is inherently about people and culture, and automation makes it easy to break down traditional barriers and simplify sharing across teams, offices, and roles.
For me, I’ve seen us foster a real shared sense of ownership; everyone feels like they have a stake and voice in both our products and the process. The result: better quality products and relationships.
Check out my next post to see what “measurement” means to us. I’ll walk through how and why to establish good feedback channels, understand how your apps behave in the wild, learn how real users interact with your apps, and incorporate qualitative and quantitative data into your next release.
If you haven’t already, create your Visual Studio Mobile Center account, connect your first app, and start shipping better apps now.
Have an account? Log in and let us know what you’re working on!
About the Author
Greg Shackles is an experienced developer, Microsoft MVP, Xamarin MVP, and spends his days as Principal Engineer at Olo. When he’s not coding, he hosts the Gone Mobile podcast, organizes the NYC Mobile .NET Developers Group, and he wrote Mobile Development with C# to help developers use their existing skills to create native mobile apps.