Best Practices and Tips for Using Xamarin.UITest

We can all agree that it is very important for mobile apps to have great user interfaces, be accessible, and be extremely performant. Given the plethora of devices with different screen sizes, screen densities, and RAM, it is of utmost importance that we not only unit test app functionality but also have a set of robust user interface tests to ensure high quality and performant apps. We, on the Mobile Customer Advisory Team, are often asked to help customers get started with UITests or help make their tests scalable and maintainable. I’d like to share some best practices and tips & tricks our team uses to help write scalable and easy to maintain UITests using Xamarin.UITest.

Page Object Pattern

Our team created the Page Object Pattern (POP) to meet the need we saw for a simple way to create good UI tests. Just as you might follow the MVC or MVVM pattern to architect app code, you can use POP to architect UITest code. Having tried both simpler and more complex architectures in the past, we found this approach to be one of the best. It is easily adopted by people who are learning how to write tests. It also provides the scalability needed to build larger, more complicated test suites. This pattern, as the name suggests, revolves around treating each page of the mobile application as an individual class of its own. These page representations handle all the interaction with the corresponding page in the app. This simplifies test methods down to just calling methods on page representations. The advantages of modelling it this way are:

When writing tests, they become much easier to read due to the semantics following a pattern of: <PageName>.<Action_performed_on_Page>
The tests are scalable as the app grows because you need only add the respective UITest page. If a new button is added to a page, you only need to edit the code in that Page class.
The pattern is intuitive and easy to implement. You don’t need to rely on any libraries and the few supporting files that we provide can easily be edited to fit your specific needs. The supporting files are:

This pattern also makes it fairly easy to do cross-platform tests as long as the apps’ pages are similar across platforms. Instructions for implementation and the basic code needed to get started with this pattern can be found in our

GitHub Sample. Let’s highlight the basic concepts that make up this pattern.

Each page in the app is a class in your test code

There should be a one-to-one mapping between pages in the app and page representations in your test code. If you navigate away from a page, or if a modal view covers the page so that you can no longer interact with the page underneath, then it should be considered a new page and you should define a new page class in your test code.

Page representations know about all the elements on that page

Page classes save queries for all the elements on the page as fields at the top of the file. Don’t define any queries inside methods. Define them all as fields at the top of the class and assign them in the constructor based on the platform. This makes it easy to reuse and change the queries as needed.

public class LogInPage : BasePage
{
    readonly Func EmailField;
    readonly Func PasswordField;
    readonly Func LogInButton;

    protected override PlatformQuery Trait => new PlatformQuery
    {
        Android = x => x.Id("log-in-image"),
        iOS = x => x.Id("sign-in-image")
    };

    public LogInPage()
    {
        // The same for both platforms
        LogInButton = x => x.Id("log-in-button");

        if (OnAndroid)
        {
            EmailField = x => x.Id("android-email-field");
            PasswordField = x => x.Id("android-password-field");
        }

        if (OniOS)
        {
            EmailField = x=>x.Id("iOS-email-field");
            PasswordField = x => x.Id("iOS-password-field");
        }
    }
}
 Also, each page must define a

** Trait property. The “trait” of a page is a query for some element that is unique to that page. The BasePage uses a WaitForElement ensure that the trait appears (i.e. the page is fully loaded) before you can execute any methods on the page.

Page classes define methods for all the actions that can be taken on that page

You should only interact with a page in the app through methods defined by the page representation. If a method navigates away from the page (e.g. taps a button that goes to another page in the app), the method should return void since you can no longer call more methods on that page. Methods that do not navigate away from the page should return their own type (using return this;) so that more methods can be called on the same object. This creates a fluent interface that is easy to work with.

new LogInPage()
  .EnterCredentials("name", "password")
  .ConfirmLogIn();

The benefit of the fluent interface is that you don’t need to save the page to a variable so the code can be more concise.

Test methods consist only of calls to page methods

There should be no calls to Xamarin.UITest in test methods, only in page methods. Take another look at the diagram above and notice that only page representations ever interact with pages in the app via Xamarin.UITest. The test method never calls into UITest directly. Don’t use app. in [Test]methods. That is the page’s job. Tests should only instantiate new pages and make calls to those pages like so:

[Test]
public void LogInAndExploreTest()
{
  new LogInPage()
    .EnterCredentials("name", "password")
    .ConfirmLogIn();

  new HomePage()
    .SelectFirstItem();

  // ...
}

You can find our sample code for POP with Xamarin.UITest

here. We have provided steps and guidelines to implement this pattern and also some tips for optimizing your tests in our wiki. We also have samples of using POP with the other testing frameworks supported in Visual Studio App Center: * Page Object Pattern : XCUITest (Apple’s UITest Framework) Sample * Page Object Pattern : Appium (Java Appium Framework) Sample * Page Object Pattern : SpecFlow with Xamarin.UITest Sample

General testing tips

These tips are not specific to POP and should be applied no matter how you choose to architect your tests.

Make each test self contained

Every test in your test suite should be able to run on its own without depending on any other tests to run successfully beforehand. Every test should start with a clean slate as if the app was just installed on the phone. For example, you should not have a single test that logs into your app at the beginning of your suite and then have every subsequent test assume that the app is already logged in. If the login test were to fail for any reason, every other test in the suite that relied on it would also fail. In order to save on valuable time, you can use backdoor methods to speed up repetitive setup tasks like logging in. You can also use backdoor methods to reset the app and put it in a particular “clean” state before each test starts.

Split tests up into small chunks wherever possible

Long tests are just as bad as interdependent tests. If the first part of a long test fails, then it won’t finish the rest of the test. For example, it’s better to have one test to log in, one test to search for an item, and one test to purchase an item. This way even if there is a bug in the search feature you can still test the purchase feature. Backdoor methods can, again, provide a fast way to skip some parts of a long process and put the app into a certain state. For example, you could use a backdoor to add a product to your shopping cart before running a check out test.

Use this handy alias

All the methods in UITest generally take in a lambda expression that specifies the element to interact with (we often refer to this as a “query”). For example, app.Tap(x => x.Id("log-in-button"))uses the lambda x => x.Id("log-in-button") to define what to tap on. All queries are expected to be of type Func<AppQuery, AppQuery>and to define them you write:

readonly Func LogInButton;

// In constructor:
LogInButton = x => x.Id("log-in-button");

However, writing

Func<AppQuery, AppQuery>for every query can get tiresome and isn’t that descriptive. We can make defining queries easier by adding the following line with our using statements at the very top of the file: using Query = System.Func<Xamarin.UITest.Queries.AppQuery, Xamarin.UITest.Queries.AppQuery>; This creates an alias of Func<AppQuery, AppQuery>with Query so we can rewrite the original definition like so: readonly Query LogInButton;

// In constructor:
LogInButton = x => x.Id("log-in-button");

Be as deterministic as possible

Avoid branching on UI state – Your test should expect a certain UI state and assert that it is as expected rather than querying the UI to determine the next step.
Avoid branching in general (if****, else**** ****and try****,**** catch****) – These are warning signs that you are not being as deterministic as you could be. They can be useful in certain situations but be conscious about whether you can use an assertion instead.
Be careful with loops, especially the while**** ****loop – Make sure you don’t have loops that will run forever if something goes wrong with your app. Limit them to a reasonable number of iterations.
Try not to use Thread.Sleep(int)– Instead, WaitForElement and related methods provide a much better alternative. The only exception to this rule is when you have an animation that needs to complete before you want to interact with any elements.
Try not to do anything randomly – Test results should be repeatable between test runs, randomness inherently takes this away.
Use assertions liberally (WaitForElement****, WaitForNoElement****, and Assert****) – We write tests to confirm app behavior, assertions are essential to confirming conditions. Best practice would be to have an assertion after every action.
Take many screenshots – In Visual Studio App Center, screenshots are the best way to see every step of a test’s execution and visually identify bugs. Best practice would be to take a screenshot following every assertion (i.e. take some action, make an assertion, take a screenshot).