How Layers Improved our UI Testing Strategies
By
Sergio Suárez
At FARFETCH since 2018. Android geek. Love learning something new every day. Curious about Casablanca outfits.
View All Posts
Re-using instead of re-doing
Within the development cycle, testing is a key part of the process. It is an essential tool that pushes the boundaries of your product. Even without thinking about it, teams test new functionalities every day. And, as in FARFETCH, when a company grows, testing strategies evolve in order to tackle problems faster. How fast? Preferably, before they occur.The objective of this article is not to explain which Quality & Assurance processes are executed to guarantee the best performance of our apps in the market. Instead, it is specifically meant to show how the Android Team adapted the existing procedures, and plans the evolution of our testing tools.
Baby steps
Early in 2019, the opportunity of exploring how to improve the testing tools presented itself. My first approach was an analysis of what testing constituted FARFETCH’s Android team. I remember thinking how difficult and time consuming it was to dig into testing strategies. From specific test results to overall walkthroughs of the app, all were a constant reminder of how little time I had been at the company. I was discovering something new with every step. And I guess that is the feeling everyone experiences from time to time.
Encapsulate common behaviours
Sales Season is a remarkable moment to be a Farfetcher. The time of the year where perfection is a must, and a crash can turn out to be a nightmare. Lucky for us, QA runs every possible scenario performing a repeatable process. It consists of executing a set of scenarios, existent in the app, aiming for perfection.
Basic tools, powerful outcomes
In December 2019, after almost an entire year had gone by, managers encouraged Farfetchers to select new challenges for the upcoming year. I had been working on testing for a while and I was pretty confident. However, I had been trying to figure out an 11% inconsistency between two tracking providers. At this point, finding the origin was not enough. How could I make sure that I’m actually solving the problem?
I just described the three moments that defined the UI testing framework. So now, let me run you through how the Android team views UI testing.
- Reduce regression time execution.
- Cover all possible user’s point of entry in the app.
- Increase tracking reliability.
Reduce regression time execution, guarantee quality while saving time
Every two weeks our QA teams run a set of tests in order to detect possible issues. And here is where engineers must step up.
What do we do with repeated tasks? Automate them.
Android provides a set of tools that allow engineers to simulate interactions with views. Many of you might be familiar with some of them, they are not exclusive to our world. And it is useful for developers since you can easily simulate a recently coded feature. However, what if it wasn’t coded recently? What if you weren’t the one building that amazing feature?
An analysis or code revision is needed from time to time in order to refactor, re-do or build new tests. On the other hand, rapidly growing a set of tests requires manpower.
From the start, qualified personnel and susceptibility to app changes arose as impediments.
Here at the Android team, we believe that anyone should be able to write UI tests. Framework (tool) knowledge shouldn’t be an obstacle. Writing a test is similar to declaring an ordered list of tasks to be completed within a context. Reducing the friction between software and collaborators is how we managed to achieve the manpower constraints.
Now, how to battle the constant changes and technical knowledge of the frameworks?
"A robot is just a class that encodes the high level actions that we want to take on a view, or anything else.”
By definition:
- What: Within the Test layer are defined the set of interactions possible with the view. Building readable functions allows you to write a test effortlessly, as if you were defining a to-do list.
- How: Robots constitute high-level classes that encapsulate the logic behind scenes and mandatory assertions to make sure the information displayed is right.
This approach provides the separation of concerns needed in order to create a test without worrying about the logic behind it. However, tests are susceptible to structural modifications on the app. Imagine a breaking change, such as the user not being able to access the "Me” section (personal details) front bottom navigation bar. Engineers create and maintain Robots, which are affected by alterations in the interactions with the view, for example, a change in how Espresso performs a click operation.
The best option might be to share an example. So let’s jump into the code for a couple of minutes.
Notable mention: The team approached a Robot as a Screen presented to the user. It resulted in a very practical translation.
Here you can find the current definition of the On Boarding test:
@Test
fun testOnBoarding() {
[...]
splashGender {
selectMen()
} result {
isGenderMenStored()
}
[...]
splashPush {
selectNotifyMe()
} result {
isFarfetchNotificationsEnabled()
}
}
The Robot class:
fun splashGender(block: SplashGenderRobot.() -> Unit) =
SplashGenderRobot().apply { block() }
class SplashGenderRobot {
[...]
init {
isDisplayed(R.id.splash_gender_image)
}
fun selectWomen() { click(R.id.splash_gender_shop_women) }
fun selectMen() { click(R.id.splash_gender_shop_men) }
infix fun result(block: ResultSplashGenderRobot.() -> Unit) =
ResultSplashGenderRobot().apply { block() }
}
class ResultSplashGenderRobot {
fun isGenderWomenStored() { [...] }
fun isGenderMenStored() { [...] }
}
A variety of extension functions specify assertions:
fun isDisplayed(@IdRes id: Int): ViewInteraction =
onView(withId(id)).check(matches(isDisplayed()))
Also, possible view interactions:
fun click(id: Int, matcher: (Int) -> Matcher<View> = { withId(id) }) {
onView(matcher(id)).click()
}
Some tips for the coding enthusiasts:
- Assertions mark the success or failure of the test. But don’t forget that Unit Testing is meant to be specific. UI Testing is a great way to check on what is presented to the user and not the logic behind it.
- On Android, we found the Robot equals Screen definition very handy . Nevertheless, it is not mandatory. For instance, it is possible to design a single Robot for the whole refine experience.
Focusing on the code for a little longer, Kotlin was key to the adoption of the pattern by the team. In order to empower non-Android members into writing tests, building one must be concise and avoid as much code as possible.
Thanks to Kotlin, scrolling to the fourth position of the Home screen and then navigate and click Shop Women button looks like this:
navigateTo - Home then {
scrollToPosition(4)
scrollToItem(CHANGE_GENDER)
clickItemWithText("Shop Women")
}
Android’s framework combined with Kotlin reduced complexity and enabled the whole team, non developer members included, to expand the test cases.
In addition, Robot structure established one-to-many relationships between them and tests. One Robot, multiple test scenarios.
Cover all possible user’s point of entry in the app
Is there a better occasion than Sales Season to double check our app? Probably not. Any tool that wants to shine at FARFETCH must deal successfully with the most critical period.
Deep-links and push notifications form a vital entry point to the app. Luckily for us, with little effort we were able to grow the functionalities and launch the app’s main context with the needed information to open any deep-link.
@Before
open fun setup() {
[...]
activityRule.launchActivity(
configureSalesLandingPageIntent()
)
}
Once the view interactions are completed, the result infix function evaluates the final state.
salesDeeplink result {
isSaleLandingPage()
isImageWithinBoundaries()
isTitleShown()
areElementsAvailable()
}
The final stage of every Robot determines specific checkpoints to be met. From items displayed, spacing, distribution, etc., to translations or even persisted elements to local storage. The pattern is a guide to construction without outlined boundaries.
Increase tracking reliability, testing tracking
Final use, combining the power of bash scripting and gradle operations.
How to ensure every tracking event is reaching its provider even when the conditions are not the best, such as poor network, or how to detect possible inconsistencies? Checking tracking by running a single walkthrough of the app might confirm that it's placed as intended. Still, running a variety of flows repeatedly is a good way to double check an event.
To achieve this, we incorporated scripting and gradle as allies. Targeting specific test cases invoking gradle commands in the order of the thousands. After the execution, tracking providers must reflect the results.
A specific testing environment, along with a basic .sh script targeting gradle mechanisms, proved that not every solution has to be complex. Reusing already built-in tools made challenges, such as inconsistencies between our providers, feasible to solve.
What about the near future?
uEveryone related to the android world may have heard the term fragmentation a few times. A correlation between multiple manufacturers and Android versions increases the degree of complexity to test apps in every possible combination. The natural evolution of our framework is tackling fragmentation running our tests with Firebase Test Lab.
Firebase’s tools will allow us to run Farfetch Android App on a large selection of devices and APIs. The objective, as always, is to provide the final users with the best shopping experience.
Happy testing!
