How to build mobile farm with a help of Selenium Grid and Robot Framework
In this article I’m going to share my experience of setting up mobile farm for running automation tests in simple steps.
TrueMoney team is using Robot Framework with Appium library for testcase automation for Android and iOs platforms. QA engineers run tests on personal PC and use one mobile device per time.
This leads to these problems:
1 Regression test run takes lots of time.
2 Regression test run requires QA engineer involvement.
3 Personal PC is occupied for regression testing.
4 History of test report is stored on a personal PC.
5 Releases of mobile applications need to be downloaded and installed on device manually.
6 Executor runs tests in one thread.
7 Executor runs tests on one device configuration.
Solution:
To setup a separate PC for regression testing. To create pipeline for test run. To find the way to run tests in parallel. To find the way to pick device configuration. To send report in test report system.
Step 1. To get hardware
We ordered a Mac mini (2018) with Processor 3.2 GHz 6-Core Intel Core i7, Memory 32 GB 2667 MHz DDR4 which has 2 USB ports, 4 USB-c ports. This allows to connect 6 mobile phones.
Step 2. To setup Pipeline.
For creating pipeline I chose Jenkins with plugins and installed it as a Docker container for faster update in future. Jenkins jobs Icreated by seed DSL script. Pipeline script i wrote on Groovy in declarative pipeline syntax.
Simplified pipeline design looks like this:
Pipeline can download mobile application from different sources, run test, store result locally on Jenkins and send it into internal testcase management system.
Step 3. To find tools to run tests in parallel.
Mobile phone supports only one instance of mobile application. It means that for running tests in parallel we need to have several devices. This will allow to split tests between devices.
After a research, I decided to implement the solution with Selenium Grid. For that you need to register devices by Appium in Selenium-Grid hub. Then you can pick device for a run by passing device configuration and hub will automatically pick available device for a test run. My solution supports such device configuration as platform name: Android or iOS, OS version number and unique device name.
Robot framework has a parallel executor — pabot. Amount of thread can be controlled by parameter “ — processes”.
Step 4. To write scripts
I developed two scripts. First script is responsible for automatic scanning and device registration in Selenium Grid hub. Second script is responsible for setting up test libraries and running tests. For this scripts I used python and shell.
Solution diagram:
Pipeline with configuration parameters:
Result:
By implementation of the device lab I freed QA engineers from an additional work of running regression tests on local PC.
I automated repetitive work of downloading and installing the latest mobile app version on devices, sending test report to the system.
Also, I decreased regression run time in N time where N is amount of connected mobile devices. For example, run of 10 testcases on 1 test device takes 41min 04s. Test run on 4 devices in parallel will take 12min 19s. So, the more devices included into a one run — the faster we get test result.
In my next articles, I’m going to explain steps in details.
Follow up!
— — — — — — — — — — — — — — — — — — — — — — — — — —
Special thanks for Nuttasate Watcharanurak for ordering MacMini and Anton Malinskiy for consultation about mobile farm and tools.
