Test hybrid app with Appium and Selenium
I recently had a discussion with a coworker whether the E2E functional test written in Selenium for hybrid apps should be migrated to Appium since the test subject is an app.
In our context, the hybrid app is the app mainly consists of web views. The whole functionality and user interactions happen in the web view, and the app is just a container for the web.
I found this discussion quite interesting so I further discovered more with a demo. I summarized my findings in this GitHub repository .
In my demo, I firstly wrote a simple Selenium test for the homepage of Thoughtworks. Then I made a hybrid app which is simply a WKWebView covering the full screen, showing the same webpage. I ported the Selenium test to Appium and performed nearly identical test as Selenium.
From my perspective, I don’t believe going straight to Appium simply because hybrid app is an app is a good idea. Although Selenium and Appium shared the same foundation of WebDriver protocol and are highly interoperable, they best serve different scenarios. I do believe Appium is more suitable for native apps with the power of leveraging accessibility locator, test-id etc, and Selenium is better to test webpages rendered as a HTML DOM no matter it’s presented in a browser or mobile web view component.
In terms of the ease of migration process, it is not extremely hard per se, but the efforts to optimize the test (e.g. changing to more reliable mobile locator) and the limitation of interaction for the web rendered mobile elements (e.g. getText() from the child) overweigh the value it may brings (i.e. closer to the user in terms of the device type).
In my opinion, if the goal of functional test is to make sure users can go through the major user paths, the sensible test strategy for the hybrid app would be:
- Unit test: verify the functionality of each unit
- Functional test: verify user journeys are achievable on the web
- Application test: verify the webpage on load correctly in the app
The details of each layer and in which granularity we hope to verify may vary depending on the requirement, but in principle it would be more stable and efficient to test the web as a web, and test an app as an app.
You are welcome to share your thoughts with me. I would love to hear your opinion.