Share
Embed
Creating parallelized Android UITest (Appium) environment using Azure, Docker and Android emulator
Agile & Automation Days 2019 #aadays2019
https://aadays.pl/speakers/tadashi-nemoto/
Tadashi Nemoto
October 29, 2019
Transcript
58 slides
-
Introduction • Tadashi Nemoto • QA Automation Engineer /
Mercari, inc. ◦ Selenium / Appium / XCUITest ◦ CI / CD pipeline ◦ Cloud-based test environment tadashi0713 tadashi-nemoto -
Agenda • About Mercari, inc. • About our automation tests
and problems • What is Docker-Android, and Docker-Android in the cloud • Demo • Challenges to be solved -
About February 1st, 2013 Established Tokyo, Sendai, Fukuoka, Palo Alto,
Portland, Boston Offices 1800+ Including subsidiaries Headcount 6 -
allow individuals to buy and sell items and share information
through their phones. We created “Mercari,” a C2C marketplace app where users can use their smartphones to easily buy and sell secondhand goods. By linking sellers who own items to buyers who see the value in these objects, our goal is to provide an easy, fun, and safe and secure service that creates new value and redefines the cycle of consumption. In modern society, where mass production and mass consumption have become commonplace, the value of items that fall into disuse each year is expected to grow to ¥7.6 trillion in Japan alone.* Of these items, there are many that are no longer useful to one person, but could be used by someone else who values them. The Mercari Group aims to use the popularization of smartphones and social media to Create value in a global marketplace where anyone can buy & sell Mission Source: Ministry of Economy, Trade and Industry (April 2018). Estimated total value of items described as “items that became no longer needed over the past year”. Excludes aggregate value of automobiles, motorcycles and mopeds 7 -
What Is Mercari? •Service start: July 2013 •OS: Android, iOS
*Can also use from web browsers •Usage fee: Free *Commission fee for sold items: 10% of the sales price •Regions/languages supported: Base specs for Japan/Japanese Many sellers enjoy having the items they no longer need purchased and used by buyers who need them, and buyers enjoy the feeling of hunting for treasure as they search through unique and diverse items for lucky finds. In addition to buying and selling, users actively communicate through the buyer/seller chat and the “Like” feature. The Mercari app is a C2C marketplace where individuals can easily sell used items. We want to provide both buyers and sellers with a service where they can enjoy safe and secure transactions. Mercari offers a unique customer experience, with a transaction environment that uses the payments Mercari holds in escrow, and simple and affordable shipping options. 8 -
By the Numbers (JP/Full Year) 232 346.8 GMV¹ 490.2 B
In: billion JPY FY 06/2017 FY 06/2018 21.2 33.4 Net Sales 46.2 B In: billion JPY FY 06/2017 FY 06/2018 8.45 10.75 MAU² 13.57 M In: Million people FY 06/2017 FY 06/2018 Source: Internal documents, from FY2018.6 Presentation Material 1. GMV after cancellation 2. Monthly Active Users in June. Number of registered users that used our app in the month 490.2 FY 06/2019 46.2 FY 06/2019 13.57 FY 06/2019 9 -
Building a new ecosystem through Merpay Merpay is a mobile
payments service operated by Merpay, Inc. The company belongs to the Mercari Group, widely recognized for its service “Mercari,” the top C2C marketplace app in Japan. Money earned by selling unwanted items on the Mercari app and money charged to the app through users’ bank accounts can be used to make payments at stores and on the Mercari app itself. Our future goal is to provide a comprehensive financial service by combining primary and secondary distribution based on data, and supporting the business of merchants while utilizing users’ credit information, such as their transaction history and ratings. Primary distribution Manufactures, retail, distributors, etc. Secondary distribution Marketplace app Attract customers Marketing Transaction data Future Business Development Building an Ecosystem -
“The Selling App.” Mercari’s mission is to “Create value in
a global marketplace where anyone can buy & sell,” and the year after our founding, we began to expand overseas. We started our business in the US in April 2014, and we adapted our branding, improved our UI/UX, and created a delivery network to match local preferences and the characteristics of the US market. We recognize that succeeding in the US, with its enormous and diverse population, is an important milestone to achieve our mission, so that is where we are devoting our energy. 11 Future Business Development Strengthening Overseas Expansion To Make Selling Easier than Buying. -
Problems • Increasing test execution time • Maintenance cost of
real devices • Test failures due to differences of real devices (e.g. display size, OS version, etc.) • Maintenance cost of local environments (e.g. Appium, Ruby, etc.) • These problems often made tests flaky and unreliable -
Our headcount tripled in 3 years 1,826 376 462 555
596 712 795 1,014 1,140 1,357 1,639 1,786 Changes in total headcount 1Q 2Q 3Q 4Q FY2016.6 1Q 2Q 3Q 4Q 1Q 2Q 3Q 4Q 1Q 2Q 3Q 4Q FY2017.6 FY2018.6 FY2019.6 from Mercari Bold Challenge -
The number of engineers also increased greatly FY2018 FY2019 FY2017
FY2016 from Mercari Bold Challenge -
Summary • We first created a local environment (local PC,
real devices) to run Android E2E・UI Tests • We faced some problems, and needed faster and more stable QA feedback • Based on the Mobile Testing Pyramid, we tried creating parallelized test environment using Azure, Docker, and Android Emulator -
Docker • Easy to scrap and build • Easily recreate
same environment(local, CI) • Isolated environment • Infrastructure as Code -
Cloud server(IaaS) • Reduces maintenance cost compared to on-premises server
• Easy to scale • Cost determined by usage -
Android Emulator • Can reduce maintenance cost of real devices
• Clean state • Can emulate ◦ SMS ◦ Push notification -
Advantages of Docker-Android • Uses fast Android x86 emulator •
Starts Appium server inside container • Can connect to Selenium Grid easily • Remote connect using noVNC • Can capture videos -
How to run Docker-Android in the cloud • Android
x86 emulator needs hardware virtualization • Can only run limited cloud service / Instances ◦ Bare metal Instance(AWS) ◦ Nested virtualization(Azure) -
Summary • With Docker-Android and Cloud server(Azure), we could create
parallelized testing environments • We can also benefit from using ◦ Docker ◦ Android Emulator ◦ Cloud server -
Challenges to be solved • Test report • Optimise distributed
tests • Server load on QA environment • Divide role of emulators and real devices -
Rspec HTML Reporter • Problems ◦ Difficult to find history
and trend ◦ Generate reports by number of parallel -
• Solution ◦ Use mock server instead ◦ Improve QA
environment infracture Server load to QA Environments API server (QA) -
Summary • We still need to solve the below challenges
for faster and more stable QA feedback ◦ Test report ◦ Optimise distributed tests ◦ Server load on QA environment ◦ Divide roles of emulators and real devices -
Message(Key takeaways) • We faced problems in testing environments, and
need faster, more stable QA feedback • Following the Mobile Testing Pyramid, we tried creating parallelized test environment using Azure, Docker, and Android Emulator • There are challenges to be solved apart from the test environment for faster and more stable QA feedback