Upgrade to Pro — share decks privately, control downloads, hide ads and more …

From code to no-code. How MagicPod and its AI e...

MagicPod
October 11, 2024

From code to no-code. How MagicPod and its AI evolve test automation with Selenium

The slide for https://conference.idstb.org/ held on Oct 10, 2024.

MagicPod

October 11, 2024
Tweet

More Decks by MagicPod

Other Decks in Technology

Transcript

  1. CEO of MagicPod, a Japanese test automation tool company The

    organizer of the Japan Selenium User Community Author of a Japanese Selenium book About me Nozomi Ito
  2. No-code, cloud-based test automation SaaS For browser and mobile app

    testing Simple UI but with powerful features Free plan and paid plan Trusted by more than 500 companies No-code test automation tools
  3. Agenda 1. My journey from Selenium to MagicPod 2. How

    MagicPod leverages Selenium internally 3. Values MagicPod adds to Selenium 4. Utilization of AI 5. MagicPod’s goal for the future of no-code
  4. Agenda 1. My journey from Selenium to MagicPod 2. How

    MagicPod leverages Selenium internally 3. Values MagicPod adds to Selenium 4. Utilization of AI 5. MagicPod’s goal for the future of no-code
  5. Established Selenium company (2012) Worked on consulting/support for test automation

    with Selenium in Japan Selenium: Open source browser automation framework
  6. Promoted Selenium in Japan Launched Japan Selenium User community (from

    PublicKey) Published "Selenium Practical introduction" Held periodical Selenium workshops Organized SeleniumConf Tokyo
  7. Challenges during the promotion Many Japanese companies started test automation

    with Selenium But large amount of Japanese companies still prefer to manual test
  8. Why they did not automate E2E tests Lack motivation and

    knowedge about Selenium Cost of maintaining E2E test was too high In Japan, most QAs don't have technical background. Lack skills to use Selenium. Even commercial tools (Katalon, Ranorex, etc.) were too difficult for them QAs Engineers
  9. From Selenium to no-code (2017) I decided to create MagicPod

    to spread E2E test automation No-code tool using Selenium/Appium internally Cloud service with easy setup Self healing AI to reduce maintenance costs Concepts
  10. Many industrial experts joined MagicPod Authors of test automation books

    Translators of test automation books Appium/Selenium conf presenters Selenium/Appium contributors/committers
  11. Agenda 1. My journey from Selenium to MagicPod 2. How

    MagicPod leverages Selenium internally 3. Values MagicPod adds to Selenium 4. Utilization of AI 5. MagicPod’s goal for the future of no-code
  12. Internal mechanism - test creation UI scan logic Analyze Server

    Upload Detect UI image UI tree XML human-readable caption locator
  13. Internal mechanism - test creation [ { "command": "input", "element":

    "\"Name\" input area", "value": "Demo user" }, ... ] Test script Database Drag & Drop Store A command is added depending on the dropped item Scan result
  14. Internal mechanism - test execution driver.findElementbyName ('"Name" input area') .sendKeys("Demo

    user"); ... driver.findElementbyName ('"Name" input area') .sendKeys("Demo user"); ... driver.findElementbyName ('"Name" input area') .sendKeys("Demo user"); ... Server JSON Node.js × Mocha × Selenium(or Appium) script Download Convert Run Test result Upload Cloud device Database [ { "command": "input", "element": "\"Name\" input area", "value": "Demo user" }, ... ]
  15. System structure of MagicPod Run Run MagicPod has 2 types

    of execution methods. Server Local Local Server Database Database Cloud Browser device Local Browser device Cloud test Local test Script Script MagicPod Desktop
  16. Agenda 1. My journey from Selenium to MagicPod 2. How

    MagicPod leverages Selenium internally 3. Values MagicPod adds to Selenium 4. Utilization of AI 5. MagicPod’s goal for the future of no-code
  17. Sophisticated locator calculation Order of locator recommendation Unique id data-testid

    → id → name → text() → alt → value → title → aria-label → placeholder → class: id = userName tag name + unique id xpath = //button[@id='log in'] label xpath = //input[@id=//label[text()='User name']/@for]] tag name xpath = //input[2] child xpath = //div[@id='mainArea']/button[1] parent xpath = //span[text()='log in']/parent::button[1] sibling xpath = //div[text()='Name']/following-sibling::input[1] full path xpath xpath = /body[1]/div[1]/div[2]/div[3]/a
  18. Sophisticated locator calculation Random unique identifiers have low priority Use

    statistical approach to judge "random" xpath=//button[text()='Sign up'] xpath=//button[2] xpath=//div[@id='mainForm']/div[1]/button[1] id=577656711eee44a
  19. Automatic wait and retry fdffff Selenium Various explicit wait commands

    Automtically wait for elements to appear with implictWait driver.manage().timeouts() .implicitlyWait( Duration.ofSeconds(2));
  20. Automatic wait and retry MagicPod Explicit wait commands like selenium

    More sophisticated implicit wait Wait for element position to be settled like Playwright Retry for stale element error fdffff WebElement e = driver.findElement( By.id("logIn")); e.click();
  21. Powerful features without code Powerful commands resembling code-based testing Web

    api call 2FA code calculation Regular expression match Date calculation
  22. Seamless frame and webview handling driver.switchTo().frame( driver.findElement( By.id("iframe1"))); driver.findElement( By.id("OK")).click();

    Automatically calculate which context to switch Frame for web test and webview for native mobile app test Selenium MagicPod “OK” Button id=OK frame: iframe1
  23. Agenda 1. My journey from Selenium to MagicPod 2. How

    MagicPod leverages Selenium internally 3. Values MagicPod adds to Selenium 4. Utilization of AI 5. MagicPod’s goal for the future of no-code
  24. 1. Human-readable UI element captions Human-readable caption ・Calculated by AI

    ・Not used by the test runner Selenium/Appium locator ・Calculated from UI tree ・Used by the test runner
  25. 1. Human-readable UI element captions Information on the UI tree:

    attributes, texts, tag names, etc. Parent-child relation between elements
  26. 1. Human-readable UI element captions OCR result (Recognize characters in

    the image) Object recognition by the deep learning engine
  27. 2. Self-healing When a test fails to find a UI

    element due to a change in the application’s page structure, AI modifies the script according to the page change.
  28. 2. Self-healing Click “register” Test script name attribute updated! What

    happens if it is executed? name=register name=registerButton
  29. 2. Self-healing The AI engine updates script and the test

    passes The user is notified of the update detail
  30. 3. Summarization of test script Navigate to URL https://hotel.testplanisphere.dev/en-US/signup.html //

    Log out once if being logged in If UI element "Logout" button is visible Click "Logout" button Navigate to URL https://hotel.testplanisphere.dev/en-US/signup.html End branch // Sing up by using an unique email address Store unique value generated based on the current time to variable UNIQUE_KEY Input test-user${UNIQUE_KEY}@example.com to "Email" input area Input ${PASSWORD} to "Password" input area Input ${PASSWORD} to "Password(confirmation)" input area Input Test user to "Name" input area Input Tokyo, Japan to "Address" input area Input 00011122233 to "Tel" input area Select I do not answer. from "Gender" combo box Input 2000-07-04 to "Date of birth" input area Check "Receive notification" checkbox Click "Sign up" button // Assert values set on the previous page are correctly displayed Assert value of UI element Email value equals test-user${UNIQUE_KEY}@example.com Assert value of UI element Name value equals Test user Assert value of UI element Membership value equals Premium Assert value of UI element Address value equals Tokyo, Japan Assert value of UI element Tel value equals 00011122233 Assert value of UI element Gender value equals not answered Assert value of UI element Date of birth value equals July 4, 2000 Assert value of UI element Notification value equals received // Confirm that the user is successfully created and log-in by the user succeeds Click "Logout" button Click Login Input test-user${UNIQUE_KEY}@example.com to "Email" input area Input ${PASSWORD} to "Password" input area Click "Login" button Assert UI element "MyPage" header exists Human-readable text The script "Check that signup works correctly" automates end-to-end testing for user signup on a hotel website: 1. It ensures logout if the user is already logged in. 2. Fills out the signup form with unique details like email, password, name, etc. 3. Submits the form and verifies entered data's correctness on the confirmation page. 4. Confirms successful signup by logging out and logging back in, checking for the "MyPage" header's presence. Test script Summary MagicPod public API Generative AI
  31. 3. Summarization of test script Prompt Create a short summary

    (less than 300 characters) of the following browser testing user flow briefly in English: [Human-readable test script ]
  32. Agenda 1. My journey from Selenium to MagicPod 2. How

    MagicPod leverages Selenium internally 3. Values MagicPod adds to Selenium 4. Utilization of AI 5. MagicPod’s goal for the future of no-code
  33. Integration of code and no-code Encourage collaboration between engineers and

    non-engineers Short-term goal of MagicPod Mutually convertible LaunchApp() input(e('register', '"name" input area'), 'Yamada Taro') input(e('register', '"gender" input area'), 'Male') tap(e('register', '"Register" button'))
  34. Execute tests directly from test cases designed for manual testing.

    Next major goal: creating test cases entirely from scratch. Log in with [email protected] / pass001 Long-term goal of MagicPod