Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Exploring Neural Networking with Kotlin Metapro...

mvndy_hd
November 15, 2019

Exploring Neural Networking with Kotlin Metaprogramming: A Cheaper Alternative to Deep Learning?

Presented at JFuture 2019.

Relevant Links:
- TornadoFX-Suite: https://github.com/ahinchman1/TornadoFX-Suite
- Automata UI research: https://github.com/ahinchman1/Finite-State-Machine-Crash-Course
- Data science research: https://github.com/ahinchman1/Data-Science-Crash-Course

mvndy_hd

November 15, 2019
Tweet

More Decks by mvndy_hd

Other Decks in Technology

Transcript

  1. Exploring Neural Networking with Kotlin Metaprogramming: A Cheaper Alternative to

    Deep Learning? Amanda Hinchman-Dominguez @hinchman_amanda
  2. UIs increasingly critical to the success of applications • An

    application’s GUI must create a rich, intuitive and pleasant user experience • Validate + guarantee quality assurance of event-driven applications through the active process of exploring an environment
  3. Black box testing • Tests for the behavior/ functionality of

    software White box testing • Tests how the system functions E2E Integration Unit Quality Assurance
  4. Black box testing • Tests for the behavior/ functionality of

    software White box testing • Tests how the system functions E2E Integration Unit Quality Assurance
  5. E2E Integration Unit Black box testing • Tests for the

    behavior/ functionality of software White box testing • Tests how the system functions UI Testing Quality Assurance
  6. Can AI truly automate testing to validate the human experience?

    • Automation should be viewed as a technique, not the design activity itself • Like many problems that require a human process, people who don’t understand the value of testing (or doesn’t want to do it) turns it into a programming problem • AI can’t answer “understand the value of an application from interacting with its environment”
  7. Metaprogramming neural networking solution can serve as a suitable option

    for automated techniques in testing • more efficient • require less data for analysis • far fewer unknowns with internal operations
  8. • The neural network itself is called the model, but

    it can also be recognized as an algorithm (pixel math). • In this sense, the neural networking is act of moving data through different layers containing certain filters. • Creates the model to achieve the same goal • The relationships that are drawn, the patterns that are created, in order to solve the same problems AI does is the neural network Neural Networking in Deep Learning Neural Networking in Metaprogramming
  9. What kind of problems does neural networking solve for automated

    testing? Supervised Learning: Includes target value Regression: Target is continuous Classification: Target is discrete
  10. Metaprogramming Locator Mechanism field fieldset form textfield field textfield field

    field textfield field fieldset form hbox textfield hbox field textfield textfield button button AI Locator Mechanism Filt + Max pooling through dense layers Locating UI Nodes with Classification
  11. • Classification - How lines are drawn around data points

    • Convolutional Neural Networking (CNN) model - Most popular for analyzing images • Matrix math to flatten pixel data as they’re passed through hidden convolutional layers & non- convolutional layering Image classification - locator mechanism
  12. • Draws lines about the pixel, no knowledge of code

    required • Reduces image pixels through filters, but provides lots of intense matrix calculations that are opaque to us due to the # of calculations required per layer Image classification - locator mechanism "Komandyseja | Music Production" by Przemek Bizoń, bisoñ studio, Agata Łobaczuk is licensed under CC BY-NC-ND 4.0 - Modified image.
  13. • Draws lines about the pixel, no knowledge of code

    required • Reduces image pixels through filters, but provides lots of intense matrix calculations that are opaque to us due to the # of calculations required per layer Image classification - locator mechanism "Komandyseja | Music Production" by Przemek Bizoń, bisoñ studio, Agata Łobaczuk is licensed under CC BY-NC-ND 4.0 - Modified image. TextView InteractivePlayerView Floating Action Button ScrollView RelativeLayout Linear Layout
  14. • Abstract Syntax Tree (AST) parser draws lines around the

    shape of the written language parsed into an intermediate representation of an AST The Metaprogramming Locator Mechanism
  15. • Abstract Syntax Tree (AST) parser draws lines around the

    shape of the written language parsed into an intermediate representation of an AST The Metaprogramming Locator Mechanism
  16. • Abstract Syntax Tree (AST) parser draws lines around the

    shape of the written language parsed into an intermediate representation of an AST The Metaprogramming Locator Mechanism
  17. • Abstract Syntax Tree (AST) parser draws lines around the

    shape of the written language parsed into an intermediate representation of an AST The Metaprogramming Locator Mechanism
  18. • Abstract Syntax Tree (AST) parser draws lines around the

    shape of the written language parsed into an intermediate representation of an AST The Metaprogramming Locator Mechanism
  19. • Abstract Syntax Tree (AST) parser draws lines around the

    shape of the written language parsed into an intermediate representation of an AST The Metaprogramming Locator Mechanism
  20. • Abstract Syntax Tree (AST) parser draws lines around the

    shape of the written language parsed into an intermediate representation of an AST The Metaprogramming Locator Mechanism
  21. • Abstract Syntax Tree (AST) parser draws lines around the

    shape of the written language parsed into an intermediate representation of an AST The Metaprogramming Locator Mechanism
  22. • Abstract Syntax Tree (AST) parser draws lines around the

    shape of the written language parsed into an intermediate representation of an AST The Metaprogramming Locator Mechanism
  23. • Abstract Syntax Tree (AST) parser draws lines around the

    shape of the written language parsed into an intermediate representation of an AST The Metaprogramming Locator Mechanism
  24. • Abstract Syntax Tree (AST) parser draws lines around the

    shape of the written language parsed into an intermediate representation of an AST The Metaprogramming Locator Mechanism
  25. • Abstract Syntax Tree (AST) parser draws lines around the

    shape of the written language parsed into an intermediate representation of an AST The Metaprogramming Locator Mechanism
  26. • Abstract Syntax Tree (AST) parser draws lines around the

    shape of the written language parsed into an intermediate representation of an AST The Metaprogramming Locator Mechanism
  27. override val root = hbox { form { fieldset("Edit person")

    { field("Owner") { textfield(model.ownerName) { ownerNameField = this } } field("Cat") { textfield(model.catName) { catNameField = this } } field("Time") { textfield(model.time) { timeField = this } } button("Save") { enableWhen(model.dirty) action { save() } } } } } field fieldset form textfield field textfield field field textfield field fieldset form hbox textfield hbox field textfield textfield button button The Metaprogramming Locator Mechanism
  28. Metaprogramming Locator Mechanism field fieldset form textfield field textfield field

    field textfield field fieldset form hbox textfield hbox field textfield textfield button button AI Locator Mechanism Filt + Max pooling through dense layers Locating UI Nodes with Classification
  29. Metaprogramming Locator Mechanism field fieldset form textfield field textfield field

    field textfield field fieldset form hbox textfield hbox field textfield textfield button button AI Locator Mechanism O(mn3) Filt + Max pooling through dense layers Locating UI Nodes with Classification O(n)
  30. Future work: Regression analysis Supervised Learning: Includes target value Regression:

    Target is continuous Classification: Target is discrete http://testerstories.com/2018/08/can-an-ai-become-a-tester/
  31. • Regression - How lines draw relationships from one point

    to another • How can we increase combinations of permutations of interactions? Future work: Making UI tests “smarter”
  32. Recap • Using automation can help the tester, but doesn’t

    replace the role of a test • Neural networking in metaprogramming is isomorphic to AI neural networking • AI models for regression and classification is expensive and difficult to maintain and wrangle • You don’t need a sledgehammer for a nail Supervised Learning: Regression Classification
  33. • TornadoFX-Suite: https://github.com/ahinchman1/TornadoFX-Suite • Automata UI research: https://github.com/ahinchman1/Finite-State-Machine-Crash-Course • Data

    science research: https://github.com/ahinchman1/Data-Science-Crash-Course Amanda Hinchman-Dominguez @hinchman_amanda Relevant Links