Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Conference for Kotliners - Kotlin: the Next Fro...

Conference for Kotliners - Kotlin: the Next Frontier in Metaprogramming

In an epic quest using Kotlin and native desktop development to generate UI testing for the TornadoFX framework, we turn to metaprogramming in Kotlin to explore AST language parsing and the philosophy of design-driven UI testing in the pursuit for smarter test-generation.

mvndy_hd

June 07, 2019
Tweet

More Decks by mvndy_hd

Other Decks in Technology

Transcript

  1. Kotlin: The Next Frontier in Metaprogramming Using metaprogramming to formalize

    the theory of human error in applications in generative UI testing Amanda Hinchman-Dominguez @hinchman_amanda
  2. UIs increasingly critical to the success of applications • An

    application’s GUI must create a rich, intuitive and pleasant user experience • Validate + guarantee quality assurance of event-driven applications through the active process of exploring an environment
  3. Black box testing • Tests for the behavior/ functionality of

    software White box testing • Tests how the system functions E2E Integration Unit Quality Assurance
  4. Black box testing • Tests for the behavior/ functionality of

    software White box testing • Tests how the system functions E2E Integration Unit Quality Assurance
  5. E2E Integration Unit Black box testing • Tests for the

    behavior/ functionality of software White box testing • Tests how the system functions UI Testing Quality Assurance
  6. A UI Test goes something like this… Grab a particular

    view Perform action on view Check UI to see if it looks as it should
  7. A UI Test goes something like this… Grab a particular

    view Perform action on view Check UI to see if it looks as it should
  8. • Kastree - a simple wrapper around the Kotlin compiler

    • Program Structure Interface (PSI): the layer in IntelliJ parses syntactic and semantic code models • An Abstract Syntax Tree (AST) parser surpasses challenges found in statically-typed metaprogramming: • Access levels • Type casting in recursion Phase 1: Detecting Inputs w/ AST Parsing
  9. override val root = hbox { form { fieldset("Edit person")

    { field("Owner") { textfield(model.ownerName) { ownerNameField = this } } field("Cat") { textfield(model.catName) { catNameField = this } } field("Time") { textfield(model.time) { timeField = this } } button("Save") { enableWhen(model.dirty) action { save() } } } } } Saving UI Nodes field fieldset form textfield field textfield field field textfield field fieldset form hbox textfield hbox field textfield textfield button button
  10. A UI Test goes something like this… Grab a particular

    view Perform action on view Check UI to see if it looks as it should
  11. A UI Test goes something like this… Grab a particular

    view Perform action on view Check UI to see if it looks as it should
  12. • Kotlin scripting to write Kotlin test files for node

    interactions • TestFX robots to stimulate user interactions, check node states with matchers • Dynamically insert randomized ids for detected nodes at runtime Phase 2: Generating Tests with TestFX
  13. • Kotlin scripting to write Kotlin test files for node

    interactions • TestFX robots to stimulate user interactions, check node states with matchers • Dynamically insert randomized ids for detected nodes at runtime Phase 2: Generating Tests with TestFX
  14. A UI Test goes something like this… Grab a particular

    view Perform action on view Check UI to see if it looks as it should
  15. How do we know if a node looks as it

    should? • Mapping a compositional abstraction for our components can help define properties surrounding actions of our views • When a control event is triggered, apply binary operations against states and check against the contract holistic result of an application’s representative compositional map
  16. Phase 3: Abstract Compositional Contracting • Check for the state

    of the node after an interaction • Map nodes to associated functions • Create function compositions from node interactions • Check for models attached to views • Model support • Scope/Context support
  17. Phase 3: Abstract Compositional Contracting • Check for the state

    of the node after an interaction • Map nodes to associated functions • Create function compositions from node interactions • Check for models attached to views • Model support • Scope/Context support
  18. Phase 3: Abstract Compositional Contracting • Check for the state

    of the node after an interaction • Map nodes to associated functions • Create function compositions from node interactions • Check for models attached to views • Model support • Scope/Context support
  19. button form hbox override val root = hbox { form

    { fieldset("Edit person") { field("Owner") { textfield(model.ownerName) { ownerNameField = this } } field("Cat") { textfield(model.catName) { catNameField = this } } field("Time") { textfield(model.time) { timeField = this } } button("Save") { enableWhen(model.dirty) action { save() } } } } } fieldset form fieldset field field field field button textfield field textfield textfield textfield field textfield textfield field textfield Post-parsing analysis
  20. UINode( nodeType = “Button”, uiLevel = 3, nodeChildren = JsonObject(),

    valueAssignment = “”, functionComposition = Digraph<UIFunctionComposition>(), associatedFunctions = ArrayList<String>() ) button Post-parsing analysis
  21. UINode( nodeType = “Button”, uiLevel = 3, nodeChildren = JsonObject(),

    valueAssignment = “”, functionComposition = Digraph<UIFunctionComposition>(), associatedFunctions = ArrayList<String>() ) button Post-parsing analysis
  22. avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { rectangle { width = 200.0 height = 200.0 fill = c("#AARRGGBB") opacity = 0.01 } imageview(catSchedule.catImage) } view.avi.children.add(image) } fun editCatSchedule(catSchedule: CatSchedule) { val catScheduleScope = CatScheduleScope() catScheduleScope.model.item = catSchedule find(Editor::class, scope = catScheduleScope).openModal() } } Associated Functions
  23. avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { } view.avi.children.add(image) } fun editCatSchedule(catSchedule: CatSchedule) { val catScheduleScope = CatScheduleScope() catScheduleScope.model.item = catSchedule find(Editor::class, scope = catScheduleScope).openModal() } } Associated Functions
  24. avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { } view.avi.children.add(image) } fun editCatSchedule(catSchedule: CatSchedule) { val catScheduleScope = CatScheduleScope() catScheduleScope.model.item = catSchedule find(Editor::class, scope = catScheduleScope).openModal() } } Associated Functions
  25. avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { } view.avi.children.add(image) } fun editCatSchedule(catSchedule: CatSchedule) { val catScheduleScope = CatScheduleScope() catScheduleScope.model.item = catSchedule find(Editor::class, scope = catScheduleScope).openModal() } } Associated Functions
  26. avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { } view.avi.children.add(image) } fun editCatSchedule(catSchedule: CatSchedule) { val catScheduleScope = CatScheduleScope() catScheduleScope.model.item = catSchedule find(Editor::class, scope = catScheduleScope).openModal() } } Associated Functions
  27. avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { } view.avi.children.add(image) } fun editCatSchedule(catSchedule: CatSchedule) { val catScheduleScope = CatScheduleScope() catScheduleScope.model.item = catSchedule find(Editor::class, scope = catScheduleScope).openModal() } } Associated Functions
  28. avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { } view.avi.children.add(image) } fun editCatSchedule(catSchedule: CatSchedule) { val catScheduleScope = CatScheduleScope() catScheduleScope.model.item = catSchedule find(Editor::class, scope = catScheduleScope).openModal() } } Associated Functions
  29. avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { } view.avi.children.add(image) } fun editCatSchedule(catSchedule: CatSchedule) { val catScheduleScope = CatScheduleScope() catScheduleScope.model.item = catSchedule find(Editor::class, scope = catScheduleScope).openModal() } } Associated Functions
  30. avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { } view.avi.children.add(image) } fun editCatSchedule(catSchedule: CatSchedule) { val catScheduleScope = CatScheduleScope() catScheduleScope.model.item = catSchedule find(Editor::class, scope = catScheduleScope).openModal() } } Associated Functions
  31. avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { } view.avi.children.add(image) } fun editCatSchedule(catSchedule: CatSchedule) { val catScheduleScope = CatScheduleScope() catScheduleScope.model.item = catSchedule find(Editor::class, scope = catScheduleScope).openModal() } } Associated Functions
  32. avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { rectangle { width = 200.0 height = 200.0 fill = c("#AARRGGBB") opacity = 0.01 } imageview(catSchedule.catImage) } view.avi.children.add(image) } fun editCatSchedule(catSchedule: CatSchedule) { val catScheduleScope = CatScheduleScope() catScheduleScope.model.item = catSchedule find(Editor::class, scope = catScheduleScope).openModal() } } Associated Functions
  33. avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { rectangle { width = 200.0 height = 200.0 fill = c("#AARRGGBB") opacity = 0.01 } imageview(catSchedule.catImage) } view.avi.children.add(image) } fun editCatSchedule(catSchedule: CatSchedule) { val catScheduleScope = CatScheduleScope() catScheduleScope.model.item = catSchedule find(Editor::class, scope = catScheduleScope).openModal() } } Associated Functions
  34. avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { rectangle { width = 200.0 height = 200.0 fill = c("#AARRGGBB") opacity = 0.01 } imageview(catSchedule.catImage) } view.avi.children.add(image) } fun editCatSchedule(catSchedule: CatSchedule) { val catScheduleScope = CatScheduleScope() catScheduleScope.model.item = catSchedule find(Editor::class, scope = catScheduleScope).openModal() } } Associated Functions
  35. avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { rectangle { width = 200.0 height = 200.0 fill = c("#AARRGGBB") opacity = 0.01 } imageview(catSchedule.catImage) } view.avi.children.add(image) } fun editCatSchedule(catSchedule: CatSchedule) { val catScheduleScope = CatScheduleScope() catScheduleScope.model.item = catSchedule find(Editor::class, scope = catScheduleScope).openModal() } } Associated Functions
  36. avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { rectangle { width = 200.0 height = 200.0 fill = c("#AARRGGBB") opacity = 0.01 } imageview(catSchedule.catImage) } view.avi.children.add(image) } fun editCatSchedule(catSchedule: CatSchedule) { val catScheduleScope = CatScheduleScope() catScheduleScope.model.item = catSchedule find(Editor::class, scope = catScheduleScope).openModal() } } Associated Functions
  37. avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { rectangle { width = 200.0 height = 200.0 fill = c("#AARRGGBB") opacity = 0.01 } imageview(catSchedule.catImage) } view.avi.children.add(image) } fun editCatSchedule(catSchedule: CatSchedule) { val catScheduleScope = CatScheduleScope() catScheduleScope.model.item = catSchedule find(Editor::class, scope = catScheduleScope).openModal() } } Associated Functions
  38. avi class BottomViewController: Controller() { private val view: BottomView by

    inject() fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { rectangle { width = 200.0 height = 200.0 fill = c("#AARRGGBB") opacity = 0.01 } imageview(catSchedule.catImage) Associated Functions
  39. button("Save") { enableWhen(model.dirty) action { save() update(model) } } Functional

    composition - what functions are spurned from interacting with a particular node?
  40. Function Composition button("Save") { enableWhen(model.dirty) action { save() update(model) }

    } private fun save() { model.commit() val catSchedule = model.item close() } private fun update(model: CatScheduleModel) { println("Contact updated!") controller.changeCatAvi(model.item) } save save action fun changeCatAvi(catSchedule: CatSchedule) { view.avi.children.clear() val image = StackPane().apply { rectangle { width = 200.0 height = 200.0 fill = c("#AARRGGBB") opacity = 0.01 } imageview(catSchedule.catImage) } view.avi.children.add(image) } changeCatAvi changeCatAvi update update
  41. Phase 3: Abstract Compositional Contracting • Check for the state

    of the node after an interaction • Map nodes to associated functions • Create function compositions from node interactions • Check for models attached to views • Model support • Scope/Context support
  42. Creating Finite State Machines to predict node behavior Cat Schedule

    Model Save Button Dirty Not Dirty Disabled Enabled
  43. Creating Finite State Machines to predict node behavior Cat Schedule

    Model Save Button Dirty Not Dirty Disabled Enabled
  44. Cat Schedule Model Save Button Dirty Not Dirty Disabled Enabled

    enum class Toggle { Enable, Disable; fun isButtonEnabled(): Boolean { return when (this) { Enable -> true Disable -> false } } fun enableOnDirty(model: ModelState): Toggle { return when (model) { ModelState.Dirty -> Enable ModelState.NotDirty -> Disable } } } enum class ModelState { Dirty, NotDirty; fun initModelStateTransition(): ModelState { return when (this) { Dirty -> NotDirty NotDirty -> Dirty } } }
  45. sealed class EditorFragment { var modelState = ModelState.NotDirty fun commitModel()

    {} fun onChangeText() { modelState.initModelStateTransition() } open class OwnerTextField: EditorFragment() {} open class CatTextField: EditorFragment() {} open class TimeTextField: EditorFragment() {} class SaveButton(var toggle: Toggle = Toggle.Disable): EditorFragment() { fun toggleButtonOnDirtyModel(model: ModelState) { toggle.enableOnDirty(model) } fun saveModel() { if (toggle.isButtonEnabled()) commitModel() } } }
  46. sealed class EditorFragment { var modelState = ModelState.NotDirty fun commitModel()

    {} fun onChangeText() { modelState.initModelStateTransition() } open class OwnerTextField: EditorFragment() {} open class CatTextField: EditorFragment() {} open class TimeTextField: EditorFragment() {} class SaveButton(var toggle: Toggle = Toggle.Disable): EditorFragment() { fun toggleButtonOnDirtyModel(model: ModelState) { toggle.enableOnDirty(model) } fun saveModel() { if (toggle.isButtonEnabled()) commitModel() } } }
  47. Specifying UIs Formally through Metaprogramming Benefits: • Implicit: Clearly expressing

    ourselves leads to a better understanding of the system • Explicit: Using formal specs to check properties - does our UI have any dead ends? Can we get locked out of a chunk of the UI?
  48. Promising discoveries in metaprogramming analysis research • Metaprogramming locator mechanism

    is more accurate/less intensive detecting UI components in applications than current AI locator mechanisms • Current AI image classification locator mechanisms suffer from partial view render recognition, detecting custom object interactions
  49. Promising discoveries in metaprogramming analysis research • Using Kotlin metaprogramming

    to further analyze the relationship of UI hierarchies, functional composition, and generated test results • Potential for generating unit testing in the future • Traditional machine learning only offers immediate extrinsic reward analysis; metaprogramming generated relation analysis may provide more in-depth analysis to supplement future deep learning
  50. Future Plans for TornadoFX-Suite • DSLs for more flexible test

    building + abstracted compositional mapping • Make AST parsing stronger + pull out into an independent library • Expand TornadoFX-Suite to analyze other UI frameworks • Collect voluntary data from future users to begin deep learning
  51. • TornadoFX-Suite: https://github.com/ahinchman1/TornadoFX-Suite • Automata UI research: https://github.com/ahinchman1/Finite-State-Machine-Crash-Course • Data

    science research: https://github.com/ahinchman1/Data-Science-Crash-Course • Neighborhood Cat Scheduler: https://github.com/Kotlin-Thursdays/Neighborhood-Cat-Scheduler • TornadoFX-DnD-TilesFX: https://github.com/ahinchman1/TornadoFX-DnD-TilesFX Amanda Hinchman-Dominguez @hinchman_amanda Relevant Links