Exploring the Testing Hyperpyramid with Kotlin & http4k

We all know that testing is an important factor in software development.

But.

What if your tests could also work for you in dimensions that you didn’t even know existed? What if they could give you superpowers? Superpowers like easily running an entire bank on a single developer workstation, generating visual documentation of all the interactions across your fleet of microservices with a simple test plugin, or building reusable infrastructure which allow you to test your entire codebase at the unit, integration or end-to-end level.

In this talk, you'll learn about why your design and technology choices matter for testing and some Kotlin and http4k techniques that allow you to do much more by actually writing less test code.

Transcript

1. Exploring the Testing Hyperpyramid with Kotlin & http4k David Denton | Ivan Sanchez KotlinConf 2023
3. What we’re told about the Testing Pyramid Hard, slow, flakey
4. We think it’s more like *this! *Probably not what a real hyperpyramid looks like
5. What makes a good test?
6. Need to master: - Time - Randomness - Concurrency - Networking - Known state/data Barriers for good testing
7. Our example system: eCommerce Website ACTOR SERVICE CLOUD EXTERNAL
8. Technique: Layering with hexagons
9. Technique: Server as a function (Req) -> Rsp (Req) -> Rsp
10. How http4k implements Server as a Function typealias HttpHandler = (Request) -> Response
11. Technique: Events aka Observability Port interface Event typealias Events = (Event) -> Unit data class HttpCall(val uri: Uri) : Event
12. Technique: Screenplay actors interface Customer { fun listItems(): List<ItemId> fun order(id: ItemId, email: String): OrderId }
13. class HttpCustomer(baseUri: Uri, clock: Clock, events: Events, http: HttpHandler) : Customer { val http = ResetRequestTracing() .then(SetHostFrom(baseUri)) .then(AppOutgoingHttp(AppEvents("Customer", clock, events), http)) override fun listItems(): List<ItemId> { val response = http(Request(GET, "/list")) return Body.auto<List<ItemId >> ().toLens()(response) } override fun order(id: ItemId, email: String): OrderId { val response = http(Request(POST, "/order/$id").query("email", email)) return Body.auto<OrderId>().toLens()(response) } } Implementing the Customer
14. Technique: Fake Adapters fun FakeWarehouse(): HttpHandler { val stockLevels = mutableMapOf(ItemId.of("1") to 5) return routes( "/v1/items" bind GET to { Response(OK) .with( Body.auto<List<InventoryItem > > ().toLens() of stockLevels.map { (item, stock) - > InventoryItem(item, "bar", stock) } ) }, "/v1/dispatch" bind POST to { val order = Body.auto<Shipment>().toLens()(it) stockLevels[order.id] = stockLevels[order.id] !! - order.amount Response(ACCEPTED) } ) } fun main() { FakeWarehouse().asServer(SunHttp(8000)).start() }
15. Constructing our service fun ShopApi(env: Environment, clock: Clock, events: Events, http: HttpHandler): HttpHandler { val appEvents = AppEvents("shop", clock, events) val outgoingHttp = AppOutgoingHttp(appEvents, http) val shop = Shop(appEvents, Warehouse.Http(env[WAREHOUSE_URL], outgoingHttp), Notifications.SES(env, outgoingHttp) ) return AppIncomingHttp(appEvents, routes(PlaceOrder(shop), ListAllItems(shop))) }
16. Service testing abstract class RecordTraces { @RegisterExtension val events = TracerBulletEvents( listOf( :: HttpTracer, :: DbTracer).map { it(ActorByService) }, listOf(PumlSequenceDiagram, PumlInteractionDiagram), TraceRenderPersistence.FileSystem(File(".generated")) ) } interface ListItemsScenario { val customer: Customer @Test fun `can list items`() { expectThat(customer.listItems()).isEqualTo(listOf(ItemId.of(“1"))) } } class ShopApiTests : RecordTraces(), ListItemsScenario { val http: HttpHandler = ShopApi(ShopTestEnv, TestClock, events, FakeWarehouse()) override val customer = HttpCustomer(Uri.of("http: // shop"), TestClock, events, http) }
17. Tests as diagrams as code
18. Service composition
19. Server System as a function class EcommerceSystem(env: Environment, clock: Clock, events: Events, theInternet: HttpHandler) : HttpHandler { private val networkAccess = NetworkAccess() private val apiGateway = ApiGateway(env, clock, events, networkAccess) private val shop = ShopApi(env, clock, events, networkAccess) private val warehouse = WarehouseApi(env, clock, events, networkAccess, Inventory.InMemory(events, clock)) init { networkAccess.http = routes( reverseProxyRouting( env[API_GATEWAY_URL].authority to apiGateway, env[SHOP_URL].authority to shop, env[WAREHOUSE_URL].authority to warehouse ), Router.orElse bind theInternet, ) } override fun invoke(request: Request) = networkAccess(request) } http://api-gateway/order/ ECOMMERCE SYSTEM
20. Server Internet as a function http://s3/item-image-1 class TheInternet: HttpHandler { val emails = Storage.InMemory<List<EmailMessage > > () val departmentStore = FakeDepartmentStore() val ses = FakeSES(emails) val s3 = FakeS3() val http = reverseProxy( "dept-store" to departmentStore, "email" to ses, "s3" to s3 ) override fun invoke(request: Request) = http(request) } THE INTERNET RP
21. Server System Internet Universe as a function R THE INTERNET UNIVERSE ECOMMERCE SYSTEM
22. Universe testing interface CustomerBuysItemScenario { val storeManager: StoreManager val customer: WebsiteCustomer @Test fun `views item, orders, receives confirmation`() { val itemId = customer.listItems().first() expectThat(customer.canSeeImage(itemId)).isTrue() val orderId = customer.order(itemId, "joe@kotlin.org") expectThat(customer.hasEmailFor(orderId, "joe@kotlin.org")).isTrue() expectThat(storeManager.hasOrderItems(orderId)).isEqualTo(listOf(itemId)) } } class UniverseTests : RecordTraces(), CustomerBuysItemScenario { val iNet = TheInternet() val env = UniverseTestEnv overrides iNet.createCloudResourcesAndEnv() val clock = TestClock val system = EcommerceSystem(env, clock, events, iNet) override val customer = HttpWebsiteCustomer(env[API_GATEWAY_URL], clock, events, system, iNet.emails) override val storeManager = InternetStoreManager(iNet) }
23. Universe level diagrams
24. After the Big Bang!
25. Practices that make this easier - Monorepo - TDD (outside in) - Contract Testing (3rd party) - ADRs
26. What do we gain? - Shift-left of testing & observability - Massive reuse of testing infra - Higher level tests with the properties of UTs - Test one or more services in the same style - Free documentation!
27. http4k v5* and beyond *coming April ’23. Contains Loom. +
28. Thank you!
 Don’t forget
 to vote. David & Ivan Talk materials @ https://http4k.org/hyperpyramid