record4s --- Extensible Records for Scala 3, and Domain Modeling with Structural Types

In this talk, we learn about the basics of my library called "record4s", which provides type-safe extensible records for Scala 3. It runs on JVM, JS, and Native platforms. You will see how it is implemented efficiently by using Scala 3 macros, and how record types as structural types, which work without reflection in Scala 3, are helpful in domain modeling.

record4s --- Scala 3のための拡張可能レコード、そして構造的型によるドメインモデリング

Scala 3向けの拙作の拡張可能レコードのライブラリ"record4s"を紹介します。JVM・JS・ネイティブのどのプラットフォームでも動作し、型安全なレコード型を提供するものです。Scala 3のマクロを用いて効率的に実装されている点や、Scala 3でリフレクションを必要としなくなった構造的型として表現されたレコード型がドメインモデリングにどう役立つかなどを説明します。

1. record4s — Extensible Records for Scala 3, and Domain Modeling with Structural Types Lintaro INA id:tarao @tarao @oarat 2024-06-08 ScalaMatsuri 2024 record4s — Scala 3 ͷͨΊͷ֦ுՄೳϨίʔυɺ ͦͯ͠ߏ଄తܕʹΑΔυϝΠϯϞσϦϯά
2. Are you tired of defining similar models? Example: a few difference case class BlogPost( id: PostId, title: String, body: String, url: URL, publishedAt: DateTime, ) case class BlogPostWithPaywall( id: PostId, title: String, body: String, paidPartOfBody: Option[String], url: URL, publishedAt: DateTime, ) ྫɿ ϒϩάهࣄΛද͢Ϟσϧʢࠨʣͱɺهࣄͷ༗ྉ෦෼΋࣋ͭϞσϧʢӈʣ paidPartOfBody ͷ༗ແ͚ͩҟͳΔ
3. Are you tired of defining similar models? Example blogPostRepository.find: PostId => Option[ BlogPost ] paywallRepository.findPaidBody: PostId => Option[String] def blogPostWithPaywall(postId: PostId): Option[ BlogPostWithPaywall ] = blogPostRepository.find(postId).map: post => // get a blog post BlogPostWithPaywall( id = post.id, title = post.title, body = post.body, paidPartOfBody = paywallRepository.findPaidBody(post.id), // add paid part of body url = post.url, publishedAt = post.publishedAt, ) ྫɿ ϒϩάهࣄΛऔಘͨ͠ޙɺ༗ྉ෦෼΋औಘ͢Δ
4. Are you tired of defining similar models? Example: getting a BlogPost blogPostRepository.find: PostId => Option[ BlogPost ] paywallRepository.findPaidBody: PostId => Option[String] def blogPostWithPaywall(postId: PostId): Option[ BlogPostWithPaywall ] = blogPostRepository.find(postId).map: post => // get a blog post BlogPostWithPaywall( id = post.id, title = post.title, body = post.body, paidPartOfBody = paywallRepository.findPaidBody(post.id), // add paid part of body url = post.url, publishedAt = post.publishedAt, ) ྫɿ ϒϩάهࣄΛऔಘͨ͠ޙɺ༗ྉ෦෼΋औಘ͢Δ ·ͣ͸ BlogPost ΛಘΔ
5. Are you tired of defining similar models? Example: adding a field blogPostRepository.find: PostId => Option[ BlogPost ] paywallRepository.findPaidBody: PostId => Option[String] def blogPostWithPaywall(postId: PostId): Option[ BlogPostWithPaywall ] = blogPostRepository.find(postId).map: post => // get a blog post BlogPostWithPaywall( id = post.id, title = post.title, body = post.body, paidPartOfBody = paywallRepository.findPaidBody(post.id), // add paid part of body url = post.url, publishedAt = post.publishedAt, ) ྫɿ ϒϩάهࣄΛऔಘͨ͠ޙɺ༗ྉ෦෼΋औಘ͢Δ paidPartOfBody Λ଍ͯ͠ BlogPostWithPaywall ʹ͢Δ
6. Are you tired of defining similar models? Example: boring passing around blogPostRepository.find: PostId => Option[ BlogPost ] paywallRepository.findPaidBody: PostId => Option[String] def blogPostWithPaywall(postId: PostId): Option[ BlogPostWithPaywall ] = blogPostRepository.find(postId).map: post => // get a blog post BlogPostWithPaywall( id = post.id, title = post.title, body = post.body, paidPartOfBody = paywallRepository.findPaidBody(post.id), // add paid part of body url = post.url, publishedAt = post.publishedAt, ) ྫɿ ϒϩάهࣄΛऔಘͨ͠ޙɺ༗ྉ෦෼΋औಘ͢Δ BlogPost ͔Β BlogPostWithPaywall ΁ͷϑΟʔϧυͷ٧Ίସ͕͑໘౗
7. Are you tired of defining similar models? Example product owner: “Hey, we need draft posts, with paywall of course!” case class DraftPost( id: PostId, title: String, body: String, previewUrl: URL, ) case class DraftPostWithPaywall( id: PostId, title: String, body: String, paidPartOfBody: Option[String], previewUrl: URL, ) ྫɿ ϓϩμΫτΦʔφʔʮԼॻ͖ػೳ͕΄͍͠Μ͚ͩͲɻ༗ྉ෦෼΋ѻ͑Δ΍ͭʂʯ
8. Are you tired of defining similar models? Example: writing the same thing draftPostRepository.find: PostId => Option[DraftPost] paywallRepository.findPaidBody: PostId => Option[String] def draftPostWithPaywall(postId: PostId): Option[DraftPostWithPaywall] = draftPostRepository.find(postId).map: post => DraftPostWithPaywall( id = post.id, title = post.title, body = post.body, paidPartOfBody = paywallRepository.findPaidBody(post.id), previewUrl = post.previewUrl, ) ྫɿ ಉ͡Α͏ͳ͜ͱΛॻ͘Ӌ໨ʹ
9. Are you tired of defining similar models? Example product owner: “Hey, we need modification date, for both published and draft posts!” ྫɿ ϓϩμΫτΦʔφʔʮߋ৽೔͕࣌΄͍͠Μ͚ͩͲɻެ։ࡁΈهࣄͱԼॻ͖྆ํʂʯ
10. Are you tired of defining similar models? Example product owner: “Hey, we need modification date, for both published and draft posts!” ྫɿ ϓϩμΫτΦʔφʔʮߋ৽೔͕࣌΄͍͠Μ͚ͩͲɻެ։ࡁΈهࣄͱԼॻ͖྆ํʂʯ ͥΜͿॻ͖׵͑Δͷ໘౗ʜ
11. Wait, we have structural types in Scala 3, right? ଴ͬͯɺ Scala 3 ʹ͸ߏ଄తܕ͕͋Δɺͦ͏Ͱ͠ΐʁ
12. Modeling with structural types Typing in Scala 3 type Model = ... // just some type type AbstractBlogPost = Model { val id: PostId val title: String val body: String } type BlogPost = AbstractBlogPost & Model { val url: URL val publishedAt: DateTime } type Paid = Model { val paidPartOfBody: Option[String] } type BlogPostWithPaywall = BlogPost & Paid type DraftPost = AbstractBlogPost & Model { val previewUrl: URL } type DraftPostWithPaywall = DraftPost & Paid ▶ Listing field types by val ▶ Merging field sets by & ▶ No repeat val ͱ &ͰܕΛఆٛ͢Δ / ಉ͜͡ͱΛ܁Γฦ͠ॻ͔ͳ͍ͰࡁΉ ͜Ε͸ Scala 3 ͷܕγεςϜͷػೳ
13. Modeling with structural types A way described in Scala 3 Book class Model(elems: (String, Any)*) extends Selectable: val fields = elems.toMap def selectDynamic(name: String): Any = fields(name) val post = Model( "id" -> postId, "title" -> title, "body" -> body, "url" -> url, "publishedAt" -> publishedAt, ). asInstanceOf[BlogPost] ߏ଄తܕͷΦϒδΣΫτͷఆٛͷ͔ͨ͠͸ެࣜ Scala 3 Book ʹઆ໌͕͋Δ
14. Modeling with structural types A way described in Scala 3 Book class Model(elems: (String, Any)*) extends Selectable: val fields = elems.toMap def selectDynamic(name: String): Any = fields(name) val post = Model( "id" -> postId, "title" -> title, "body" -> body, "url" -> url, "publishedAt" -> publishedAt, ). asInstanceOf[BlogPost] ▶ Unsafe asInstanceOf ߏ଄తܕͷΦϒδΣΫτͷఆٛͷ͔ͨ͠͸ެࣜ Scala 3 Book ʹઆ໌͕͋Δ ͨͩ͠ɺ asInstanceOf ͕ඞཁͰΠέͯͳ͍
15. Modeling with structural types A way described in Scala 3 Book class Model(elems: (String, Any)*) extends Selectable: val fields = elems.toMap def selectDynamic(name: String): Any = fields(name) val post = Model( "id" -> postId, "title" -> title, "body" -> body, "url" -> url, "publishedAt" -> publishedAt, ). asInstanceOf[BlogPost] val fields = post.fields + ("paidPartOfBody" -> paidPartOfBody) val postWithPaywall = Model(fields.toSeq*) .asInstanceOf[BlogPostWithPaywall] ▶ Unsafe asInstanceOf ▶ Adding a field by hand ߏ଄తܕͷΦϒδΣΫτͷఆٛͷ͔ͨ͠͸ެࣜ Scala 3 Book ʹઆ໌͕͋Δ ͨͩ͠ɺ asInstanceOf ͕ඞཁͰΠέͯͳ͍ / ϑΟʔϧυͷ௥Ճ͕ແཧ΍Γ
16. Modeling with structural types A way described in Scala 3 Book class Model(elems: (String, Any)*) extends Selectable: val fields = elems.toMap def selectDynamic(name: String): Any = fields(name) val post = Model( "id" -> postId, "title" -> title, "body" -> body, "url" -> url, "publishedAt" -> publishedAt, ). asInstanceOf[BlogPost] val fields = post.fields + ("paidPartOfBody" -> paidPartOfBody) val postWithPaywall = Model(fields.toSeq*) .asInstanceOf[BlogPostWithPaywall] ▶ Unsafe asInstanceOf ▶ Adding a field by hand ߏ଄తܕͷΦϒδΣΫτͷఆٛͷ͔ͨ͠͸ެࣜ Scala 3 Book ʹઆ໌͕͋Δ ͨͩ͠ɺ asInstanceOf ͕ඞཁͰΠέͯͳ͍ / ϑΟʔϧυͷ௥Ճ͕ແཧ΍Γ
17. Modeling with structural types Imagine that instantiation goes straightforward An ideal way of instantiation val post: BlogPost = Model( id = postId, title = title, body = body, url = url, publishedAt = publishedAt ) ૉ௚ʹΠϯελϯεԽ͍ͨ͠
18. Modeling with structural types Imagine that objects can be extended An ideal way of adding a field val postWithPaywall: BlogPostWithPaywall = post + (paidPartOfBody = paidPartOfBody) ΦϒδΣΫτʹϑΟʔϧυΛ଍֦ͯ͠ு͍ͨ͠
19. Are these possible? ͦΜͳ͜ͱ͕Ͱ͖ͪΌ͏
20. Are these possible? With record4s, Yes! ͦΜͳ͜ͱ͕Ͱ͖ͪΌ͏ ͦ͏ɺ record4s ͳΒͶʂ
21. tarao/record4s Extensible records for Scala github.com Try now with Scala CLI: //> using dep "com.github.tarao::record4s:0.12.0" import com.github.tarao.record4s.% The previous examples work with: import com.github.tarao.record4s.% type Model = % val Model = % GitHub Ͱެ։ࡁΈ / Scala CLI Ͱࠓ͙͢ࢼͦ͏ʂ લͷྫ΋ Model ΛΤΠϦΞεʹ͢Δͱͦͷ··ಈ͘
22. What record4s can do Record construction val person = %( name = "tarao", age = 3, ) // person: % { // val name: String // val age: Int // } = %(name = tarao, age = 3) record4s ͰͰ͖Δ͜ͱ Ϩίʔυͷ࡞੒ɿ ߏ଄తܕͷΦϒδΣΫτ͕࡞ΒΕΔ
23. What record4s can do Record construction Type-safe field access person.name // res0: String = "tarao" person.age // res1: Int = 3 person.email // error: // value email is not a member of // %{val name: String; val age: Int} record4s ͰͰ͖Δ͜ͱ ܕ҆શͳϑΟʔϧυΞΫηεɿ ଘࡏ͠ͳ͍ϑΟʔϧυ͸ίϯύΠϧΤϥʔʹ
24. What record4s can do Record construction Type-safe field access Extension person + ( email = "tarao@example.com", occupation = "engineer", ) // res2: % { // val name: String // val age: Int // val email: String // val occupation: String // } = %(name = tarao, age = 3, // email = tarao@example.com, // occupation = engineer) record4s ͰͰ͖Δ͜ͱ Ϩίʔυͷ֦ுʢෳ਺ϑΟʔϧυಉ࣌ʹ௥ՃՄೳʣ
25. What record4s can do Record construction Type-safe field access Extension Concatenation val email = %(email = "tarao@example.com") // email: % { // val email: String // } = %(email = tarao@example.com) person ++ email // res3: % { // val name: String // val age: Int // val email: String // } = %(name = tarao, age = 3, // email = tarao@example.com) record4s ͰͰ͖Δ͜ͱ Ϩίʔυͷ݁߹
26. What record4s can do Record construction Type-safe field access Extension Concatenation Field update person + (age = person.age + 1) // res4: % { // val name: String // val age: Int // } = %(name = tarao, age = 4) person + (name = %(first = "tarao", last = "fuguta")) // res5: % { // val name: % { // val first: String // val last: String // } // val age: Int // } = %(name = %(first = tarao, last = fuguta), age = 3) record4s ͰͰ͖Δ͜ͱ ϑΟʔϧυͷߋ৽ɿ ผͷܕͰ্ॻ͖ͯ͠΋Α͍
27. Other features Type tagging Selecting/unselecting/reordering/renaming fields Pattern matching Conversion from/to case classes JSON conversion (using circe or uPickle) Scala.js support Scala Native support ଞʹ΋͍Ζ͍Ζػೳ͕๛෋
28. Tips — Methods on records Methods on records can be defined as extension methods extension (post: BlogPost) def summary(length: Int = 200): String = post.body.take(length) val post: BlogPost = ... val postWithPaywall: BlogPostWithPaywall = ... post.summary() // works postWithPaywall.summary() // also works // Note: we have BlogPostWithPaywall <: BlogPost // since BlogPostWithPaywall has more fields // (That is structural subtyping) Ϩίʔυͷϝιου͸֦ுϝιουͱͯ͠ఆٛՄೳ ߏ଄త෦෼ܕʹͳ͍ͬͯΔΦϒδΣΫτʹରͯ͠΋ಉ͡ϝιουΛݺ΂Δ
29. Tips — Decoding JSON by example Normally, you have to specify record type to decode from JSON import com.github.tarao.record4s.upickle.Record.readWriter import upickle.default.read val json = """{"name":"tarao","age":3}""" type Person = % { val name: String; val age: Int } // you have to write this! val r = read[Person](json) ௨ৗɺ JSON ͔Βσίʔυ͢Δʹ͸σίʔυઌͷܕΛॻ͘ඞཁ͕͋Δ ʢͦͯͦ͠Ε͸͠͹͠͹໘౗ʣ
30. Tips — Decoding JSON by example Giving an example let you omit the type definition import com.github.tarao.record4s.upickle.Record.readWriter import upickle.default.{Reader, read} def decodeByExample[R <: `%`: Reader](json: String, example: R): R = read[R](json) val json = """{"name":"tarao","age":3}""" val example = %(name = "ikura", age = 1) // no type definition val r = decodeByExample(json, example) ۩ମྫΛ༩͑Ε͹ܕఆٛ͸লུՄೳ
31. Internals ಺෦࣮૷
32. Field access 1. Use Selectable from the standard library abstract class % extends Selectable: def __data: Map[String, Any] def selectDynamic(name: String): Any = __data(name) final class MapRecord(val __data: Map[String, Any]) extends % 2. Field access compiles to selectDynamic call val r = %(name = "tarao", age = 3) // returns a MapRecord r.age // ⇝ r.selectDynamic("age").asInstanceOf[Int] 1. ඪ४ϥΠϒϥϦͷ Selectable Λ࢖͏ 2. ϑΟʔϧυΞΫηε͸ίϯύΠϧ࣌ʹselectDynamic ͷݺͼग़͠ʹஔ׵͞ΕΔ
33. Concatenation val r1 = %(...) val r2 = %(...) r1 ++ r2 inlining ⇝ val r1: R1 = %(...) val r2: R2 = %(...) val c = summon[Concat[R1, R2]] new MapRecord(r1.__data.concat(r2.__data)) .asInstanceOf[c.Out] Ϩίʔυͷ݁߹͸ Map ͷ concat ʹ͢Δ
34. Concatenation val r1 = %(...) val r2 = %(...) r1 ++ r2 inlining ⇝ val r1: R1 = %(...) val r2: R2 = %(...) val c = summon[Concat[R1, R2]] new MapRecord(r1.__data.concat(r2.__data)) .asInstanceOf[c.Out] // transparent macro to calculate Out from R1 and R2 transparent inline given [R1, R2]: Concat[R1, R2] = ${ Macros.derivedTypingConcatImpl } Ϩίʔυͷ݁߹͸ Map ͷ concat ʹ͢Δ Concat ͸ given ͕ϚΫϩͰ࣮૷͞Ε͍ͯͯɺ݁߹݁ՌͷܕΛܭࢉ͢Δ
35. Concatenation val r1 = %(...) val r2 = %(...) r1 ++ r2 inlining ⇝ val r1: R1 = %(...) val r2: R2 = %(...) val c = summon[Concat[R1, R2]] new MapRecord(r1.__data.concat(r2.__data)) .asInstanceOf[c.Out] Wait, is this really safe? Ϩίʔυͷ݁߹͸ Map ͷ concat ʹ͢Δ ͱ͜ΖͰ͜Ε͸ຊ౰ʹ҆શʁ
36. Key duplication problem val r1 = %(age = 3) val r2: % { val name: String } = %(name = "tarao", age = "unknown") // legal since % { val name: String; val age: String } <: % { val name: String } val r = r1 ++ r2 // r: % { // val age: Int // val name: String // } = %(age = "unknwon", name = "tarao") r.age // java.lang.ClassCastException: // class java.lang.String cannot be cast to class java.lang.Integer ݁߹ͷӈลͷ੩తܕʹݱΕͳ͍ϑΟʔϧυ͕ඇޓ׵ͳܕͰ্ॻ͖Ͱ͖ͯ͠·͏
37. Key duplication problem val r1 = %(age = 3) val r2: % { val name: String } = %(name = "tarao", age = "unknown") // legal since % { val name: String; val age: String } <: % { val name: String } val r = r1 ++ r2 // r: % { // val age: Int // val name: String // } = %(age = "unknwon", name = "tarao") r.age // java.lang.ClassCastException: // class java.lang.String cannot be cast to class java.lang.Integer We need to “tidy up” RHS value by its static type ݁߹ͷӈลͷ੩తܕʹݱΕͳ͍ϑΟʔϧυ͕ඇޓ׵ͳܕͰ্ॻ͖Ͱ͖ͯ͠·͏ ӈล஋͸੩తܕʹ ἧ͓͑ͯ͘ඞཁ͕͋Δ
38. Key duplication problem Something like this: val r1 = %(...) val r2 = %(...) r1 ++ r2 inlining ⇝ val r1: R1 = %(...) val r2: R2 = %(...) val c = summon[Concat[R1, R2]] new MapRecord(r1.__data.concat(r2.__data)) .asInstanceOf[c.Out] ͜͜Λ
39. Key duplication problem Something like this: val r1 = %(...) val r2 = %(...) r1 ++ r2 inlining ⇝ val r1: R1 = %(...) val r2: R2 = %(...) val c = summon[Concat[R1, R2]] new MapRecord(r1.__data.concat(r2.as[R2].__data)) .asInstanceOf[c.Out] ͳΜΒ͔͜ͷΑ͏ʹ
40. Performance characteristics ύϑΥʔϚϯεಛੑ
41. Runtime performance Creation time ▶ Grows linearly against record size ▶ Almost the same as Map Ϩίʔυͷ࡞੒࣌ؒ͸αΠζʹରͯ͠ઢܗ Map ͱಉ౳
42. Runtime performance Field access time ▶ Effectively constant time ϑΟʔϧυΞΫηε͸࣮࣭ఆ਺࣌ؒ
43. Compile-time performance Compilation time of concatenation ▶ Grows quadratically against record size ▶ Due to subtyping check of structural types ▶ But far better than shapeless Record Ϩίʔυ݁߹ͷίϯύΠϧ࣌ؒ͸αΠζͷೋ৐ʹൺྫʢߏ଄త෦෼ܕͷݕࠪͷͨΊʣ ͱ͸͍͑ shapeless ΑΓͣͬͱ͍͍
44. Conclusion record4s provides extensible records for Scala 3 It is useful for some kind of domain modeling It is carefully designed using inlining and macros It has good enough performance characteristics record4s ͸ Scala 3 ޲͚ͷ֦ுՄೳϨίʔυΛఏڙ͠ɺυϝΠϯϞσϦϯάʹ໾ཱͭ ϚΫϩ౳Λ༻͍ͯ஫ҙਂ࣮͘૷͞Ε͍ͯͯɺे෼ͳύϑΥʔϚϯεಛੑΛඋ͍͑ͯΔ