From Simulacrum to Typeclassic

Simulacrum simplifies development of type class libraries. It is used in a number of open source libraries, including Cats. In this talk, we’ll tour the features of Simulacrum, and look at the forthcoming Typeclassic project, which merges Simulacrum with complementary projects like machinist and export-hook.

Presented at the Typelevel Summit Philadelphia on March 2, 2016.

Transcript

1. From Simulacrum to Typeclassic March 2016
2. About Me… • Michael Pilquist (@mpilquist) • Using Scala professionally since 2008 • Primary author of scodec & simulacrum • Work at CCAD, LLC. (Combined Conditional Access Development) • Joint-venture between Comcast and ARRIS Group, Inc. • Build state of the art control systems that manage set-top boxes and consumer devices 2
3. My First Type Class 3 trait Semigroup[A] { def combine(x: A, y: A): A }
4. 4 trait Semigroup[A] { def combine(x: A, y: A): A } object Semigroup { def apply[A](implicit s: Semigroup[A]): Semigroup[A] = s } Semigroup[Int].combine(1, 2) Add implicit instance summoning method to companion Example usage
5. 5 trait Semigroup[A] { def combine(x: A, y: A): A } object Semigroup { def apply[A](implicit s: Semigroup[A]): Semigroup[A] = s implicit class Ops[A](val value: A)( implicit s: Semigroup[A]) { def combine(y: A): A = s.combine(value, y) } } import Semigroup.Ops 1.combine(2) Add extension methods via implicit class Example usage
6. 6 trait Semigroup[A] { def combine(x: A, y: A): A } object Semigroup { def apply[A](implicit s: Semigroup[A]): Semigroup[A] = s implicit class Ops[A](val value: A)( implicit s: Semigroup[A]) { def combine(y: A): A = s.combine(value, y) def |+|(y: A): A = s.combine(value, y) } } Add an operator alias Operator is added as an extension method only — not to the type class trait
7. 7 trait Semigroup[A] extends Any with Serializable { def combine(x: A, y: A): A } object Semigroup { def apply[A](implicit s: Semigroup[A]): Semigroup[A] = s implicit class Ops[A](val value: A)( implicit s: Semigroup[A]) { def combine(y: A): A = s.combine(value, y) def |+|(y: A): A = combine(y) } } Make type class a universal trait and add serializability
8. 8 trait Semigroup[A] extends Any with Serializable { def combine(x: A, y: A): A def combineAllOption(as: TraversableOnce[A]): Option[A] = as.reduceOption(combine) def combineN(x: A, n: Int): A = n match { case i if i <= 0 => sys.error("n must be >= 1") case 0 => x case i => combine(x, combineN(x, i - 1)) } } Add derived operations
9. 9 trait Semigroup[A] extends Any with Serializable { def combine(x: A, y: A): A def combineAllOption(xs: TraversableOnce[A]): Option[A] = ? def combineN(x: A, n: Int): A = ??? } object Semigroup { def apply[A](implicit s: Semigroup[A]): Semigroup[A] = s implicit class Ops[A](val value: A)( implicit s: Semigroup[A]) { def combine(y: A): A = s.combine(value, y) def |+|(y: A): A = combine(y) def combineN(n: Int): A = s.combineN(value, n) } } Add derived operations Extension method added for combineN but not combineAllOption
10. Type class hierarchies 10 trait Monoid[A] extends Semigroup[A] { def empty: A def isEmpty(a: A)(implicit eq: Eq[A]): Boolean = eq.equal(a, empty) def combineAll(as: TraversableOnce[A]): A = as.foldLeft(empty)(combine) }
11. Reusing extension methods from parent type 11 trait Monoid[A] extends Semigroup[A] { def empty: A def isEmpty(a: A)(implicit eq: Eq[A]): Boolean = eq.equal(a, empty) def combineAll(as: TraversableOnce[A]): A = as.foldLeft(empty)(combine) } object Monoid { def apply[A](implicit m: Monoid[A]): Monoid[A] = m implicit class Ops[A](override val value: A)( implicit m: Monoid[A]) extends Semigroup.Ops(value)(m) { def isEmpty(implicit eq: Eq[A]): Boolean = m.isEmpty(value) } } Reuses the semigroup Ops class
12. Using monoid ops 12 import Monoid.Ops 1 |+| 2 Importing monoid ops brings in the semigroup extension methods import Semigroup.Ops import Monoid.Ops 1 |+| 2 [error] value |+| is not a member of Int Importing both ops classes results in conflicting implicits and a less than helpful error message
13. Type classes in Scala 13 • Type classes with extension methods • require lots of boilerplate • result in extra allocations per-call • require discipline to keep aligned with type class methods
 • Organizing type classes in to a library is difficult • must balance modularity, performance, usability, and implicit resolution
14. Type classes in Scala 14 Simulacrum is a macro annotation based code generator that provides semi- first class syntax for building type classes scalaVersion := "2.11.7" addCompilerPlugin("org.scalamacros" % "paradise" % "2.1.0" cross CrossVersion.full) libraryDependencies += "com.github.mpilquist" %% "simulacrum" % "0.7.0"
15. Using simulacrum instead… 15 import simulacrum.{ typeclass, op } @typeclass trait Semigroup[A] { @op("|+|", alias = true) def combine(x: A, y: A): A def combineAllOption(as: TraversableOnce[A]): Option[A] = ? def combineN(x: A, n: Int): A = ??? } @typeclass trait Monoid[A] extends Semigroup[A] { def empty: A def isEmpty(a: A)(implicit eq: Eq[A]): Boolean = ??? def combineAll(as: TraversableOnce[A]): A = ??? }
16. Using simulacrum instead… 16 Semigroup[Int].combineAll(List(1, 2, 3)) Each @typeclass has a summoning apply method in the companion import Semigroup.ops._ 1 |+| 2 A la carte imports supported object all extends Semigroup.ToSemigroupOps with Monoid.ToMonoidOps import all._ 1 |+| 2 0.isEmpty Bulk imports supported without clashes from parent traits
17. Simulacrum Features 17 • Implemented as a macro annotation using Macro Paradise • Support for type classes that abstract over proper types and single argument type constructors • Local syntactic transformation • e.g., no forced type checking of parents • Caveats: • Does not support binary type constructors • Makes override keyword mandatory in places it is normally optional • Supported in Intellij IDEA
18. Performance 18 import Semigroup.ops._ showCode(reify { 1 |+| 2 }.tree) Semigroup.ops.toAllSemigroupOps(1)(Semigroup.sgInt).|+|(2) 0: getstatic #13 // Field Semigroup$ops$.MODULE$:LSemigroup$ops$; 3: iconst_1 4: invokestatic #19 // Method scala/runtime/BoxesRunTime.boxToInteger: (I)Ljava/lang/Integer; 7: getstatic #24 // Field Semigroup$.MODULE$:LSemigroup$; 10: invokevirtual #28 // Method Semigroup$.sgInt:()LSemigroup; 13: invokevirtual #32 // Method Semigroup$ops$.toAllSemigroupOps:(Ljava/ lang/Object;LSemigroup;)LSemigroup$AllOps; 16: iconst_2 17: invokestatic #19 // Method scala/runtime/BoxesRunTime.boxToInteger: (I)Ljava/lang/Integer; 20: invokeinterface #38, 2 // InterfaceMethod Semigroup$AllOps.$bar$plus$bar: (Ljava/lang/Object;)Ljava/lang/Object; 25: invokestatic #42 // Method scala/runtime/BoxesRunTime.unboxToInt: (Ljava/lang/Object;)I
19. 19 0: getstatic #13 // Field Semigroup$ops$.MODULE$:LSemigroup$ops$; 3: iconst_1 4: getstatic #18 // Field Semigroup$.MODULE$:LSemigroup$; 7: invokevirtual #22 // Method Semigroup$.sgInt:()LSemigroup; 10: invokevirtual #26 // Method Semigroup$ops$.toAllSemigroupOps$mIc$sp: (ILSemigroup;)LSemigroup$AllOps; 13: iconst_2 14: invokeinterface #32, 2 // InterfaceMethod Semigroup$AllOps.$bar$plus$bar $mcI$sp:(I)I @typeclass trait Semigroup[@specialized(Int) A] { @op("|+|", alias = true) def combine(x: A, y: A): A } Object allocation
20. 20 import language.experimental.macros import machinist.Ops trait Semigroup[@specialized(Int) A] { def combine(x: A, y: A): A } object Semigroup { implicit class Ops[A](lhs: A)(implicit ev: Semigroup[A]) { def |+|(rhs: A): A = macro MyOps.binop[A, A] } // Instances… } object MyOps extends Ops { val operatorNames = Map( "$bar$plus$bar" -> "combine" ) } 0: getstatic #13 // Field Semigroup$.MODULE$:LSemigroup$; 3: invokevirtual #17 // Method Semigroup$.sgInt:()LSemigroup; 6: iconst_1 7: iconst_2 8: invokeinterface #23, 3 // InterfaceMethod Semigroup.combine$mcI$sp:(II)I
21. Simulacrum vs Machinist • Forced to choose between boilerplate and performance • Shallow integration opportunity — Simulacrum should allow the ops trait to be partially written (this doesn’t exist today!) @typeclass trait Semigroup[@specialized(Int) A] { @op("|+|") def combine(x: A, y: A): A }
 object Semigroup { trait Ops[A] { def |+|(rhs: A): A = macro MyOps.binop[A, A] } } • Deep integration opportunity — automatically generate bindings to Machinist macros 21
22. Instances • Where does the compiler search for type class instances? • Imported or defined implicit values in the current scope or enclosing scope • Implicit scope • Implicit scope includes: • Type class companion • Companion of applied type parameter • Companions of every super type of the type class and the applied type param • How are type class hierarchies impacted by instance placement? • e.g., Monoid companion defines a Monoid[Int], but that instance is not in the implicit scope of Semigroup[Int] 22
23. Deriving Instances 23 scala> case class Foo(x: Int, y: Int) defined class Foo scala> Foo(1, 2) |+| Foo(3, 4) res1: Foo = Foo(4,6)
24. Deriving Instances 24 import shapeless.{ ::, HNil, Generic } implicit val hnil: Semigroup[HNil] = new Semigroup[HNil] { def combine(x: HNil, y: HNil) = x } implicit def hcons[H, T <: HList](implicit H: Semigroup[H], T: Semigroup[T]): Semigroup[H :: T] = new Semigroup[H :: T] { def combine(x: H :: T, y: H :: T) = H.combine(x.head, y.head) :: T.combine(x.tail, y.tail) } implicit def generic[A, R](implicit g: Generic.Aux[A, R], r: Semigroup[R]): Semigroup[A] = new Semigroup[A] { def combine(x: A, y: A) = g.from(r.combine(g.to(x), g.to(y))) }
25. Deriving Instances 25 scala> case class Foo(x: Int, y: Int) defined class Foo scala> Foo(1, 2) |+| Foo(3, 4) res1: Foo = Foo(4,6) • Where do we put the derivations? • Type class companion? • Requires a Shapeless dependency • Standalone object? • Results in derived instances taking precedence over implicit scope instances!
26. Export Hook 26 import export._ @typeclass trait Semigroup[@specialized(Int) A] { @op("|+|", alias = true) def combine(x: A, y: A): A } object Semigroup extends SemigroupLowPriority { implicit val sgInt: Semigroup[Int] = new Semigroup[Int] { def combine(x: Int, y: Int) = x + y } } @imports[Semigroup] trait SemigroupLowPriority
27. Export Hook 27 trait DerivedSemigroup[A] extends Semigroup[A] @exports object DerivedSemigroup { implicit val hnil: DerivedSemigroup[HNil] = ??? implicit def hcons[H, T <: HList](implicit H: Semigroup[H], T: Lazy[DerivedSemigroup[T]] ): DerivedSemigroup[H :: T] = ??? implicit def generic[A, R](implicit g: Generic.Aux[A, R], r: DerivedSemigroup[R] ): DerivedSemigroup[A] = ??? }
28. Export Hook 28 scala> import Semigroup.ops._ import Semigroup.ops._ scala> import DerivedSemigroup.exports._ import DerivedSemigroup.exports._ scala> case class Foo(x: Int, y: Int) defined class Foo scala> Foo(1, 2) |+| Foo(3, 4) res0: Foo = Foo(4,6)
29. Implicit Priority • Exporters can export at a specific priority level • Priority levels can be customized 29 implicit val priority = ExportPriority[ ExportHighPriority, ExportOrphan, ExportSubclass, ExportAlgebraic, ExportGeneric, ExportInstantiated, ExportDefault, ExportLowPriority]
30. Local Implicits • https://github.com/mpilquist/local-implicits • Proof of concept of syntax for manipulating implicits • Implemented as a compiler plugin
 2 |+| 3 // 5 imply(intMultiplication) { 2 |+| 3 } // 6 imply(minMonoid) { 2 |+| 3 } // 2 imply(maxMonoid) { 2 |+| 3 } // 3 • Supports local instances • "Modular Type Classes" http://www.mpi-sws.org/~dreyer/papers/mtc/main-long.pdf 30
31. Imp • https://github.com/non/imp • Zero-cost macro to summon implicit instances • Alternative to implicitly
 val s = imp[Semigroup[Int]] • Macro can be used to implement type class apply methods
 object Semigroup { def apply[A](implicit ev: Semigroup[A]): Semigroup[A] = macro summon[Semigroup[A]] } 
 31
32. 32 2012 2016 2015 Spire Macros Simulacrum Local Implicits Export Hook Imp 2014 Machinist
33. Type Class Design Factors 33 Defining Simulacrum Providing Instances Export Hook Using Local Implicits Performance Runtime: Machinist, Imp
 Compile time: ???
34. Type Classic • https://github.com/typelevel/typeclassic • Integration/merger: • simulacrum • machinist • imp • export-hook • local-implicits • Omnibus of type class infrastructure, allowing concise and efficient use • Integration points: • Ops backed by Machinist style macros • Instance prioritization via Export Hook style macros 34
35. Type Classic - Potential Integrations • Scalaz / Cats style Unapply support (SI-2712 workaround) • Enables ops that are otherwise not possible (e.g., flatten) • Auto-generation of unapply equivalents? (e.g., traverseU, sequenceU) • Shapeless’s Lazy • Provides a workaround for scalac abandoning implicit search too early • Shapeless’s cachedImplicit 35 Defining Unapply Providing Instances Lazy Using IDE Support Performance Compile time: cachedImplicit, custom implicit resolution
36. Type Classic - Usage Modes • Usage Modes • Mode 1: Compile time of type class provider only (Simulacrum) • Mode 2: Compile time of type class provider and use site (Machinist, Imp, Local Implicits) • Mode 3: Compile time of type class provider and use site + 
 small, stable runtime component (Export Hook) • Library authors have requested each of these modes • Should Type Classic support each of these directly? • Can we address a root cause instead? • Consideration: how is IDE support impacted by each mode? • Q: As a library author who wants only mode 1 or 2, what factors would motivate you to use Type Classic mode 3? 36
37. Takeaways • Simulacrum simplifies defining type classes • Machinist allows type class operations to be optimized • Export hook simplifies providing instances and supports derivation • Each address key factors in type class usability, but integration is limited • Type Classic will address integration issues • Success of Type Classic depends on community participation 37