Don't Fear the Implicits: Everything You Need to Know About Typeclasses

Slides from Scala Days Berlin 2016

Developers who are new to Scala often shy away from coming into contact with implicits, and by extension, understanding typeclasses. In big organizations that have been adopting Scala at scale, you sometimes even come across hard rules that put a ban on the use of implicits because that language feature is considered to be too advanced and not understood by a lot of developers. On the other hand, implicits and typeclasses are used heavily not only by a lot of the most important Scala frameworks and libraries, but also in the standard library. Given the fact that it is so hard to evade them when writing real world Scala code, I would like to encourage developers adopting Scala to overcome their fear of implicits and instead embrace the typeclass pattern. In this talk, as an intermediate Scala developer, you will learn everything you really need to know about typeclasses: What they are good for and how they compare to what you are familiar with from object-oriented languages, when you should and should not use them, how the pattern can be encoded in Scala and how to write your own typeclasses, how to provide instances of typeclasses for your own or existing types, and how to do all of this with minimal boilerplate. Throughout the talk, you will see numerous examples of typeclasses used in the Scala ecosystem and the standard library, and you'll see that you don't need to know anything about category theory to benefit from embracing typeclasses.

Transcript

1. Don’t Fear the Implicits Everything You Need to Know About Typeclasses Daniel Westheide
3. About me > consultant at innoQ Germany > author of The Neophyte’s Guide to Scala > Twitter: @kaffeecoder > Website: danielwestheide.com
4. Implicits
5. def map[S](f: T => S) (implicit executor: ExecutionContext): Future[S] def ?(message: Any)(implicit timeout: Timeout) def !(message: Any) (implicit sender: ActorRef = Actor.noSender) def errorsAsJson(implicit lang: Lang): JsValue dependencies configuration context context
6. Typeclasses
7. Magic Button(c) Jake Rust, https://www.flickr.com/photos/jakerust/16846017825/, CC-BY 2.0, www.gotcredit.com
8. @Configuration @EnableAuthorizationServer public class AuthorizationServerConfiguration extends AuthorizationServerConfigurerAdapter { ... }
9. La la la, I can’t see you, darn implicits! I can’t see you (c) Newtown grafitti https://www.flickr.com/photos/newtown_grafitti/5225673037 CC-BY 2.0
10. val xs = Vector(1, 2).map(_ * 2) val sum = xs.sum CanBuildFrom Numeric
12. Arcane (c) Bart, https://www.flickr.com/photos/cayusa/2651494125, CC-BY 2.0
13. Example > given: big data framework for processing distributed data sets > need: processing of various types of quantities
14. DistributedDataset trait DistributedDataset[A] { def foldLeft[B](z: B)(op: (B, A) => B): B def count: Int }
15. case class Kilograms(value: BigDecimal) { def +(y: Kilograms): Kilograms = Kilograms(value + y.value) def -(y: Kilograms): Kilograms = Kilograms(value - y.value) def *(y: BigDecimal): Kilograms = Kilograms(value * y) def /(y: BigDecimal): Kilograms = Kilograms(value / y) } object Kilograms { val zero: Kilograms = Kilograms(BigDecimal(0)) def sum(xs: DistributedDataset[Kilograms]): Kilograms = xs.foldLeft(zero)(_ + _) def mean(xs: DistributedDataset[Kilograms]): Kilograms = sum(xs) / xs.count }
16. case class Kilometers(value: BigDecimal) { def +(y: Kilometers): Kilometers = Kilometers(value + y.value) def -(y: Kilometers): Kilometers = Kilometers(value - y.value) def *(y: BigDecimal): Kilometers = Kilometers(value * y) def /(y: BigDecimal): Kilometers = Kilometers(value / y) } object Kilometers { val zero: Kilometers = Kilometers(BigDecimal(0)) def sum(xs: DistributedDataset[Kilometers]): Kilometers = xs.foldLeft(zero)(_ + _) def mean(xs: DistributedDataset[Kilometers]): Kilometers = sum(xs) / xs.count }
17. I know! Let’s use inheritance! paris rodin thinker (c) Mark B. Schlemmer https://www.flickr.com/photos/mbschlemmer/3122967980 CC-BY 2.0
18. trait Quantity[A <: Quantity[A]] { def value: BigDecimal def unit(x: BigDecimal): A def +(y: A): A = unit(value + y.value) def -(y: A): A = unit(value - y.value) def *(y: BigDecimal): A = unit(value * y) def /(y: BigDecimal): A = unit(value / y) } object Quantity { def sum[A <: Quantity[A]](xs: DistributedDataset[A], zero: A): A = xs.foldLeft(zero)(_ + _) def mean[A <: Quantity[A]](xs: DistributedDataset[A], zero: A): A = sum(xs, zero) / xs.count }
19. case class Kilograms(value: BigDecimal) extends Quantity[Kilograms] { override def unit(x: BigDecimal): Kilograms = Kilograms(x) } case class Kilometers(value: BigDecimal) extends Quantity[Kilometers] { override def unit(x: BigDecimal): Kilometers = Kilometers(x) }
20. Maybe I should use an adapter… USB ethernet adapter (c) Ash Kyd https://www.flickr.com/photos/ashkyd/14359608157 CC-BY 2.0
21. import org.joda.time.Duration /* Adapter pattern */ case class Milliseconds(underlying: Duration) extends Quantity[Milliseconds] { override def value: BigDecimal = underlying.getMillis override def unit(x: BigDecimal): Milliseconds = Milliseconds(Duration.millis(x.toLong)) }
22. val durations: DistributedDataset[Milliseconds] = DistributedDataset.distribute(Seq( Milliseconds(Duration.standardMinutes(3)), Milliseconds(Duration.standardSeconds(17)), Milliseconds(Duration.standardSeconds(5)), Milliseconds(Duration.standardHours(1)))) val meanDuration = Quantity.mean(durations, Milliseconds(Duration.ZERO)) Lots of new objects boilerplate losing the original type
23. Try it differently! old woman in the woods (c) Matt Wiebe https://www.flickr.com/photos/mattwieve/17721503894 CC-BY 2.0
24. import org.joda.time.Duration case class Kilograms(value: BigDecimal) case class Kilometers(value: BigDecimal) trait Quantity[A] { def value(x: A): BigDecimal def unit(x: BigDecimal): A def zero: A = unit(BigDecimal(0)) def plus(x: A, y: A): A = unit(value(x) + value(y)) def minus(x: A, y: A): A = unit(value(x) - value(y)) def times(x: A, y: BigDecimal): A = unit(value(x) * y) def div(x: A, y: BigDecimal): A = unit(value(x) / y) }
25. import org.joda.time.Duration object Quantity { val kilogramQuantity: Quantity[Kilograms] = new Quantity[Kilograms] { override def value(x: Kilograms): BigDecimal = x.value override def unit(x: BigDecimal): Kilograms = Kilograms(x) } val kilometerQuantity: Quantity[Kilometers] = new Quantity[Kilometers] { override def value(x: Kilometers): BigDecimal = x.value override def unit(x: BigDecimal): Kilometers = Kilometers(x) } // to be continued }
26. import org.joda.time.Duration object Quantity { // continued val durationQuantity: Quantity[Duration] = new Quantity[Duration] { override val zero: Duration = Duration.ZERO override def value(x: Duration): BigDecimal = x.getMillis override def plus(x: Duration, y: Duration): Duration = x.plus(y) override def minus(x: Duration, y: Duration): Duration = x.minus(y) override def unit(x: BigDecimal): Duration = Duration.millis(x.toLong) } // to be continued }
27. import org.joda.time.Duration object Quantity { // continued def sum[A](xs: DistributedDataset[A], quantity: Quantity[A]): A = xs.foldLeft(quantity.zero)(quantity.plus) def mean[A](xs: DistributedDataset[A], quantity: Quantity[A]): A = quantity.div(sum(xs, quantity), xs.count) }
28. val durations: DistributedDataset[Duration] = DistributedDataset.distribute(Seq( Duration.standardMinutes(3), Duration.standardSeconds(17), Duration.standardSeconds(5), Duration.standardHours(1))) val meanDuration = Quantity.mean(durations, Quantity.durationQuantity)
29. def sum[A](xs: DistributedDataset[A], quantity: Quantity[A]): A = xs.foldLeft(quantity.zero)(quantity.plus) def mean[A](xs: DistributedDataset[A], quantity: Quantity[A]): A = quantity.div(sum(xs, quantity), xs.count)
30. def sum[A](xs: DistributedDataset[A])(quantity: Quantity[A]): A = xs.foldLeft(quantity.zero)(quantity.plus) def mean[A](xs: DistributedDataset[A])(quantity: Quantity[A]): A = quantity.div(sum(xs)(quantity), xs.count) separate parameter list separate parameter list
31. import org.joda.time.Duration val durations: DistributedDataset[Duration] = DistributedDataset.distribute(Seq( Duration.standardMinutes(3), Duration.standardSeconds(17), Duration.standardSeconds(5), Duration.standardHours(1))) val meanDuration = Quantity.mean(durations, Quantity.durationQuantity)
32. import org.joda.time.Duration val durations: DistributedDataset[Duration] = DistributedDataset.distribute(Seq( Duration.standardMinutes(3), Duration.standardSeconds(17), Duration.standardSeconds(5), Duration.standardHours(1))) val meanDuration = Quantity.mean(durations)(Quantity.durationQuantity) separate parameter list
33. def sum[A](xs: DistributedDataset[A]) (implicit quantity: Quantity[A]): A = xs.foldLeft(quantity.zero)(quantity.plus) def mean[A](xs: DistributedDataset[A]) (implicit quantity: Quantity[A]): A = quantity.div(sum(xs), xs.count)
34. import org.joda.time.Duration object Quantity { implicit val kilogramQuantity: Quantity[Kilograms] = ??? implicit val kilometerQuantity: Quantity[Kilometers] = ??? implicit val durationQuantity: Quantity[Duration] = ??? }
35. import org.joda.time.Duration val durations: DistributedDataset[Duration] = DistributedDataset.distribute(Seq( Duration.standardMinutes(3), Duration.standardSeconds(17), Duration.standardSeconds(5), Duration.standardHours(1))) val meanDuration = Quantity.mean(durations)(Quantity.durationQuantity)
36. import org.joda.time.Duration val durations: DistributedDataset[Duration] = DistributedDataset.distribute(Seq( Duration.standardMinutes(3), Duration.standardSeconds(17), Duration.standardSeconds(5), Duration.standardHours(1))) val meanDuration = Quantity.mean(durations) implicits filled in by compiler
37. And this is how we discovered typeclasses…
38. Context bounds def sum[A: Quantity](xs: DistributedDataset[A]): A = { val quantity = implicitly[Quantity[A]] xs.foldLeft(quantity.zero)(quantity.plus) } def mean[A: Quantity](xs: DistributedDataset[A]): A = { val quantity = implicitly[Quantity[A]] quantity.div(sum(xs), xs.count) }
39. Implicit resolution Imported Inherited in package object Local Explicit Implicit scope > companion object of typeclass > companion object of A > companion objects of super types
40. Local implicit import org.joda.time.Duration implicit val durationQuantity: Quantity[Duration] = ??? val durations: DistributedDataset[Duration] = DistributedDataset.distribute(Seq( Duration.standardMinutes(3), Duration.standardSeconds(17), Duration.standardSeconds(5), Duration.standardHours(1))) val meanDuration = Quantity.mean(durations)
41. Imported implicits import org.joda.time.Duration trait LowPriorityImplicits { implicit val durationQuantity: Quantity[Duration] = ??? } object Implicits extends LowPriorityImplicits
42. Imported implicits import org.joda.time.Duration import Implicits._ val durations: DistributedDataset[Duration] = ??? val meanDuration = Quantity.mean(durations) imported implicit Quantity[Duration]
43. Inherited implicits import org.joda.time.Duration object MixinExample extends App with LowPriorityImplicits { val durations: DistributedDataset[Duration] = ??? val meanDuration = Quantity.mean(durations) println(meanDuration) } mixed in implicit Quantity[Duration]
44. Overriding implicits import org.joda.time.Duration object OverridingExample extends App with LowPriorityImplicits { implicit val myDurationQuantity: Quantity[Duration] = ??? val durations: DistributedDataset[Duration] = ??? val meanDuration = Quantity.mean(durations) println(meanDuration) }
45. Operators package quantities package object lib { implicit class QuantityOps[A](a: A)(implicit quantity: Quantity[A]) { def +(a2: A): A = quantity.plus(a, a2) def -(a2: A): A = quantity.minus(a, a2) def *(y: BigDecimal): A = quantity.times(a, y) def /(y: BigDecimal): A = quantity.div(a, y) } }
46. Operators import quantities.v10.lib._ val nineKilos = Kilograms(3) * 3 val twelveKilos = nineKilos + Kilograms(3)
47. Usability
48. Error messages import scala.annotation.implicitNotFound @implicitNotFound("No instance of typeclass Quantity found for type ${A}.") trait Quantity[A] { def value(x: A): BigDecimal def unit(x: BigDecimal): A def zero: A = unit(BigDecimal(0)) def plus(x: A, y: A): A = unit(value(x) + value(y)) def minus(x: A, y: A): A = unit(value(x) - value(y)) def times(x: A, y: BigDecimal): A = unit(value(x) * y) def div(x: A, y: BigDecimal): A = unit(value(x) / y) }
49. Shortcut for implicitly object Quantity { def apply[A](implicit quantity: Quantity[A]): Quantity[A] = quantity def sum[A : Quantity](xs: DistributedDataset[A]): A = xs.foldLeft(Quantity[A].zero)(Quantity[A].plus) def mean[A : Quantity](xs: DistributedDataset[A]): A = Quantity[A].div(sum(xs), xs.count) }
50. Typeclass constructors object Quantity { def simple[A](unitF: BigDecimal => A) (valueF: A => BigDecimal): Quantity[A] = new Quantity[A] { override def value(x: A): BigDecimal = valueF(x) override def unit(x: BigDecimal): A = unitF(x) } implicit val kilogramQuantity = Quantity.simple(Kilograms)(_.value) implicit val kilometerQuantity = Quantity.simple(Kilometers)(_.value) }
51. Typeclass constructors: Ordering import org.joda.time.{DateTime, Duration, LocalDate} implicit lazy val defaultDurationOrdering: Ordering[Duration] = Ordering.by(_.getMillis) implicit lazy val defaultDateTimeOrdering: Ordering[DateTime] = Ordering.fromLessThan(_ isBefore _)
52. Combinators
53. Combinators: Circe import io.circe.{Encoder, Json} import io.circe.Encoder._ case class BookId(value: Int) extends AnyVal object BookId { implicit val bookIdEncoder: Encoder[BookId] = Encoder[Int].contramap(_.value) } package io.circe trait Encoder[A] extends Serializable { self => def apply(a: A): Json final def contramap[B](f: B => A): Encoder[B] = new Encoder[B] { final def apply(a: B) = self(f(a)) } }
54. Generic typeclass instances
55. Generic instances: Circe package io.circe object Encoder { implicit final def encodeOption[A](implicit e: Encoder[A]): Encoder[Option[A]] = new Encoder[Option[A]] { final def apply(a: Option[A]): Json = a match { case Some(v) => e(v) case None => Json.Null } } }
56. Deriving instances of Foo[A] from instances of Bar[A]
57. Example: Plugging typeclass-based JSON libraries into Akka HTTP
58. Akka HTTP JSON package de.heikoseeberger.akkahttpcirce import akka.http.scaladsl.marshalling.{ Marshaller, ToEntityMarshaller } import akka.http.scaladsl.model.MediaTypes.`application/json` import io.circe.{ Encoder, Json, Printer } trait CirceSupport { implicit def circeToEntityMarshaller[A] (implicit encoder: Encoder[A], printer: Json => String = Printer.noSpaces.pretty): ToEntityMarshaller[A] = Marshaller.StringMarshaller.wrap(`application/json`) (printer).compose(encoder.apply) }
59. import io.circe.Encoder import io.circe.Encoder._ import io.circe.generic.semiauto._ case class Book(id: BookId, title: String, author: String) object Book { implicit val bookEncoder: Encoder[Book] = deriveEncoder[Book] } import akka.http.scaladsl.server.Directives._ import de.heikoseeberger.akkahttpcirce.CirceSupport trait Routes extends CirceSupport { val bookRepository = new BookRepository val route = path("books" / IntNumber) { bookId => get { complete { bookRepository.find(BookId(bookId)) } } } }
60. Default instances > Provide default instances that make sense most of the time > Make it easy to opt out of default instances
61. Default instances > Don’t provide default instances whose behaviour is often undesired > Examples: > PTypeH[AnyRef] > EmptyValue[HttpResponse] with generic Marshaller[Option[A], B]
62. Serializable trait Quantity[A] extends Serializable { def value(x: A): BigDecimal def unit(x: BigDecimal): A def zero: A = unit(BigDecimal(0)) def plus(x: A, y: A): A = unit(value(x) + value(y)) def minus(x: A, y: A): A = unit(value(x) - value(y)) def times(x: A, y: BigDecimal): A = unit(value(x) * y) def div(x: A, y: BigDecimal): A = unit(value(x) / y) } thankful Spark, Crunch, etc. users
63. Implicits FTW?
65. Questions to ask > Is retroactive extension an important use case? > Can you provide good defaults? > How much boilerplate do you avoid? How much clarity do you lose? > Can you support two usage models, one of them not relying on implicits?
66. Summary > Typeclasses enable retroactive extension > Alternative to inheritance and adapters > Provide simple constructors and combinators > Powerful due to automatic derivation > Interactions of multiple typeclasses difficult to understand > Don’t throw typeclasses at every problem
67. Thank you for your attention! Twitter: @kaffeecoder Website: danielwestheide.com