Extensible Effects: beyond the Monad Transformers

This presentation includes these following three things:
1. Monad
2. Monad Transformers
3. Extensible-Effects
Main topics are how to use atnos-eff and comparison between mtl and eff.

1. Extensible-Effects: beyond the Monad Transformers Hiroki Fujino Unipos GmbH 1
2. Introduction 2 • Hiroki Fujino • Head of Engineer in Unipos • Came from Japan since this February • My interests: Scala, Architecture Design, Concurrency Programming, etc.
3. Agenda • Monad • Monad Transformers • Extensible Effects 3 Prerequisites: Basic knowledge of Scala(includes Experience with using Monad) https://github.com/Hiroki6/SampleAtnosEff Sample Code:
4. Monad 4 Allows composition of dependent effectful functions. Option Future Either Writer Reader State IO https://typelevel.org/cats/typeclasses/monad.html
5. Monad 5 trait Monad[M[_]] { def pure[A](x: A): M[A] def map[A, B](x: M[A])(f: A => B): M[B] def flatMap[A, B](x: M[A])(f: A => M[B]): M[B] }
6. Option Monad 6 sealed trait Option[+A] case object None extends Option[Nothing] case class Some[A](value: A) extends Option[A] new Monad[Option] { def pure[A](x: A) = Some(x) def flatMap[A, B](x: Option[A])(f: A => Option[B]): Option[B] = x match { case Some(x) => f(x) case None => None } }
7. Option Monad 7 def hogeOption(a: Int): Option[Int] = Some(a) hogeOption(1).flatMap(a => hogeOption(a).map(b => a+b)) // Some(2) for { a <- hogeOption(1) b <- hogeOption(a) } yield (a + b) // Some(2)
8. 8 How to use multi monads?
9. Two Monads in program 9 def findUser(userId: UserId): IO[Option[User]] def getTeam(userId: UserId): IO[Option[Team]] def createResponse(user: User, team: Team): Response def program(userId: UserId): IO[Option[Response]] = for { userOpt <- findUser(userId) teamOpt <- getTeam(userId) user <- userOpt team <- teamOpt } yield createResponse(user, team)
10. 10 [error] found : Option[Response] [error] required: cats.effect.IO[Option[Response]] [error] user <- userOpt def findUser(userId: UserId): IO[Option[User]] def getTeam(userId: UserId): IO[Option[Team]] def createResponse(user: User, team: Team): Response def program(userId: UserId): IO[Option[Response]] = for { userOpt <- findUser(userId) teamOpt <- getTeam(userId) user <- userOpt team <- teamOpt } yield createResponse(user, team) } Composition of IO } Composition of Option Two Monads with For-statement
11. Two Monads with For-statement 11 def program(userId: UserId): IO[Option[Response]] = for { userOpt <- findUser(userId) teamOpt <- getTeam(userId) } yield { (userOpt, teamOpt) match { case (Some(user), Some(team)) => Some(createResponse(user, team)) case (_, _) => None } } def program(userId: UserId): IO[Option[Response]] = for { userOpt <- findUser(userId) teamOpt <- getTeam(userId) } yield for { user <- userOpt team <- teamOpt } yield createResponse(user, team)
12. Monad Transformers 12
13. Monad Transformers 13 G F •Popular solution to create monad stack •Monads do not compose(G[_], F[_] => F[G[_]]) •But, if G[_] is implemented monad(Option, Either, etc.), F[G[_]] can be monad •Option •Either •Reader etc.
14. Monad Transformers 14 case class OptionT[F[_], A](value: F[Option[A]]) { def map[B](f: A => B)(implicit F: Functor[F]): OptionT[F, B] = OptionT(F.map(value)(_ map f)) def flatMap[B](f: A => OptionT[F, B])(implicit F: Monad[F]): OptionT[F, B] = OptionT(F.flatMap(value)(_.fold(F.pure[Option[B]](None))(f(_).value))) } Option F
15. Two Monads with OptionT 15 def program(userId: UserId): OptionT[IO, Response] = for { user <- OptionT(findUser(userId)) team <- OptionT(getTeam(userId)) } yield createResponse(user, team) program(userId).value // IO[Option[Response]] Option IO def flatMap[B](f: A => OptionT[F, B])(implicit F: Monad[F]): OptionT[F, B] } Composition of OptionT
16. Implementation of Monad Transformers 16 OptionT EitherT WriterT ReaderT StateT These are implemented in cats and Scalaz.
17. 17 Transformer can wrap another transformer type EitherIO[A] = EitherT[IO, Error, A] OptionT[EitherIO, A] // OptionT[EitherT[IO, Error, A]] IO Either Option
18. Three Monad in Program 18 type EitherIO[A] = EitherT[IO, Error, A] def program(userId: String, teamName: String): OptionT[EitherIO, Response] = { for { user <- OptionT[EitherIO, User] { EitherT.liftF[IO, Error, Option[User]](findUser(userId)) } team <- OptionT.liftF[EitherIO, Team] { EitherT.fromEither[IO](createTeam(userId, teamName)) } _ <- OptionT.liftF[EitherIO, Unit] { EitherT.liftF[IO, Error, Unit](storeTeam(team)) } } yield createResponse(user, team) } program(userId, teamName) // OptionT[EitherIO, Response] .value // EitherIO[Option[Response]] == EitherT[IO, Error, Option[Response]] .value // IO[Either[Error, Option[Response]]] IO[Option[User]] Either[Error, Team] IO[Unit]
19. Three Monad in Program 19 type EitherIO[A] = EitherT[IO, Error, A] def program(userId: String, teamName: String): OptionT[EitherIO, Response] = { for { user <- OptionT[EitherIO, User] { EitherT.liftF[IO, Error, Option[User]](findUser(userId)) } team <- OptionT.liftF[EitherIO, Team] { EitherT.fromEither[IO](createTeam(userId, teamName)) } _ <- OptionT.liftF[EitherIO, Unit] { EitherT.liftF[IO, Error, Unit](storeTeam(team)) } } yield createResponse(user, team) } program(userId, teamName) // OptionT[EitherIO, Response] .value // EitherIO[Option[Response]] == EitherT[IO, Error, Option[Response]] .value // IO[Either[Error, Option[Response]]] IO[Option[User]] Either[Error, Team] IO[Unit]
20. Issues of Monad Transformers • Cumbersome lift • Overhead of composition • Fixed order of Monad Stack 20
21. Cumbersome lift 21 Option Either IO def storeTeam(team: Team): IO[Unit] OptionT.liftF[EitherIO, Unit] { EitherT.liftF[IO, Error, Unit](storeTeam(team)).map(_.some) } IO Either IO lift lift
22. Overhead of composition 22 OptionT { Monad[EitherIO].flatMap(value)(_.fold(F.pure[Option[User]](None))(f(_).value)) } OptionT[EitherIO, User](EitherT.liftF[IO, Error, Option[User]](findUser(userId))).flatMap(f) EitherT(Monad[IO].flatMap(value) { case l @ Left(_) => F.pure(l.rightCast) case Right(b) => f(b).value })
23. Fixed order of Monad Stack 23 program.value // Option[Either[String, Int]] Definition Execution Either Option Option Either ❌ def program: EitherT[Option, String, Int] = for { t <- EitherT.fromEither[Option](Right(1)) result <- EitherT.liftF[Option, String, Int](Some(t)) } yield result
24. 24 Extensible Effects
25. 25 “Extensible Effects: an alternative to monad transformers” –Oleg Kiselyov in 2013
26. Extensible Effects(Eff) Library in Scala 26 •atnos-eff •emm
27. Today’s Topic about Eff 27 atnos-eff •How to use atnos-eff •Advantage and Disadvantage •How to define your own Eff monad •Detail of atons-eff(Union Type, Interpreter) Not covered: Explain these below in this presentation:
28. Eff 28 Option State •Eff is monad. •Eff has effect stack in a flat state. •List of effects. order of effects is not important. Either Reader ex. Fx.fx4[Reader[Int, ?], Either[String, ?], Option, State[Int, ?]]
29. Monad Transformers Eff Comparison of concept 29 Definition Execution Option Either Either Option Option Either Either Option Either Option
30. Monad Transformers Eff Comparison of concept 30 Definition Execution Option Either Either Option Either Option Either Option No layer of Monad Stack No need to lift Option Either
31. MTL vs atnos-eff 31 Monad Transformers Eff Definition Execution program.value // Option[Either[String, Int]] def program: EitherT[Option, String, Int] = for { t <- EitherT.fromEither[Option](Right(1)) result <- EitherT.liftF[Option, String, Int](Some(t)) } yield result def program[R :_stringEither :_option]: Eff[R, Int] = for { t <- fromEither[R, String, Int](Right(1)) result <- fromOption[R, Int](Some(t)) } yield result type Stack = Fx.fx2[Option, StringEither] // Either[String, Option[Int]] program[Stack].runOption.runEither.run // Option[Either[String, Int]] program[Stack].runEither.runOption.run
32. atnos-eff 32 Definition Execution def program[R :_stringEither :_option]: Eff[R, Int] = for { t <- fromEither[R, String, Int](Right(1)) result <- fromOption[R, Int](Some(t)) } yield result Smart Constructor Eff Monad type Stack = Fx.fx2[Option, StringEither] program[Stack] // Eff[Fx2[Option, StringEither], Int] .runOption // Eff[Fx1[StringEither], Option[Int]] .runEither // Eff[NoFx, Either[String, Option[Int]]] .run // Either[String, Option[Int]] Interpreter
33. Eff Monad 33 import org.atnos.eff.{Eff, Member} import org.atnos.eff.either._ import org.atnos.eff.option._ type StringEither[A] = String Either A type _stringEither[R] = StringEither |= R def program[R :_stringEither :_option]: Eff[R, Int] type Stack = Fx.fx2[Option, StringEither] program[Stack] Eff sealed trait Eff[R, A] { def map[B](f: A => B): Eff[R, B] def flatMap[B](f: A => Eff[R, B]): Eff[R, B] } Effect Stack Option Either https://github.com/atnos-org/eff/blob/master/shared/src/main/scala/org/atnos/eff/Eff.scala
34. atnos-eff 34 https://atnos-org.github.io/eff/org.atnos.site.OutOfTheBox.html
35. Smart Constructor 35 trait OptionCreation { type _option[R] = Option |= R /** create an Option effect from a single Option value */ def fromOption[R :_option, A](o: Option[A]): Eff[R, A] = send[Option, R, A](o) } Option fromOption Eff Transforms effect into Eff Option Either https://github.com/atnos-org/eff/blob/master/shared/src/main/scala/org/atnos/eff/ OptionEffect.scala
36. atnos-eff 36 Definition import org.atnos.eff.either._ import org.atnos.eff.option._ def program[R :_stringEither :_option]: Eff[R, Int] = for { t <- fromEither[R, String, Int](Right(1)) result <- fromOption[R, Int](Some(t)) } yield result Smart Constructor
37. Interpreter 37 trait OptionInterpretation { def runOption[R, U, A](effect: Eff[R, A])(implicit m: Member.Aux[Option, R, U]): Eff[U, Option[A]] } implicit class OptionEffectOps[R, A](e: Eff[R, A]) { def runOption(implicit member: Member[Option, R]): Eff[member.Out, Option[A]] = OptionInterpretation.runOption(e)(member.aux) } runOption Eff Can execute the effect Option Either https://github.com/atnos-org/eff/blob/master/shared/src/main/scala/org/atnos/eff/ OptionEffect.scala
38. Interpreter 38 import org.atnos.eff.syntax.option._ import org.atnos.eff.Fx /** Stack declaration **/ type Stack = Fx.fx2[Option, StringEither] program[Stack] // Eff[Fx2[Option, StringEither], Int] .runOption // Eff[Fx1[StringEither], Option[Int]] Option Can execute the effect runOption Eff Either
39. atnos-eff 39 Definition Execution def program[R :_stringEither :_option]: Eff[R, Int] = for { t <- fromEither[R, String, Int](Right(1)) result <- fromOption[R, Int](Some(t)) } yield result import org.atnos.eff.syntax.either._ import org.atnos.eff.syntax.option._ import org.atnos.eff.syntax.eff._ type Stack = Fx.fx2[Option, StringEither] program[Stack] // Eff[Fx2[Option, StringEither], Int] .runOption // Eff[Fx1[StringEither], Option[Int]] .runEither // Eff[NoFx, Either[String, Option[Int]]] .run // Either[String, Option[Int]] Interpreter
40. Three effects in eff 40 type ErrorOr[A] = Either[Error, A] type _option[R] = Option |= R type _io[R] = |=[IO, R] type _errorOr[R] = ErrorOr |= R def program[R :_io :_option :_errorOr] (userId: UserId, teamName: TeamName): Eff[R, Response] = for { userOpt <- fromIO(findUser(userId)) user <- fromOption(userOpt) team <- fromEither(createTeam(userId, teamName)) _ <- fromIO(storeTeam(team)) } yield createResponse(user, team) type Stack = Fx.fx3[Option, ErrorOr, IO] program[Stack](userId, teamName) .runOption // Eff[Fx2[Either, IO], Option[Response]] .runEither[Error] // Eff[Fx1[IO], Either[Error, Option[Response]] .to[IO] // IO[Either[Error, Option[Response]]]
41. type ErrorOr[A] = Either[Error, A] type _option[R] = Option |= R type _io[R] = |=[IO, R] type _errorOr[R] = ErrorOr |= R def program[R :_io :_option :_errorOr] (userId: String, teamName: String): Eff[R, Response] = for { userOpt <- fromIO(findUser(userId)) user <- fromOption(userOpt) team <- fromEither(createTeam(userId, teamName)) _ <- fromIO(storeTeam(team)) } yield createResponse(user, team) MTL Eff type Stack = Fx.fx3[Option, ErrorOr, IO] program[Stack](userId, teamName) .runOption // Eff[Fx2[Either, IO], Option[Response]] .runEither[Error] // Eff[Fx1[IO], Either[Error, Option[Response]] .to[IO] // IO[Either[Error, Option[Response]]] program(userId, teamName) // OptionT[EitherIO, Response] .value // EitherT[IO, Error, Option[Response]] .value // IO[Either[Error, Option[Response]]] type EitherIO[A] = EitherT[IO, Error, A] def program(userId: String, teamName: String): OptionT[EitherIO, Response] = { for { user <- OptionT[EitherIO, User] (EitherT.liftF[IO, Error, Option[User]](findUser(userId))) team <- OptionT.liftF[EitherIO, Team] (EitherT.fromEither[IO](createTeam(userId, teamName))) _ <- OptionT.liftF[EitherIO, Unit] (EitherT.liftF[IO, Error, Unit](storeTeam(team))) } yield createResponse(user, team) } MTL vs atnos-eff 41
42. Apply Eff to entire program 42 class CreatePostService[R: _io: _connectionIO: _errorOr] (…) { override def execute(in: CreatePostParam): Eff[R, CreatePostDTO] = { for { _ <- userRepository.resolve[R](Id(in.userId)).validate("user is not exist.") post <- Post.create[R](in.userId, in.text, None) _ <- postRepository.store[R](post) } yield CreatePostDTO(post.id.value) } } def resolve[R: _connectionIO](id: Id[E]): Eff[R, Option[E]] def store[R: _connectionIO](entity: E): Eff[R, Unit] Service Repository Domain Model def create[R: _errorOr](…): Eff[R, Post]
43. Apply Eff to entire program 43 trait CreatePostController { type R = Fx.fx3[IO, ConnectionIO, Validated] def run(createPostParam: CreatePostParam): IO[ThrowableEither[ErrorOr[CreatePostDTO]]] = CreatePostService[R] .execute(createPostParam) .runValidatedNel[String] .runConnectionIO(Database.xa) .ioAttempt .to[IO] } Controller
44. Advantage of Eff 44 • Readability • Easy to compose • No match type game for { userOpt <- fromIO(findUser(userId)) user <- fromOption(userOpt) team <- fromEither(createTeam(userId, teamName)) _ <- fromIO(storeTeam(team)) } yield createResponse(user, team)
45. Disadvantage of Eff 45 • Dependency of syntax • Confused to multiple run
46. Dependency of syntax 46 type R = Fx.fx3[IO, ConnectionIO, Validated] CreatePostService[R].execute(createPostParam) def resolve[R: _connectionIO](id: Id[E]): Eff[R, Option[E]] def create[R: _errorOr](userId: String, text: String, parentPostId: Option[String]): Eff[R, Post] class CreatePostService[R: _io: _connectionIO: _errorOr] Controller Service Repository Domain Model
47. Confused to multiple run 47 type Stack = Fx.fx3[Option, ErrorOr, IO] program[Stack](userId, teamName) .runOption // Eff[Fx2[Either, IO], Option[Response]] .runEither[Error] // Eff[Fx1[IO], Either[Error, Option[Response]] .to[IO] // IO[Either[Error, Option[Response]] program[Stack](userId, teamName) .runEither[Error] // Eff[Fx2[Option, IO], Either[Error, Response]] .runOption // Eff[Fx1[IO], Option[Either[Error, Response]] .to[IO] // IO[Option[Either[Error, Response]]])
48. Summary 48 •Compose two monads •Compose more than three monads •Entire Architecture Monad Transformer is suit. Eff is better. Eff is good, but there are some challenges.
49. References 49 •Emm A Sane Alternative to Monad Transformers in Scala •Monad transformers down to earth •A Journey into Extensible Effects in Scala •Extensible Effects vs Monad Transformers
50. Appendix 50 type Stack = Fx.fx2[Option, StringEither] def program: Eff[Stack, Int] = for { t <- fromEither[Stack, String, Int](Right(1)) result <- fromOption[Stack, Int](Some(t)) } yield result program.runOption.runEither.run = def program[R :_stringEither :_option]: Eff[R, Int] = for { t <- fromEither[R, String, Int](Right(1)) result <- fromOption[R, Int](Some(t)) } yield result type Stack = Fx.fx2[Option, StringEither] program[Stack].runOption.runEither.run
51. Appendix 51 def program[R: _stateInt :_option]: Eff[R, Int] def program[R](implicit stateInt: _stateInt[R], option: _option[R]): Eff[R, Int] =
52. Appendix 52 /** create an Either effect from a single Either value */ def fromEither[R, E, A](Either: E Either A)(implicit member: (E Either ?) |= R): Eff[R, A] = Either.fold[Eff[R, A]](left[R, E, A], right[R, E, A]) /** create an Either effect from a value possibly throwing an exception */ def fromCatchNonFatal[R, E, A](a: =>A)(onThrowable: Throwable => E)(implicit member: (E Either ?) |= R): Eff[R, A] = fromEither(Either.catchNonFatal(a).leftMap(onThrowable)) /** create an Either effect from a value possibly throwing a Throwable */ def catchNonFatalThrowable[R, A](a: =>A)(implicit member: (Throwable Either ?) |= R): Eff[R, A] = fromCatchNonFatal(a)(identity) SmartConstructor is not only one each Effect. https://github.com/atnos-org/eff/blob/master/shared/src/main/scala/org/atnos/eff/ EitherEffect.scala
53. Appendix 53 Interpreter is not only one each Effect. https://github.com/atnos-org/eff/blob/master/shared/src/main/scala/org/atnos/eff/ EitherEffect.scala /** catch possible left values */ def attemptEither[R, E, A](effect: Eff[R, A])(implicit member: (E Either ?) /= R): Eff[R, E Either A] = catchLeft[R, E, E Either A](effect.map(a => Either.right(a)))(e => pure(Either.left(e))) /** catch and handle a possible left value */ def catchLeft[R, E, A](effect: Eff[R, A])(handle: E => Eff[R, A])(implicit member: (E Either ?) /= R): Eff[R, A] = catchLeftEither[R, E, A](effect)(handle)(cats.instances.either.catsStdInstancesForEither[E]) /** run the Either effect, handling E (with effects) and yielding A */ def runEitherCatchLeft[R, U, E, A](r: Eff[R, A])(handle: E => Eff[U, A])(implicit m: Member.Aux[(E Either ?), R, U]): Eff[U, A] = runEither(r).flatMap(_.fold(handle, pure))