Functional Reactive Processing

Primer into functional and actor based programming

Transcript

1. Functional Reactive Programming Wednesday 2 October 13
2. @gideondk Functional, actor based programming and native application development Gideon de Kok Wednesday 2 October 13
3. Functional / Reactive Programming Wednesday 2 October 13
4. We need “new” tools, why? Wednesday 2 October 13
5. End of “traditional” web era Wednesday 2 October 13
6. Real-time Web / Big Data Wednesday 2 October 13
7. Increasing demands in respect to scalability, fault tolerence Wednesday 2 October 13
8. In terms of vertical scalability : Wednesday 2 October 13
9. Free lunch is over! Herb Sutter, 2005 Wednesday 2 October 13
10. 1970 1975 1980 1985 1990 1995 2000 2005 2010 2015 Transistors Frequency Cores Wednesday 2 October 13
11. “Traditional” threads are too hard to manage Wednesday 2 October 13
12. Shared mutable state, locks, mutexes Wednesday 2 October 13
13. In terms of horizontal scalability : Wednesday 2 October 13
14. Difficult to build and manage distributed enviroments Wednesday 2 October 13
15. Supervision, fault tolerence, redundancy often not by default Wednesday 2 October 13
16. Distributed mutable state, objects, transactions == Distributed evil Wednesday 2 October 13
17. Let’s look at the two specific domains Wednesday 2 October 13
18. Big data applying filtering, transformations, and reductions to huge streams of data Wednesday 2 October 13
19. The real-time Web constant exchange of data, both in large and small packages Wednesday 2 October 13
20. Big data implications Wednesday 2 October 13
21. OO isn’t that well suited transformations are not inherent properties of objects Wednesday 2 October 13
22. A flat imperative model neither imperative functions are not easily composable Wednesday 2 October 13
23. Real-time web implications Wednesday 2 October 13
24. Traditional stacks aren’t suited Wednesday 2 October 13
25. Modern stacks neither ;-) Wednesday 2 October 13
26. Callback hell anyone? fs.readdir(source, function(err, files) { if (err) { console.log('Error finding files: ' + err) } else { files.forEach(function(filename, fileIndex) { console.log(filename) gm(source + filename).size(function(err, values) { if (err) { console.log('Error identifying file size: ' + err) } else { console.log(filename + ' : ' + values) aspect = (values.width / values.height) widths.forEach(function(width, widthIndex) { height = Math.round(width / aspect) console.log('resizing ' + filename + 'to ' + height + 'x' + height) this.resize(width, height).write(destination + 'w' + width + '_' + filename, function(err) { if (err) console.log('Error writing file: ' + err) }) }.bind(this)) } }) }) } }) Wednesday 2 October 13
27. We need a reactive system Wednesday 2 October 13
28. a system capable of reacting to events – load – failure – users Wednesday 2 October 13
29. a system which is event-driven, scalable, resilient and responsive Wednesday 2 October 13
30. (Re)defining the stack Wednesday 2 October 13
31. Starting with a deterministic core is essential : Wednesday 2 October 13
32. Non determinism is the root of all evil in both concurrent as distributed environments Wednesday 2 October 13
33. Starting with a expressive core is just as essential : Wednesday 2 October 13
34. Accidental complexity should be eliminated while working in complex domains Wednesday 2 October 13
35. Welcoming Functional Programming (and Scala) * *Doesn’t need batteries Wednesday 2 October 13
36. State Wednesday 2 October 13
37. State Wednesday 2 October 13
38. Immutable values Wednesday 2 October 13
39. scala> val a = 7 a: Int = 7 scala> a = 9 <console>:8: error: reassignment to val Wednesday 2 October 13
40. val res = async { a + 2 } + async { val d = a; d - 2 } Wednesday 2 October 13
41. Purity Wednesday 2 October 13
42. function(5) == function(5) Wednesday 2 October 13
43. Functions first class citizens Wednesday 2 October 13
44. def a(): Function0[Int] def b(f: Function0[String]) functions taking functions returning functions Wednesday 2 October 13
45. Mapping Wednesday 2 October 13
46. Category Theory Wednesday 2 October 13
47. Objects & Arrows Wednesday 2 October 13
48. def length(s: String): Int = s.length Simple function Wednesday 2 October 13
49. String => Int String maps to Int Wednesday 2 October 13
50. Function as value val length = (s: String) => s.length Wednesday 2 October 13
51. Multiple arguments (Int, Int) => Int Wednesday 2 October 13
52. Multiple arguments val addAndDivide = (a: Int, b: Int) => (b + a) / a addAndDivide(10,4) res0: Int = 3 Wednesday 2 October 13
53. Curried Int => Int => Int Wednesday 2 October 13
54. Curried val addAndDivide = (a: Int) => (b: Int) => (b + a) / a addAndDivide(10)(4) res0: Int = 3 Wednesday 2 October 13
55. Partial application val addTwoAndDivideByTwo: Int => Int = addAndDivide(2) addTwoAndDivideByTwo(20) res1: Int = 11 Wednesday 2 October 13
56. Higher ordered functions Wednesday 2 October 13
57. Higher ordered functions def mapToStringList(l: List[Int], f: Int 㱺 String): List[String] Wednesday 2 October 13
58. Type signature (List[Int], Int 㱺 String) 㱺 List[String] Wednesday 2 October 13
59. More sane example val isInRangeF = (min: Int, max: Int) 㱺 (x: Int) 㱺 x > min && x < max Wednesday 2 October 13
60. Type signature (Int, Int) 㱺 Int 㱺 Boolean Wednesday 2 October 13
61. Usage val newFunction = isInRangeF(20,30) newFunction: Int 㱺 Boolean = <function1> newFunction(22) res0: Boolean = true isInRangeF(40,45)(50) res1: Boolean = false Wednesday 2 October 13
62. In the wild (Ruby) a = [ "a", "b", "c", "d" ] b = a.map {|x| x + "!" } b #=> [ "a!", "b!", "c!", "d!" ] Wednesday 2 October 13
63. In the wild (Scala) List(List(1,2), List(2), List(4,5)).map(_.sum) res0: List(3, 2, 9) List(2,3,4,5,6,7).foldLeft(0)(_+_) res1: Int = 27 List("Hello", "I", "want", "to", "be", "a", "sentence").reduceLeft(_ + " " + _) res2: String = Hello I want to be a sentence Wednesday 2 October 13
64. Map val l = List(1,2,3,4,5,6) l.map(x => x + 2) res0: List[Int] = List(3,4,5,6,7,8) Wednesday 2 October 13
65. Map trait List[Int] { def map[Int](f: Int 㱺 String): List[String] } Wednesday 2 October 13
66. Map trait List[A] { def map[B](f: A 㱺 B): List[B] } Wednesday 2 October 13
67. Deep mapping? Wednesday 2 October 13
68. What is a List? Wednesday 2 October 13
69. List[Int] 1 2 3 4 5 Wednesday 2 October 13
70. Context zero to a infinite number of values Wednesday 2 October 13
71. Map applies function to context Wednesday 2 October 13
72. Map: List applies function to zero or a infinite number of values Wednesday 2 October 13
73. Map also available in different shaped types Wednesday 2 October 13
74. val a: Option[Int] = Some(2) a.map(x => x + 4) res0: Some(6) Option Wednesday 2 October 13
75. val a: Future[String] = Future(“a”) a.map(x => x + “b”) Future Wednesday 2 October 13
76. IO val a: IO[File] a.map(x => x.path) res0: IO[String] Wednesday 2 October 13
77. Context mapping behavior attribute of specific “type class” Wednesday 2 October 13
78. Functor Wednesday 2 October 13
79. trait Functor[A] { def map[B](f: A 㱺 B): Functor[B] } Functor Wednesday 2 October 13
80. Null? def fetchUserFromDatabase(email: String): User val user = fetchUserFromDatabase("gideondk@me.com") var firstName = "" if (user != null) { firstName = user.firstName } else { throw Exception("User doesn't exist") } Wednesday 2 October 13
81. Option! def fetchUserFromDatabase(email: String): Option[User] val firstName: Option[String] = fetchUserFromDatabase("gideondk@me.com").map(_.firstName) Wednesday 2 October 13
82. Option: context a potentially existing value Wednesday 2 October 13
83. Map: Option applies function to a potentially existing value Wednesday 2 October 13
84. Future def fetchFromSite(url: String): Future[HttpResponse] val json: Future[JsObject] = fetchFromSite("http://google.com/ search.json?q=cats").map(x => toJson(x.toString)) val value: JsObject = Await.result(json, 2 seconds) Wednesday 2 October 13
85. Future: context context with a eventually resolved value Wednesday 2 October 13
86. Map: Future applies function to a eventually resolved value Wednesday 2 October 13
87. What happens in context, stays in context Wednesday 2 October 13
88. But what if a function returns a context within a context.. Wednesday 2 October 13
89. within a context within a context within a context within a context within a context? Wednesday 2 October 13
90. Wednesday 2 October 13
91. val userFut: Future[User] = db.user.fetch("user@domain.com") val orgFut: Future[Future[Organisation]] = userFut.map(x 㱺 db.organisation.fetch(x.organisationId)) Wednesday 2 October 13
92. val fut: Future[Organisation] = Await.result(orgFut, 1 second) val org: Organisation = Await.result(org, 1 second) Wednesday 2 October 13
93. Monads Wednesday 2 October 13
94. Monads: flatten multiple contexts as a “stack” Wednesday 2 October 13
95. trait Monad[A] { def flatMap[B](f: A 㱺 Monad[B]): Monad[B] } Monad Wednesday 2 October 13
96. trait Future[A] { def flatMap[B](f: A 㱺 Future[B]): Future[B] } Future Monad Wednesday 2 October 13
97. Future: Monad val userFut: Future[User] = db.user.fetch("user@domain.com") val orgFut: Future[Organisation] = userFut.flatMap(x 㱺 db.organisation.fetch(x.organisationId)) val org: Organisation = Await.result(orgFut, 1 second) Wednesday 2 October 13
98. Option: Monad val email: Option[String] = request.headers.get(“email”) val domain: Option[String] = email.flatMap(x => x.split("@").tail.headOption) val domainLength: Option[Int] = domain.map(_.length) Wednesday 2 October 13
99. Multiple Monads db.user.fetch("user@domain.com").flatMap { u => db.organisation.fetch(u.organizationId).flatMap { o 㱺 db.city.fetch(o.cityId) flatMap { c => db.country.fetch(c.countryId) } } } Wednesday 2 October 13
100. Multiple Monads for { user <- db.user.fetch("user@domain.com") org <- db.organisation.fetch(user.organizationId) city <- db.city.fetch(org.cityId) country <- db.coutry.fetch(city.countryId) } yield country Wednesday 2 October 13
101. Break! Wednesday 2 October 13
102. Wednesday 2 October 13
103. Dataflows Wednesday 2 October 13
104. Dataflows Wednesday 2 October 13
105. Contexts are able to “box” behavior Wednesday 2 October 13
106. Contexts help to seperate the determistic and non-deterministic Wednesday 2 October 13
107. Contexts purify the unpure* *Book of Monad – 2:22 Wednesday 2 October 13
108. While immutability helps determinism and contexts box non-determinism Wednesday 2 October 13
109. A lot of data flows are based on state Wednesday 2 October 13
110. Concurrent state Wednesday 2 October 13
111. State monads work great in singular data flows Wednesday 2 October 13
112. But what should we use to enable stateful programming in concurrent and distributed environments? Wednesday 2 October 13
113. Concurrency, Parallelism? Wednesday 2 October 13
114. customers cashier Wednesday 2 October 13
115. customers cashiers Wednesday 2 October 13
116. Concurrency & Parallelism no or, but and! Wednesday 2 October 13
117. Concurrency is hard… Wednesday 2 October 13
118. Techological Error handling is tricky Race conditions, mutexes, locks Wednesday 2 October 13
119. Cognitive Concurrency proof data structures are difficult to design Understanding in-code bottlenecks is painful Bugfixing and error hunting sometimes mere impossible Wednesday 2 October 13
120. Multiple concurrency methologies Wednesday 2 October 13
121. Shared-state concurrency Wednesday 2 October 13
122. class User(var firstName: String, var lastName: String) { ! def setFirstName(name: String) = firstName = name ! def setLastName(name: String) = lastName = name ! def getFullName = firstName + " " + lastName } val user = User("John", "Doe") val a = Async { ! user.setFirstName("Johnny") ! user.setLastName("Walker") } val b = Async { ! user.getFullName } b == "Johnny Doe" Wednesday 2 October 13
123. class User(var firstName: String, var lastName: String) { ! val lock = new ReentrantLock ! def setFirstName(name: String) = { ! ! lock.lock ! ! firstName = name ! ! lock.unlock ! } ! def setLastName(name: String) = { ! ! lock.lock ! ! lastName = name ! ! lock.unlock ! } ! def getFullName = { ! ! lock.lock ! ! firstName + " " + lastName ! ! lock.unlock ! } } // Thread 1 user.setFirstName("Johnny") // Thread 2 user.getFullName // Thread 1 user.setLastName("Walker") Wednesday 2 October 13
124. Wednesday 2 October 13
125. Message passing for concurrency Wednesday 2 October 13
126. What would it look like if we didn’t create the problem in the first place? Wednesday 2 October 13
127. Focussing on isolated state through non-mutable messages Wednesday 2 October 13
128. Welcoming actors! Wednesday 2 October 13
129. Let’s illustrate the concept! Wednesday 2 October 13
130. request response request parser response formatter data query data result ? ! Wednesday 2 October 13
131. request response request parser response formatter data query data result ? ! Wednesday 2 October 13
132. Wednesday 2 October 13
133. customer cashier barista Wednesday 2 October 13
134. m oney coffee prepare order retrieve order em pty cup filled cup Wednesday 2 October 13
135. m oney coffee prepare order retrieve order em pty cup filled cup Wednesday 2 October 13
136. scaling the café (naively) Wednesday 2 October 13
137. Wednesday 2 October 13
138. work duration 15s 120s 5s Wednesday 2 October 13
139. unnecessary scaling of cashiers Wednesday 2 October 13
140. businesses scale where it hurts Wednesday 2 October 13
141. incoming served waiting Wednesday 2 October 13
142. incoming served waiting Wednesday 2 October 13
143. Wednesday 2 October 13
144. state behavior mailbox Actor Wednesday 2 October 13
145. Actor state behavior mailbox event driven threat Wednesday 2 October 13
146. state behavior mailbox event driven threat Actor Wednesday 2 October 13
147. Tell & Ask / ! & ? Wednesday 2 October 13
148. val printer = actor(new Act { ! become { ! ! case x: String ⇒ println(x) ! } }) printer ! "Print this!" Wednesday 2 October 13
149. val a = actor(new Act { ! become { ! ! case "hello" ⇒ sender ! "hi" !} }) val result: Future[String] = (a ? "hello").mapTo[String] Wednesday 2 October 13
150. In action! Wednesday 2 October 13
151. object CoffeeShop { ! case class Money ! case class Coffee ! case class Cup(contents: Option[Coffee]) ! case class OrderCoffee(cashier: ActorRef) ! implicit val system = ActorSystem("coffee-shop") ! class Customer extends Actor { ! var money = Some(Money()) ! var cup = None ! def receive = { ! ! case OrderCoffee(cashier) => ! ! ! val m = money ! ! ! money = None ! ! ! cashier ! m ! ! case c: Cup => ! ! ! cup = c ! ! ! println("Whooohooo! Coffee :-)") ! } } } Wednesday 2 October 13
152. class Cassier(barista: ActorRef) extends Actor { ! var orders = Map[String, ActorRef] ! def receive = { ! ! case Money => ! ! ! val customer = sender ! ! ! val newOrder = Order(java.util.UUID.randomUUID.toString) ! ! ! orders = orders ++ (newOrder.id -> customer) ! ! ! val cup = Cup(contents = None) ! ! ! barista ! (order, cup) ! ! case x: (Order, Cup) => val customer = orders(x._1.id) ! ! customer ! x._2 ! } } Wednesday 2 October 13
153. class Barista extends Actor { ! def receive = { ! ! case x: (Order, Cup) => ! ! ! sender ! (x._1 -> prepareCoffee(x._2))) ! } ! def prepareCoffee(cup: Cup) = { ! ! thread.sleep(120) ! ! cup.copy(contents = Some(Coffee())) ! } } Wednesday 2 October 13
154. val customer = system.actorOf(Props[Customer]) val barista = system.actorOf(Props[Barista]) val cassier = system.actorOf(Props(new Cassier(barista))) customer ! OrderCoffee(cassier) Wednesday 2 October 13
155. One actor processes one message at a time Wednesday 2 October 13
156. “Actors do one thing at a time, they do it very well, and then they quickly move on.” Wednesday 2 October 13
157. Multiple actors proces multiple messages at a time Wednesday 2 October 13
158. Meet routers! Wednesday 2 October 13
159. ? Wednesday 2 October 13
160. Transparent load balancing Wednesday 2 October 13
161. Without any additional code in actor implementation Wednesday 2 October 13
162. val baristas = system.actorOf(Props[Barista]).withRouter(RoundRobinRouter(nrOfInstances = 5))) val cassiers = system.actorOf(Props(new Cassier(barista))).withRouter(RoundRobinRouter(nrOfInstances = 3))) customer ! OrderCoffee(cassiers) customerTwo ! OrderCoffee(cassiers) customerThree ! OrderCoffee(cassiers) Wednesday 2 October 13
163. People die, actors too Wednesday 2 October 13
164. Exception handling in threads is tough Wednesday 2 October 13
165. Barista burns hand :-( Wednesday 2 October 13
166. Probable choice: stop working, tell supervisor Wednesday 2 October 13
167. Supervisor responsable for barista replacement Wednesday 2 October 13
168. Actor state behavior Wednesday 2 October 13
169. Actor state behavior Wednesday 2 October 13
170. Actor behavior Supervisor Escalate Wednesday 2 October 13
171. class CoffeeShopOwner extends Actor { ! var baristas: List[Barista] ! override val supervisorStrategy = ! OneForOneStrategy(maxNrOfRetries = 10, withinTimeRange = 1 minute) { ! case _: BurnedHandException ⇒ Resume ! case _: PassedOut ! ! ⇒ Restart ! case _: Died ! ! ! ! ⇒ Stop ! case _: Exception ⇒ Escalate ! } } Wednesday 2 October 13
172. Actors are restarted by default (UserGuardian) Wednesday 2 October 13
173. Just let it crash™ Wednesday 2 October 13
174. But what about multiple server instances? Wednesday 2 October 13
175. Actors run transparantly local & remote Wednesday 2 October 13
176. val actor = context.actorOf[Barista]("my-service") Wednesday 2 October 13
177. akka { ! actor { ! ! deployment { ! ! ! /path/to/service { ! ! ! ! router = "round-robin" ! ! ! ! nr-of-instances = 2 ! ! ! ! remote { ! ! ! ! ! nodes = ["node1:2552", "node2:2552"] ! ! ! ! } ! ! ! } ! ! } ! } } Wednesday 2 October 13
178. Write as local, deploy in the cloud (zero code change) Wednesday 2 October 13
179. ? Wednesday 2 October 13
180. Pushing configurations can be cumbersome Wednesday 2 October 13
181. Akka Cluster Wednesday 2 October 13
182. Functional Groningen Wednesday 2 October 13