Share
Embed
Transcript
66 slides
-
akka-stream Λ࢝ΊΔͱ͖ʹ͓֮͑ͯ ͖͍ͨ͜ͱ Տ ਸ Twitter: @kawachi ɹ ɹ Photo
by Junkichi Egashira is licensed under CC BY 4.0 International 1 -
͍ɻIterator ͰͰ͖·͢ɻ def q1(fileName: String, n: Int): Boolean = {
val src = io.Source.fromFile(fileName) val lines: Iterator[String] = src.getLines() var count = 0 while (lines.hasNext && n < count) { lines.next() count += 1 } count == n } 4 -
՝1: ϦιʔεཧͰؒҧ͍͍͢ def q1Again(fileName: String, n: Int): Boolean = {
val src = io.Source.fromFile(fileName) try { val lines: Iterator[String] = src.getLines() var count = 0 while (lines.hasNext && n < count) { lines.next() count += 1 } count == n } finally { src.close() } } 5 -
def hasMoreThan(lines: Iterator[String], n: Int) = { var count =
0 while (lines.hasNext && count < n) { lines.next() count += 1 } count == n } def q2(fileName: String, n: Int, m: Int) = { val src = io.Source.fromFile(fileName) try { val lines: Iterator[String] = src.getLines() (hasMoreThan(lines, n), hasMoreThan(lines.filter(_.isEmpty), m)) } finally { src.close() } } 7 -
՝2: εςʔτϑϧͳͷͰؒҧ͍͑͢ val lines: Iterator[String] = src.getLines() (hasMoreThan(lines, n), hasMoreThan(lines.filter(_.isEmpty),
m)) // ^^^^^ Good ^^^^^ !!! >_< 8 -
՝3: εϨου͕ block ͞ΕΔ Iterator .hasNext ͱ .next() ͕ϕʔεɻ
while (lines.hasNext) { lines.next() // <- !!! blocking !!! … } 9 -
• ՝1: ϦιʔεཧͰؒҧ͍͍͢ • → ϦιʔεཧΛϑΝΠϧಡΈࠐΈ Actor ʹہॴԽͰ͖Δɻͦͷ Actor ֎
Ͱؒҧ͑ͳ͍ɻ • ՝2: εςʔτϑϧͳͷͰؒҧ͍͑͢ • → Actor Λ࡞Δࡍͷ akka.actor.Props immutable ͳɻ࠶ར༻Մೳɻ • ՝3: εϨου͕ block ͞ΕΔ • → ϝοηʔδۦಈͳͷͰඇಉظ API Λࣗવʹѻ͑Δ શղܾʂ 11 -
ࠓճհ͢Δ akka-stream Ͱ Actor ͱൺͯ • ߏཁૉ͕ग़ྗ/ೖྗ͢Δཁૉʹܕ͕͍͍ͭͯΔ • όοϑΝᷓΕ͠ͳ͍ Iterator
ͱൺͯ • Ϧιʔεཧ͕ہॴԽ͞Εؒҧ͑ʹ͍͘ • ߏཁૉ͕ immutable ͳ • ඇಉظ API ͱΈ߹ΘͤΒΕΔ 13 -
ґଘੑՃ Play 2.5 ͔Β Play ͷ࣮ʹ akka-stream ͕ΘΕ͍ͯΔ → ৽ͨʹґଘੑΛՃ͢Δඞཁͳ͠
Play ڥͰͳ͍߹ґଘੑΛՃ libraryDependencies += "com.typesafe.akka" %% "akka-stream" % "2.4.11" 15 -
ྫ: 1~5 Λྲྀͯ͠ * 2 ͯ͠ println() import akka.actor.ActorSystem import
akka.stream.ActorMaterializer import akka.stream.scaladsl._ val src = Source(1 to 5) val flow = Flow[Int].map(_ * 2) val sink = Sink.foreach[Int](println) val stream = src.via(flow).to(sink) implicit val system = ActorSystem() implicit val materializer = ActorMaterializer() stream.run() 17 -
ετϦʔϜͷهड़ val src = Source(1 to 5) val flow =
Flow[Int].map(_ * 2) val sink = Sink.foreach[Int](println) val stream = src.via(flow).to(sink) • هड़ ͱ࣮ߦ͕͞Ε͍ͯΔ • هड़ઃܭॻΛॻ͘࡞ۀ • ෭࡞༻ͳ͠ • src, flow, sink, stream ͷ࠶ར༻OK! 18 -
RunnableGraph RunnableGraph run() ՄೳͳετϦ ʔϜͷهड़ɻ run() ͢Δʹೖग़ྗ͕શͯଓ͞Ε ͍ͯΔඞཁ͕͋Δɻ Source
Flowɺ Sink run() Ͱ͖ͳ ͍ɻ 21 -
ετϦʔϜͷ࣮ߦ • ࣮ߦ։࢝͢Δ͜ͱΛ materialize (͋Δ͍ run)ͱݺͿ • ࣮ߦʹ Materializer ͕ඞཁ
• Ұهड़ͨ͠ετϦʔϜԿͰ࣮ߦͰ͖Δ implicit val system = ActorSystem() implicit val materializer = ActorMaterializer()(system) stream.run()(materializer) 22 -
run() ͭ·Γ materialize ͷΓɻ ͱͯ͠ಘΒΕΔͷ Materialized value ͷΈɻ ༻్ •
੍ޚϙΠϯτ • ֎෦͔ΒετϦʔϜΛ੍ޚ͢Δ • ࢹϙΠϯτ • ࣮ߦঢ়ଶॲཧ݁Ռ 24 -
val stream: RunnableGraph[Future[IOResult]] = FileIO.fromPath(Paths.get("src_file")) .to(FileIO.toPath(Paths.get("dest_file"))) val result: Future[IOResult] =
stream.run() // ↑ Materialized value result.onSuccess { case ioResult => println(s"${ioResult.count} byteॲཧ͠·ͨ͠") } 25 -
RunnableGraph ͷ(ຊͷ)ܕ RunnableGraph[+Mat] Mat ͕ materialized value ͷܕ run() ͢Δͱ
Mat ܕͷ materialized value ͕Γͱͯ͠ฦ͞ ΕΔɻ 26 -
Source, Flow, Sink ͷ(ຊͷ)ܕ Source[+Out,+Mat] Flow[-In,+Out,+Mat] Sink[-In,+Mat] ߏཁૉ (Source, Flow,
Sink) ͕ͦΕͧΕ Mat ܕͷ materialized value Λ࣋ͭɻ ͜ΕΒΛΈ߹ͤͯ RunnableGraph Λ࡞ͬͯ run() ͢Δɻ run() ͷΓͱͯ͠ಘΒΕΔͷ RunnableGraph ͷ Mat ͚ͩɻ 27 -
߹࣌ͷ materialized value • via(), to() ࠨଆͷ materialzed value Λ߹ޙͷ
materialized value ʹ࠾༻ • viaMat(), toMat() Ͱࣗ༝ʹͰ͖Δ import akka.stream.scaladsl.Keep source.viaMat(flow)(Keep.left) // ࠨͷ Mat (σϑΥϧτ) source.viaMat(flow)(Keep.right) // ӈͷ Mat source.viaMat(flow)(Keep.both) // ྆ Mat ͷλϓϧ source.viaMat(flow)(Keep.none) // NotUsed 30 -
Source, Flow ڞ௨ͷجຊతͳૢ࡞ map(), filter(), filterNot(), take(), takeWhile(), drop(), dropWhile(),
fold(), reduce() ඪ४ϥΠϒϥϦͷίϨΫγϣϯAPIʹࣅͨϝιου܈ Source Flow ͷग़ྗཁૉ͕ίϨΫγϣϯͷཁૉʹରԠ͢Δͱ ଊ͑Εײతʹૢ࡞Մೳ flatMap() ͳ͍ɻmapConcat() ͋Δ͍ flatMapConcat() ͕૬ɻ 32 -
ετϦʔϜͳΒͰͷૢ࡞ • takeWithin(d: FiniteDuration) • dropWithin(d: FiniteDuration) • ࢦఆ࣌ؒʹ౸ணͨ͠ͷ͚ͩऔΓग़͢/ࣺͯΔ •
groupedWithin(n: Int, d: FiniteDuration) • ࢦఆͨ࣌ؒ͠ʹ౸ணͨ͠ཁૉʢ࠷େn݅ʣΛ·ͱΊͯ Seq Ͱग़ྗ 33 -
Source ͷ࡞Γํ (1) • Source.apply[T](iterable: Iterable[T]): Source[T, NotUsed] Source.fromIterator[T](f: ()
㱺 Iterator[T]): Source[T, NotUsed] • Iterable Iterator ͔Β Source Λ࡞Δ • Source.repeat[T](element: T): Source[T, NotUsed] • elementΛӬԕʹग़͢ 34 -
Source ͷ࡞Γํ (2) • Source.tick[T](initialDelay: FiniteDuration, interval: FiniteDuration, tick: T):
Source[T, Cancellable] • Ұఆ࣌ؒຖʹ tick Λग़͢ • Source.actorRef[T](bufferSize: Int, overflowStrategy: OverflowStrategy): Source[T, ActorRef] • materialized value ͷ ActorRef ʹϝοηʔδΛ͛Δͱ Source ͔ Βग़ྗ͞ΕΔ ੍ޚϙΠϯτͱͯ͠ materialized value ͕ΘΕΔྫ 35 -
Sink ͷ࡞Γํ • Sink.foreach[T](f: T 㱺 Unit): Sink[T, Future[Done]] •
TܕͷཁૉΛडऔΔͨͼʹfΛ࣮ߦ͢Δ • Sink.fold[U, T](zero: U)(f: (U, T) 㱺 U): Sink[T, Future[U]] • Sink.reduce[T](f: (T, T) 㱺 T): Sink[T, Future[T]] • ΈࠐΈ 37 -
Flow ͷ࡞Γํ • Flow.apply[T]: Flow[T,T,NotUsed] • ্ྲྀ͔Βड৴ͨ͠ཁૉΛͦͷ··Լྲྀૹ৴͢Δ • Flow[Int].map(_.toString) ͷΑ͏ʹมͷελʔτϙΠϯτ
ͱͯ͠͏ • Flow.fromSinkAndSource[I,O](sink: Graph[SinkShape[I],_], source: Graph[SourceShape[O],_): Flow[I,O,NotUsed] • Sink ͱ Source Λ·ͱΊͯ Flow ʹ͢Δ 38 -
Materialize ࣌ʹɺ1 actor Ͱ࣮ߦ͢ΔΑ͏ʹࣗಈతʹ༥߹(fuse) ͞ΕΔɻ༥߹ʹΦʔόϔου͕͋Δɻ͋Β͔͡Ί༥߹͢Δ͜ ͱͰ materialize ࣌ͷΦʔόʔϔουΛআڈͰ͖Δɻ val fusedStream
= akka.stream.Fusing.aggressive(myStream) application.conf Ͱࣗಈతͳ༥߹ΛແޮԽʢσόοά༻ʣ akka.stream.materializer.auto-fusing=off 43 -
ͱ͜ΖͰ .async ͷ࣮ override def async: Repr[Out] = addAttributes(Attributes.asyncBoundary) ଐੑ
(attribute) ͰετϦʔϜߏཁૉΛϚʔΫɻ ࣮ߦ࣌ʹ Materializer ͕ଐੑΛݟͯ༥߹͢Δ͔அɻ ଞʹͲΜͳଐੑ͕ʁ 44 -
import akka.event.Logging.WarningLevel import akka.stream.{ActorAttributes, Attributes} // ߏཁૉʹ໊લΛ͚Δ src.named("My source") //
ཁૉΛϩάग़ྗ͢ΔࡍͷϩάϨϕϧ src.log("foo").withAttributes(Attributes.logLevels(WarningLevel)) // ActorMaterializer Ͱ࣮ߦ͢Δͱ͖ͷ dispatcher Λࢦఆ src.addAttributes(ActorAttributes.dispatcher("my-dispatcher")) // Τϥʔ࣌ͷ supervision strategy src.addAttributes(ActorAttributes.supervisionStrategy(decider)) 45 -
ϦΞΫςΟϒγεςϜ(ϦΞΫςΟϒϚχϑΣετ1ΑΓ) ϝοηʔδۦಈ (Message Driven): ϦΞΫςΟϒγεςϜ ඇಉ ظͳϝοηʔδύογϯά ʹґͬͯίϯϙʔωϯτؒͷڥքΛ ཱ֬͢Δɻ… ·ͨɺγεςϜʹ
ϝοηʔδΩϡʔ Λ࡞ͯ͠ ࢹ͠ɺඞཁͳΒ όοΫϓϨογϟʔ Λద༻͢Δ͜ͱͰϑϩʔ ੍ޚ͕ՄೳʹͳΔɻ… 1 http://www.reactivemanifesto.org/ja 47 -
ඇಉظڥքΛ·͙ͨϝοηʔδͷड৴όοϑΝ akka.stream.materializer.max-input-buffer-size = 16 val materializer = ActorMaterializer( ActorMaterializerSettings(system) .withInputBuffer(
initialSize = 64, maxSize = 64)) ੑೳνϡʔχϯά༻ 50 -
໌ࣔతͳྲྀྔௐ Source/Flow ݻఆαΠζͷόοϑΝΛઃ͚Δɻ def buffer(size: Int, overflowStrategy: OverflowStrategy): Repr[Out] Ұఆͷʹ͑Δ
def throttle(elements: Int, per: FiniteDuration, maximumBurst: Int, mode: ThrottleMode): Repr[Out] 52 -
mapAsync(), mapAsyncUnordered() def send(email: Email): Future[X] = ??? val src:
Source[Email] = ??? // 4 ฒྻͰϝʔϧΛૹ৴ (Γ Source[X]) src.mapAsync(4)(email) // ޙ͔ΒདྷͨͷͰऴΘͬͨॱʹԼྲྀྲྀ͢ src.mapAsyncUnordered(4)(email) 54 -
Reactive streams Reactive stream ͷ PublisherɺSubscriber ͔Β Source, Sink Λ࡞Δ
Source.fromPublisher(myPublisher) Sink.fromSubscriber(mySubscriber) 55 -
ઐ༻ͷ stage • Reactive kafka • Akka stream contrib •
MQTT, AMQP ͳͲ ࣗͰ GraphStage Λ࣮͢Δ (akka blog) Writing Akka Streams Connectors for existing APIs 56 -
γϯϓϧͳذɺ߹ྲྀ (merge, alsoTo) val mergedSrc = src1 .merge(src2) .merge(src3) val
branchedSrc = src1 .alsoTo(sink1) .alsoTo(sink2) 59 -
ෳࡶͳذɺ߹ྲྀ Graph DSL RunnableGraph.fromGraph(GraphDSL.create() { implicit builder => import GraphDSL.Implicits._
val in = Source(1 to 10) val out = Sink.ignore val bcast = builder.add(Broadcast[Int](2)) val merge = builder.add(Merge[Int](2)) val f1, f2, f3, f4 = Flow[Int].map(_ + 10) in ~> f1 ~> bcast ~> f2 ~> merge ~> f3 ~> out bcast ~> f4 ~> merge ClosedShape }) 60 -
in ~> f1 ~> bcast ~> f2 ~> merge ~>
f3 ~> out bcast ~> f4 ~> merge ݟ͕ͨͦͬ͘Γʂײతʂ 61 -
GraphDSL Ͱ͙͑͢ Δذͱ߹ྲྀ • ߹ྲྀ (fan-in) • Merge, MergePreferred •
Zip, ZipWith • ذ (fan-out) • Broadcast • Balance • Unzip, UnzipWith 62 -
·ͱΊ • ετϦʔϜॲཧΛೃછΈͷ͋Δ API ͰߏͰ͖Δ • ϓϩάϥϜΛؒҧ͑ʹ͍͘Α͏ʹ͞Ε͍ͯΔ • ඇಉظ࣮ߦͱback pressure
ʹΑΔϑϩʔ੍ޚ͕σϑΥϧτ Ͱ͍ͭͯ͘Δ • ෳࡶͳذɺ߹ྲྀ graph DSL Ͱײతʹॻ͚Δ 63 -
News: GearpumpMaterializer Apache Gearpump ͷ࣍ϦϦʔε(0.8.2)͔Β Gearpump ্Ͱ stream Λ materialize
͢Δ GearpumpMaterializer ؚ͕·ΕΔɻ ࢄίϯϐϡʔςΟϯά + akka-stream ͷ໌͚ The Gearpump Materializer, Kam Kasravi 65