Share
Embed
Transcript
169 slides
-
Running Scala on AWS Lambda in a snappy way ScalaMatsuri
2019 @todokr BizReach, Inc. "84-BNCEBͰ4DBMBΛαΫαΫಈ͔͢ and fun! -
The opinions expressed here do not represent those of any
organization. ͜͜Ͱࣔ͞Ε͍ͯΔݟղɺ ॴଐ৫ͷݟղΛө͢ΔͷͰ͋Γ·ͤΜ -
Who? Shunsuke Tadokoro @todokr ‣A software engineer working at BizReach,
Inc. ‣Love: Scala, Clojure, Emacs, etc. ‣Contributed: Play Framework, ScalikeJDBC, etc. ‣Wrote: ాॴॣ༎ גࣜձࣾϏζϦʔνͷιϑτΣΞΤϯδχΞ -
sbt-project-switcher ‣ A sbt plugin to switch sub projects quickly⚡
TCUͷαϒϓϩδΣΫτΛαΫαΫΓସ͑ΔϓϥάΠϯ -
AWS Lambda ‣a serverless computing service what we call 'FaaS'
‣Runtimes ‣ Python ‣ Node.js ‣ Java8 ‣ Go ‣ C# ‣ PowerShell ‣ Ruby ‣ custom runtime ‣Just write function and execute ͍ΘΏΔ'BB4ͱݺΕΔαʔόʔϨείϯϐϡʔςΟϯάαʔϏε ͨͩؔΛॻ͍࣮ͯߦ͢Δ͚ͩ -
AWS Lambda ‣ Fully managed ‣ No server management ‣
High Availability & Scalable ‣ So, you can focus on creating value ϑϧϚωʔδυɿ αʔόʔͷཧ͕ෆཁɺϏδωεՁΛੜΉ͜ͱʹूதͰ͖Δ -
AWS Lambda ‣ Fully managed ‣ No server management ‣
High Availability & Scalable ‣ So, you can focus on creating value ‣ Low cost ‣ Free tier ‣ Pay only for compute time ‣ No charge when code is not running ίετɿ ແྉ͋ΓɺίϯϐϡʔςΟϯά࣌ؒʹରͯ͠՝ۚ -
I hope to run Scala on AWS Lambda! ‣ No
context switch cost to my brain ‣ Scala in server app, Python in X, Node.js in Y, Go in Z… ‣ powers of concentration is precious resource ίϯςΩετεΠονͷ੍ͷͨΊ ͜Ε1ZUIPO ͋Ε/PEF ͋ͬͪ(Pͱ͍͏ͷͳΔ͘ආ͚͍ͨ -
I hope to run Scala on AWS Lambda! ‣ No
context switch cost to my brain ‣ Scala in server app, Python in X, Node.js in Y, Go in Z… ‣ powers of concentration is precious resource ‣ Types ‣ Lambda handler is rarely maintained ‣ Information richness comes from types save time to understand code -BNCEBϋϯυϥҰॻ͔ΕΔͱ์ஔ͞Ε͕ͪ ܕͷදݱྗίʔυͷཧղΛॿ͚ͯ͘ΕΔ -
I hope to run Scala on AWS Lambda! ‣ No
context switch cost to my brain ‣ Scala in server app, Python in X, Node.js in Y, Go in Z… ‣ powers of concentration is precious resource ‣ Types ‣ Lambda handler is rarely maintained ‣ Information richness comes from types save time to understand code ‣ Test ‣ Able to use testing library you get used to ׳ΕͨςετϥΠϒϥϦΛ͏͜ͱ͕Ͱ͖Δ -
However... ‣ It's often said that "Running JVM language
on Lambda is too slow!" ‣ So called "cold-start problem" ʮ-BNCEBͰ+7.ݴޠ͍ʯͱ͠͠ݴΘΕΔ -
However... ‣ It's often said that "Running JVM language
on Lambda is too slow!" ‣ So called "cold-start problem" ‣ There is limitation of function package size ‣ Jar size: up to 50 MB ‣ Need to care about libraries to depend on ؔύοέʔδͷαΠζ.#Ҏͱ͍͏੍ݶଘࡏ͢Δ -
So, questions are... ‣ How slow Scala on AWS Lambda
is? ‣ How can we make package size small? ‣ How can we run Scala on AWS Lambda in a snappy way? Ͳͷ͘Β͍͍ͷ͔ʁͲ͏ͨ͠ΒύοέʔδαΠζΛখ͘͞Ͱ͖Δ͔ʁ Ͳ͏ͨ͠Β-BNCEBͰ4DBMBΛαΫαΫಈ͔͢͜ͱ͕Ͱ͖Δ͔ʁ -
" Hello world (avg duration & memory usage of 5
times) ·ͣ)FMMPXPSME͔Β࣮ݧ -
Hello world in Scala import collection.JavaConverters._ import com.amazonaws.services.lambda.runtime.Context class Hello
{ def handleRequest( request: java.util.Map[String, String], context: Context ): java.util.Map[String, String] = { val name = request.asScala("name") Map("message" -> s"Hello, $name.").asJava } } 4DBMBͰͷ)FMMPXPSME+BSͷαΠζ.# ࠷େϝϞϦ༻ྔ.# ॴཁ࣌ؒ NT Lambda memory: 128MB Jar size: 9.7MB Max memory used: 115MB Duration: 8,588ms -
Hello world in Scala import collection.JavaConverters._ import com.amazonaws.services.lambda.runtime.Context class Hello
{ def handleRequest( request: java.util.Map[String, String], context: Context ): java.util.Map[String, String] = { val name = request.asScala("name") Map("message" -> s"Hello, $name.").asJava } } 4DBMBͰͷ)FMMPXPSME+BSͷαΠζ.# ࠷େϝϞϦ༻ྔ.# ॴཁ࣌ؒ NT Lambda memory: 128MB Jar size: 9.7MB Max memory used: 115MB Duration: 8,588ms -
Hello world in Scala import collection.JavaConverters._ import com.amazonaws.services.lambda.runtime.Context class Hello
{ def handleRequest( request: java.util.Map[String, String], context: Context ): java.util.Map[String, String] = { val name = request.asScala("name") Map("message" -> s"Hello, $name.").asJava } } 4DBMBͰͷ)FMMPXPSME+BSͷαΠζ.# ࠷େϝϞϦ༻ྔ.# ॴཁ࣌ؒ NT Lambda memory: 128MB Jar size: 9.7MB Max memory used: 115MB Duration: 8,588ms -
Hello world in Scala import collection.JavaConverters._ import com.amazonaws.services.lambda.runtime.Context class Hello
{ def handleRequest( request: java.util.Map[String, String], context: Context ): java.util.Map[String, String] = { val name = request.asScala("name") Map("message" -> s"Hello, $name.").asJava } } 4DBMBͰͷ)FMMPXPSME+BSͷαΠζ.# ࠷େϝϞϦ༻ྔ.# ॴཁ࣌ؒ NT Lambda memory: 128MB Jar size: 9.7MB Max memory used: 115MB Duration: 8,588ms -
2nd trial... TODO screen shot 2nd trial is blazing fast!
ճͷॴཁ࣌ؒNTͱ͍ -
2nd trial... TODO screen shot 2nd trial is blazing fast!
1st trial took 8 seconds, so called "cold-start" problem ॳճඵɺ͍ΘΏΔʮίʔϧυελʔτʯ -
What is cold-start? We should know Lambda lifecycle! ؔύοέʔδͷμϯϩʔυ Download
code 1. Download the function package (zip or jar, file on S3) you wrote -
What is cold-start? We should know Lambda lifecycle! ίʔυ࣮ߦ༻ͷαϯυϘοΫείϯςφͷىಈ Download
code Start new container 2. Start the sandbox container where your code will be executed -
What is cold-start? We should know Lambda lifecycle! ϥϯλΠϜͷىಈ +BWBͷ߹ɺ+7.ͷىಈΫϥεϑΝΠϧͷϩʔυͳͲ
Download code Start new container Bootstrap runtime 3. Bootstrap runtime to run your code, including JVM spin-up, load class files, etc. -
What is cold-start? We should know Lambda lifecycle! ؔύοέʔδͷ࣮ߦ Download
code Start new container Bootstrap runtime Run code 4. Run your function package code and affect the world -
What is cold-start? We should know Lambda lifecycle! ίϯςφΛϑϦʔζ͠ɺ࣍ճͷݺͼग़͠Λͭ ͨͩ͠ɺ࣍ճͷݺͼग़͕͠ಉ͡ΠϯελϯεͰ࣮ߦ͞ΕΔอূͳ͍
Download code Start new container Bootstrap runtime Run code Freeze the container 5. Freeze the container and wait for a next invocation However there's no guarantee it will run on the same instance. Container may be removed. -
Cold-start is a process that follows "download" to "run".
Note that downloading jar and spinning-up JVM relatively take time. What is cold-start? We should know Lambda lifecycle! ίʔϧυελʔτίʔυͷμϯϩʔυ͔Β࣮ߦ·ͰΛܦΔ KBSͷμϯϩʔυ+7.ͷىಈൺֱత͕͔͔࣌ؒΔॲཧ Download code Start new container Bootstrap runtime Run code Freeze the container -
Warm-start is a process which reuses the same instance, so
there's no spin-up time. This is why each invocations after the first is fast. What is cold-start? We should know Lambda lifecycle! ΥʔϜελʔτͱ-BNCBΠϯελϯε͕࠶ར༻͞ΕΔ࣮ߦ ͜Ε͕ճҎ߱ͷݺͼग़͕͔ͬͨ͠ཧ༝ Download code Start new container Bootstrap runtime Run code Freeze the container -
first-time-only misunderstanding ‣ Not the first time only! ‣ If
there is no available container, cold-start happens ‣ Lambda containers have the limitation that they can handle only single request at a time ‣ If two requests come in at once, two containers are required to handle these requests ‣ If there is no available one, it starts from container creation ෳಉ࣌ϦΫΤετͳͲɺ࣮ߦՄೳͳίϯςφ͕ଘࡏ͠ͳ͍߹ʹى ͖Δ -
first-time-only misunderstanding ‣ Not the first time only! ‣ If
there is no available container, cold-start happens ‣ Lambda containers have the limitation that they can handle only single request at a time ‣ If two requests come in at once, two containers are required to handle these requests ‣ If there is no available one, it starts from container creation ‣ When a large request spike comes, the throughput problem occurs or it exceed Lambda's concurrency limitation େ͖ͳϦΫΤετͷεύΠΫ͕དྷͨ߹ʹɺεϧʔϓοτ͕ʹͳͬͨΓ -BNCEBͷಉ࣮࣌ߦͷ੍ݶʹͻ͔͔ͬͬͯ͠·͏͜ͱ -
Factors increase cold-start latency ‣ Attached memory size ‣ Memory
footprint of JVM spin-up is relatively large ‣ CPU power is proportional to memory size -BNCEBʹׂΓͯΒΕͨϝϞϦαΠζ ଞͷϥϯλΠϜʹൺɺ+7.ىಈʹൺֱతଟ͘ͷϝϞϦΛඞཁͱ͢Δ -
Factors increase cold-start latency ‣ Attached memory size ‣ Memory
footprint of JVM spin-up is relatively large ‣ CPU power is proportional to memory size -BNCEBʹׂΓͯΒΕͨϝϞϦαΠζ ଞͷϥϯλΠϜʹൺɺ+7.ىಈʹൺֱతଟ͘ͷϝϞϦΛඞཁͱ͢Δ -
Factors increase cold-start latency ‣ Attached memory size ‣ Memory
footprint of JVM spin-up is relatively large ‣ CPU power is proportional to memory size ‣ Function package size ‣ The larger jar size is, the more download time takes ؔύοέʔδͷαΠζ େ͖͚Εେ͖͍΄Ͳɺμϯϩʔυʹ͕͔͔࣌ؒΔ -
Factors increase cold-start latency ‣ Attached memory size ‣ Memory
footprint of JVM spin-up is relatively large ‣ CPU power is proportional to memory size ‣ Function package size ‣ The larger jar size is, the more download time takes ؔύοέʔδͷαΠζ େ͖͚Εେ͖͍΄Ͳɺμϯϩʔυʹ͕͔͔࣌ؒΔ -
Factors increase cold-start latency ‣ Attached memory size ‣ Memory
footprint of JVM spin-up is relatively large ‣ CPU power is proportional to memory size ‣ Function package size ‣ The larger jar size is, the more download time takes ‣ Number of class files to be loaded ‣ Large number of classes to be loaded and initialized introduces large amount of latency ΫϥεϑΝΠϧͷ ϩʔυ͓ΑͼॳظԽ͞ΕΔΫϥεϑΝΠϧ͕ଟ͚Εɺͦͷ͘ͳΔ -
Factors increase cold-start latency ‣ Attached memory size ‣ Memory
footprint of JVM spin-up is relatively large ‣ CPU power is proportional to memory size ‣ Function package size ‣ The larger jar size is, the more download time takes ‣ Number of class files to be loaded ‣ Large number of classes to be loaded and initialized introduces large amount of latency ‣ VPC, etc. ͦͷଞɺ71$ͳͲӨڹ -
Another factor in Scala app ‣ Scala class files tend
to be numerous ‣ These class files are loaded and initialized when it used 4DBMBͷΫϥεϑΝΠϧ͕ଟ͘ͳΓ͕ͪ ར༻ͷࡍʹ͜ΕΒϩʔυɾॳظԽ͞ΕΔ $ tree . /unzipped-jar/scala scala/ ├── AnyVal.class ├── AnyValCompanion.class ├── App.class ├── Array$$anon$2.class ├── Array$.class ... │ ├── Regex$Replacement.class │ ├── Regex.class │ └── UnanchoredRegex.class └── volatile.class 34 directories, 2503 files $ tree ./ ... |____Hello$.class |____Hello.class |____Request.class |____Request$.class ... | |____PartialFunction$anonfun$apply$1.class | |____SttpBackendOptions$Proxy$$anon$1.class ... -
Scala on Lambda problems ‣ Java runtime is at a
disadvantage ‣ JVM spin-up memory footprint ‣ Jar size to be downloaded ‣ Number of classes to be loaded and initialized ىಈ࣌ͷϝϞϦϑοτϓϦϯτμϯϩʔυ͢ΔKBSͷαΠζɺ ΫϥεϑΝΠϧͷϩʔυͱॳظԽͳͲͷͰɺ+BWBϥϯλΠϜෆར -
Scala on Lambda problems ‣ Java runtime is at a
disadvantage ‣ JVM spin-up memory footprint ‣ Jar size to be downloaded ‣ Number of classes to be loaded and initialized ‣ Scala's another disadvantage ‣ Scala class files tend to be numerous 4DBMBͷΫϥεϑΝΠϧ͕ଟ͘ͳΓ͕ͪͳͷෆརͳ -
Scala on Lambda problems ‣ Java runtime is at a
disadvantage ‣ JVM spin-up memory footprint ‣ Jar size to be downloaded ‣ Number of classes to be loaded and initialized ‣ Scala's another disadvantage ‣ Scala class files tend to be numerous ‣ Thus, Scala cold-start in Java runtime takes longer than other language runtime ‣ even "Hello world", it took 8s ‣ when in warm-start, 0.5ms Αͬͯɺ4DBMBY+BWBϥϯλΠϜͷίʔϧυελʔτ ଞͷϥϯλΠϜΑΓ͕͔͔࣌ؒͬͯ͠·͏ -
hmm... so how can we run Scala snappy on Lambda?
͏ʔΉɺ͡Ό͋Ͳ͏ͨ͠ΒαΫαΫಈ͔ͤΔͩΖ͏͔ʁ -
Run Scala on Lambda without JVM ‣ However JVM is
great, the short-lived Lambda instance doesn't take advantage of the characteristics of that ‣ Such as dynamic class loading or HotSpot JIT compile, etc. +7.Ғେ͕ͩɺੜଘظ͕͍ؒ-BNCEBΠϯελϯεͰ ͦͷྑ͍ಛੑ͕׆͔ͮ͠Β͍ -
Run Scala on Lambda without JVM ‣ However JVM is
great, the short-lived Lambda instance doesn't take advantage of the characteristics of that ‣ Such as dynamic class loading or HotSpot JIT compile, etc. ‣ It might be awesome if it runs without JVM! ‣ Not to wait JVM spin-up and loading classes ‣ Make executable package size small ‣ Keep memory footprint small +7.ͳ͠Ͱ࣮ߦͰ͖ͨΒྑ͍݁ՌʹͳΔ͔͠Εͳ͍ʂ +7.ͷεϐϯΞοϓΛͨͣɺؔύοέʔδͱϝϞϦ༻ྔΛখ͘͞ -
How? ‣ Scala.js with Node.js runtime ‣ Scala Native with
Custom runtime 4DBMB/BUJWFΛΧελϜϥϯλΠϜͰಈ͔͢ -
How? ‣ Scala.js with Node.js runtime ‣ Scala Native with
Custom runtime ‣ Native image built by GraalVM with Custom runtime (SBBM7.Ͱੜͨ͠ωΠςΟϒΠϝʔδΛΧελϜϥϯλΠϜͰಈ͔͢ -
Scala.js ‣ A statically typed AltJS ‣ Generate JavaScript code
from Scala ‣ Provided as a sbt compiler plugin // plugins.sbt addSbtPlugin("org.scala-js" % "sbt-scalajs" % "0.6.26") // build.sbt enablePlugins(ScalaJSPlugin) // Greet.scala object Greet extends App { println("Hello from Scala.js!") } ੩తܕ͚"MU+4ͷҰछͰɺ4DBMBͷίʔυ͔Β+BWB4DSJQUΛੜ͢Δ TCUͷίϯύΠϥϓϥάΠϯͱͯ͠ఏڙ͞Ε͍ͯΔ -
class of ES6 & Scala.js class Person(val firstName: String, val
lastName: String) { def fullName(): String = s"$firstName $lastName" } class Person { constructor(firstName, lastName) { this.firstName = firstName; this.lastName = lastName; } fullName() { return `${this.firstName} ${this.lastName}`; } } ES6 Scala.js -
Scala.js facade for Node.js ‣ scalajs-io/nodejs https://github.com/scalajs-io/nodejs ‣ A facade
library provides Node.js API interface ‣ Enable type hinting and checking // before (not type safe) import js.Dynamic def getToken(): UndefOr[String] = Dynamic.global.process.env.ACCESS_TOKEN.asInstanceOf[UndefOr[String]] // after import io.scalajs.nodejs.{Environment, process} def getToken2(): Environment = process.env TDBMBKTJPOPEFKTͱ͍͏/PEFKTͷϑΝαʔυϥΠϒϥϦ͕͋Δ /PEFKTͷ"1*Λ҆શʹݺͼग़͢͜ͱ͕Ͱ͖Δ -
Scala Native ‣ Generate executable binary file from Scala ‣
Provided as a sbt compiler plugin // plugins.sbt addSbtPlugin("org.scala-native" % "sbt-scala-native" % "0.3.8") // build.sbt scalaVersion := "2.11.12" enablePlugins(ScalaNativePlugin) // Greet.scala object Greet extends App { println("Hello from Scala Native!") } 4DBMBͷίʔυ͔Β࣮ߦՄೳͳόΠφϦϑΝΠϧΛੜ͢Δ TCUͷίϯύΠϥϓϥάΠϯͱͯ͠ఏڙ͞Ε͍ͯΔ -
Scala Native ‣ Try generating executable and running without JVM
# compile & generate machine code $ sbt nativeLink # execute $ time ./target/scala-2.11/greet-out Hello from Scala Native! real 0m0.019s user 0m0.004s sys 0m0.007s όΠφϦΛੜ࣮ͯ͠ߦͯ͠ΈΔ -
AWS Lambda custom runtime ‣ Any language that can run
in the Lambda execution environment is available ‣ Simple HTTP based interface ‣ to get the event payload for a new invocation ‣ to return back the response from the function -BNCEBͷڥͰ࣮ߦͰ͖ΔݴޠͳΒԿͰར༻Մೳ ϦΫΤετϨεϙϯεͷͨΊͷγϯϓϧͳ)551ΠϯλϑΣʔεΛ࣋ͭ -
GraalVM ‣ Polyglot VM developed by Oracle ‣ Enable to
run JVM langs, JS, Python, Ruby, etc. with zero overhead ‣ SubstrateVM ‣ A GraalVM component that allows AOT compilation of Java code into executable native images ‣ Community Edition / Enterprise Edition 0SBDMF͕։ൃ͍ͯ͠Δ1PMZHMPU7. 4VCTUSBUF7.࣮ߦՄೳͳωΠςΟϒΠϝʔδΛੜ͢Δίϯϙʔωϯτ -
Java frameworks use SubstrateVM ‣ Quarkus https://quarkus.io ‣ A "Supersonic
Subatomic Java" framework ‣ Micronaut Framework https://micronaut.io ‣ A full-stack framework for microservice and serverless applications ‣ Project Helidon https://helidon.io ‣ A collection of Java libraries for writing microservices (SBBM7.ͰωΠςΟϒΠϝʔδΛੜ͢Δ+BWBϑϨʔϜϫʔΫ -
Hello world in Scala.js // build.sbt lazy val commonSettings =
Seq( scalaVersion := "2.12.8", scalacOptions += "-P:scalajs:sjsDefinedByDefault", scalaJSLinkerConfig ~= { _.withModuleKind(ModuleKind.CommonJSModule) }) lazy val hello = (project in file("hello")) .settings( commonSettings, artifactPath in (Compile, fastOptJS) := baseDirectory.value / "hello.js", artifactPath in (Compile, fullOptJS) := baseDirectory.value / "hello-opt.js", ).enablePlugins(ScalaJSPlugin) ‣ build.sbt -
Hello world in Scala.js $PNNPO+4ελΠϧͰϞδϡʔϧΛੜ͢Δ // build.sbt lazy val commonSettings
= Seq( scalaVersion := "2.12.8", scalacOptions += "-P:scalajs:sjsDefinedByDefault", scalaJSLinkerConfig ~= { _.withModuleKind(ModuleKind.CommonJSModule) }) lazy val hello = (project in file("hello")) .settings( commonSettings, artifactPath in (Compile, fastOptJS) := baseDirectory.value / "hello.js", artifactPath in (Compile, fullOptJS) := baseDirectory.value / "hello-opt.js", ).enablePlugins(ScalaJSPlugin) ‣ Emit CommonJS style modules -
Hello world in Scala.js ੜ͞Εͨ+4͕ஔ͞ΕΔύεΛఆٛ // build.sbt lazy val commonSettings
= Seq( scalaVersion := "2.12.8", scalacOptions += "-P:scalajs:sjsDefinedByDefault", scalaJSLinkerConfig ~= { _.withModuleKind(ModuleKind.CommonJSModule) }) lazy val hello = (project in file("hello")) .settings( commonSettings, artifactPath in (Compile, fastOptJS) := baseDirectory.value / "hello.js", artifactPath in (Compile, fullOptJS) := baseDirectory.value / "hello-opt.js", ).enablePlugins(ScalaJSPlugin) ‣ Define the path which is generated JS placed -
Hello world in Scala.js // Hello.scala import scala.scalajs.js import scala.scalajs.js.annotation.JSExportTopLevel
@js.native trait Request extends js.Object { val name: String = js.native } object Hello { type Callback = js.Function2[Error, js.Any, Unit] @JSExportTopLevel("handleRequest") def handleRequest(req: Request, c: js.Object, callback: Callback): Unit = { val message = s"Hello, ${req.name}!" callback(null, js.Dynamic.literal("message" -> message)) } } ‣ Hello.scala -
Hello world in Scala.js +4ͷΦϒδΣΫτΛܕ҆શʹѻ͏ͨΊͷϑΝαʔυܕ // Hello.scala import scala.scalajs.js import
scala.scalajs.js.annotation.JSExportTopLevel @js.native trait Request extends js.Object { val name: String = js.native } object Hello { type Callback = js.Function2[Error, js.Any, Unit] @JSExportTopLevel("handleRequest") def handleRequest(req: Request, c: js.Object, callback: Callback): Unit = { val message = s"Hello, ${req.name}!" callback(null, js.Dynamic.literal("message" -> message)) } } ‣ A facade type to handle JS object type safely -
Hello world in Scala.js ίʔϧόοΫؔͷܕΛఆٛ // Hello.scala import scala.scalajs.js import
scala.scalajs.js.annotation.JSExportTopLevel @js.native trait Request extends js.Object { val name: String = js.native } object Hello { type Callback = js.Function2[Error, js.Any, Unit] @JSExportTopLevel("handleRequest") def handleRequest(req: Request, c: js.Object, callback: Callback): Unit = { val message = s"Hello, ${req.name}!" callback(null, js.Dynamic.literal("message" -> message)) } } ‣ Define the type of callback function -
Hello world in Scala.js ͕͜͜ϋϯυϥͷओཁͳύʔτ // Hello.scala import scala.scalajs.js import
scala.scalajs.js.annotation.JSExportTopLevel @js.native trait Request extends js.Object { val name: String = js.native } object Hello { type Callback = js.Function2[Error, js.Any, Unit] @JSExportTopLevel("handleRequest") def handleRequest(req: Request, c: js.Object, callback: Callback): Unit = { val message = s"Hello, ${req.name}!" callback(null, js.Dynamic.literal("message" -> message)) } } ‣ This is the main part of handler -
Hello world in Scala.js ΦϒδΣΫτΛHMPCBMʹΤΫεϙʔτ͢ΔΞϊςʔγϣϯ // Hello.scala import scala.scalajs.js import
scala.scalajs.js.annotation.JSExportTopLevel @js.native trait Request extends js.Object { val name: String = js.native } object Hello { type Callback = js.Function2[Error, js.Any, Unit] @JSExportTopLevel("handleRequest") def handleRequest(req: Request, c: js.Object, callback: Callback): Unit = { val message = s"Hello, ${req.name}!" callback(null, js.Dynamic.literal("message" -> message)) } } ‣ A annotation exports that object to the JS global scope -
Hello world in Scala.js ϦΫΤετ͔ΒϝοηʔδΛΈཱͯɺίʔϧόοΫؔΛݺͿ // Hello.scala import scala.scalajs.js import
scala.scalajs.js.annotation.JSExportTopLevel @js.native trait Request extends js.Object { val name: String = js.native } object Hello { type Callback = js.Function2[Error, js.Any, Unit] @JSExportTopLevel("handleRequest") def handleRequest(req: Request, c: js.Object, callback: Callback): Unit = { val message = s"Hello, ${req.name}!" callback(null, js.Dynamic.literal("message" -> message)) } } ‣ Create message from request and invoke callback function -
Hello world in Scala.js ‣ Let's run! 1BDLBHFTJ[F .BYNFNPSZVTBHF $PMETUBSU
%VSBUJPO NT 4DBMB+7. .# .# NT 4DBMBKT ,# .# NT -
Hello world in Scala.js ‣ Let's run! 1BDLBHFTJ[F .BYNFNPSZVTBHF $PMETUBSU
%VSBUJPO NT 4DBMB+7. .# .# NT 4DBMBKT ,# .# NT -
Hello world in Scala.js ‣ Let's run! 1BDLBHFTJ[F .BYNFNPSZVTBHF $PMETUBSU
%VSBUJPO NT 4DBMB+7. .# .# NT 4DBMBKT ,# .# NT -
Hello world in Scala.js ‣ Let's run! 1BDLBHFTJ[F .BYNFNPSZVTBHF $PMETUBSU
%VSBUJPO NT 4DBMB+7. .# .# NT 4DBMBKT ,# .# NT -
Hello world in Scala.js ‣ Let's run! 1BDLBHFTJ[F .BYNFNPSZVTBHF $PMETUBSU
%VSBUJPO NT 4DBMB+7. .# .# NT 4DBMBKT ,# .# NT So fast! ΊͬͪΌ͍ʂ -
Hello world in Scala Native Building executable for Lambda is
complex a bit -BNCEB͚࣮ߦՄೳϑΝΠϧΛ4DBMB/BUJWFͰ࡞Δͷগʑෳࡶ -
Hello world in Scala Native Building executable for Lambda is
complex a bit ‣ Write Scala Native code 4DBMB/BUJWFͷίʔυΛॻ͘ -
Hello world in Scala Native Building executable for Lambda is
complex a bit ‣ Write Scala Native code ‣ Build the Amazon Linux based docker container ‣ Install required libs such as LLVM, curl, openssl, re2, etc. ‣ Publish those as a Lambda Layer "NB[PO-JOVYϕʔεͷ%PDLFSίϯςφΛϏϧυ͢Δɻ ඞཁͳϥΠϒϥϦΛΠϯετʔϧ͠ɺͦΕΒΛ-BNCEB-BZFSͱͯ͠ஔ -
Hello world in Scala Native Building executable for Lambda is
complex a bit ‣ Write Scala Native code ‣ Build the Amazon Linux based docker container ‣ Install required libs such as LLVM, curl, openssl, re2, etc. ‣ Publish those as a Lambda Layer ‣ Build executable with the docker container and deploy %PDLFSΛ࣮ͬͯߦՄೳϑΝΠϧΛϏϧυ͠ɺσϓϩΠ -
Hello world in Scala Native ‣ build.sbt // build.sbt lazy
val commonSettings = Seq( scalaVersion := "2.11.12", nativeLinkingOptions ++= Seq("-static-libstdc++", "-L/lib/"), nativeGC := "immix", // default is Boehm GC nativeCompileOptions := Seq("-O3") ) lazy val hello = (project in file("hello")) .settings( commonSettings, libraryDependencies ++= Seq( "com.softwaremill.sttp" %%% "core" % "1.5.11", "io.argonaut" % "argonaut_native0.3_2.11" % "6.2.2" ) ).enablePlugins(ScalaNativePlugin) -
Hello world in Scala Native ‣ Settings for Scala version,
linkage option, compile option, etc. // build.sbt lazy val commonSettings = Seq( scalaVersion := "2.11.12", nativeLinkingOptions ++= Seq("-static-libstdc++", "-L/lib/"), nativeGC := "immix", // default is Boehm GC nativeCompileOptions := Seq("-O3") ) lazy val hello = (project in file("hello")) .settings( commonSettings, libraryDependencies ++= Seq( "com.softwaremill.sttp" %%% "core" % "1.5.11", "io.argonaut" % "argonaut_native0.3_2.11" % "6.2.2" ) ).enablePlugins(ScalaNativePlugin) 4DMBMBόʔδϣϯϦϯέʔδɺίϯύΠϧͷΦϓγϣϯΛઃఆ -
Hello world in Scala Native ‣ Libraries written for Scala
Native are available // build.sbt lazy val commonSettings = Seq( scalaVersion := "2.11.12", nativeLinkingOptions ++= Seq("-static-libstdc++", "-L/lib/"), nativeGC := "immix", // default is Boehm GC nativeCompileOptions := Seq("-O3") ) lazy val hello = (project in file("hello")) .settings( commonSettings, libraryDependencies ++= Seq( "com.softwaremill.sttp" %%% "core" % "1.5.11", "io.argonaut" % "argonaut_native0.3_2.11" % "6.2.2" ) ).enablePlugins(ScalaNativePlugin) 4DBMB/BUJWFʹରԠ͍ͯ͠ΔϥΠϒϥϦͳΒར༻͕Մೳ -
Hello world in Scala Native ‣ STTP: HTTP lib, to
invoke Custom runtime API ‣ Argonaut: JSON lib, to handle JSON request / response // build.sbt lazy val commonSettings = Seq( scalaVersion := "2.11.12", nativeLinkingOptions ++= Seq("-static-libstdc++", "-L/lib/"), nativeGC := "immix", // default is Boehm GC nativeCompileOptions := Seq("-O3") ) lazy val hello = (project in file("hello")) .settings( commonSettings, libraryDependencies ++= Seq( "com.softwaremill.sttp" %%% "core" % "1.5.11", "io.argonaut" % "argonaut_native0.3_2.11" % "6.2.2" ) ).enablePlugins(ScalaNativePlugin) 4551)551ϥΠϒϥϦ ΧελϜϥϯλΠϜ"1*Λݺͼग़͢༻ "SHPOBVU+40/ϥΠϒϥϦ ϦΫΤετϨεϙϯεͷ+40/༻ -
Hello world in Scala Native ‣ Hello.scala import com.softwaremill.sttp._ import
argonaut._, Argonaut._ object Hello { def main(args: Array[String]): Unit = { implicit val backend = CurlBackend() implicit val requestCodec = casecodec1(Request.apply, Request.unapply)("name") val runtimeApi = System.getenv("AWS_LAMBDA_RUNTIME_API") while (true) { val next = sttp.get(uri"http://$runtimeApi/2018-06-01/runtime/invocation/next").send() val requestId = next.headers("Lambda-Runtime-Aws-Request-Id") val request = Parse.decodeOption[Request](next.unsafeBody).get val message = Json("message" -> jString(s"Hello, ${request.name}!")).nospaces sttp.post( uri"http://$runtimeApi/2018-06-01/runtime/invocation/$requestId/response" ).body(message).send() } } } case class Request(name: String) -
Hello world in Scala Native ‣ Initialize: get runtime api
path, etc. import com.softwaremill.sttp._ import argonaut._, Argonaut._ object Hello { def main(args: Array[String]): Unit = { implicit val backend = CurlBackend() implicit val requestCodec = casecodec1(Request.apply, Request.unapply)("name") val runtimeApi = System.getenv("AWS_LAMBDA_RUNTIME_API") while (true) { val next = sttp.get(uri"http://$runtimeApi/2018-06-01/runtime/invocation/next").send() val requestId = next.headers("Lambda-Runtime-Aws-Request-Id") val request = Parse.decodeOption[Request](next.unsafeBody).get val message = Json("message" -> jString(s"Hello, ${request.name}!")).nospaces sttp.post( uri"http://$runtimeApi/2018-06-01/runtime/invocation/$requestId/response" ).body(message).send() } } } case class Request(name: String) ॳظԽॲཧϥϯλΠϜ"1*ͷύεऔಘͳͲ -
Hello world in Scala Native ‣ Process 1 request with
1 loop import com.softwaremill.sttp._ import argonaut._, Argonaut._ object Hello { def main(args: Array[String]): Unit = { implicit val backend = CurlBackend() implicit val requestCodec = casecodec1(Request.apply, Request.unapply)("name") val runtimeApi = System.getenv("AWS_LAMBDA_RUNTIME_API") while (true) { val next = sttp.get(uri"http://$runtimeApi/2018-06-01/runtime/invocation/next").send() val requestId = next.headers("Lambda-Runtime-Aws-Request-Id") val request = Parse.decodeOption[Request](next.unsafeBody).get val message = Json("message" -> jString(s"Hello, ${request.name}!")).nospaces sttp.post( uri"http://$runtimeApi/2018-06-01/runtime/invocation/$requestId/response" ).body(message).send() } } } case class Request(name: String) ϧʔϓͰϦΫΤετΛॲཧ͢Δ -
Hello world in Scala Native ‣ Fetch event from next
API and parse as request import com.softwaremill.sttp._ import argonaut._, Argonaut._ object Hello { def main(args: Array[String]): Unit = { implicit val backend = CurlBackend() implicit val requestCodec = casecodec1(Request.apply, Request.unapply)("name") val runtimeApi = System.getenv("AWS_LAMBDA_RUNTIME_API") while (true) { val next = sttp.get(uri"http://$runtimeApi/2018-06-01/runtime/invocation/next").send() val requestId = next.headers("Lambda-Runtime-Aws-Request-Id") val request = Parse.decodeOption[Request](next.unsafeBody).get val message = Json("message" -> jString(s"Hello, ${request.name}!")).nospaces sttp.post( uri"http://$runtimeApi/2018-06-01/runtime/invocation/$requestId/response" ).body(message).send() } } } case class Request(name: String) OFYU"1*Λୟ͖ɺ݁ՌΛ3FRVFTUΦϒδΣΫτͱͯ͠ύʔε -
Hello world in Scala Native ‣ Send build message to
response API import com.softwaremill.sttp._ import argonaut._, Argonaut._ object Hello { def main(args: Array[String]): Unit = { implicit val backend = CurlBackend() implicit val requestCodec = casecodec1(Request.apply, Request.unapply)("name") val runtimeApi = System.getenv("AWS_LAMBDA_RUNTIME_API") while (true) { val next = sttp.get(uri"http://$runtimeApi/2018-06-01/runtime/invocation/next").send() val requestId = next.headers("Lambda-Runtime-Aws-Request-Id") val request = Parse.decodeOption[Request](next.unsafeBody).get val message = Json("message" -> jString(s"Hello, ${request.name}!")).nospaces sttp.post( uri"http://$runtimeApi/2018-06-01/runtime/invocation/$requestId/response" ).body(message).send() } } } case class Request(name: String) ΈཱͯͨϝοηʔδΛϨεϙϯε"1*ʹૹΔ -
Hello world in Scala Native Build code with docker FROM
amazonlinux:latest ENV SCALA_VERSION="2.11.12" WORKDIR /build/ RUN yum install -y java-1.8.0 && \ yum install -y tar.x86_64 && \ yum install -y gzip gunzip RUN curl -O http://downloads.typesafe.com/scala/${SCALA_VERSION}/scala-${SCALA_VERSION}.tgz RUN tar -xzvf scala-${SCALA_VERSION}.tgz && \ rm -rf scala-${SCALA_VERSION}.tgz && \ echo "export SCALA_HOME=/home/ec2-user/scala-${SCALA_VERSION}" >> ~/.bashrc && \ echo "export PATH=$PATH:/home/ec2-user/scala-${SCALA_VERSION}/bin:/opt/llvm-3.9.0/bin" >> ~/.bashrc && \ source ~/.bashrc #SBT RUN curl https://bintray.com/sbt/rpm/rpm | tee /etc/yum.repos.d/bintray-sbt-rpm.repo RUN yum install -y sbt WORKDIR /build/runtime/ RUN yum install -y -q yum-utils RUN yum-config-manager --enable epel > /dev/null RUN yum -y update RUN yum -y group install "development tools" RUN yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm RUN echo $'[alonid-llvm-3.9.0] \n\ name=Copr repo for llvm-3.9.0 owned by alonid \n\ baseurl=https://copr-be.cloud.fedoraproject.org/results/alonid/llvm-3.9.0/epel-7-$basearch/ \n\ type=rpm-md \n\ skip_if_unavailable=True \n\ gpgcheck=1 \n\ gpgkey=https://copr-be.cloud.fedoraproject.org/results/alonid/llvm-3.9.0/pubkey.gpg \n\ repo_gpgcheck=0 \n\ enabled=1 \n\ enabled_metadata=1' >> /etc/yum.repos.d/epel.repo RUN yum install -y clang-3.9.0 RUN yum install -y llvm-3.9.0 llvm-3.9.0-devel RUN yum install -y zip which libunwind libunwind-devel python-pip jq libcurl-devel RUN mkdir -p /build/runtime/lib/ && cp /usr/lib64/libunwind.so /build/runtime/lib/libunwind.so.8 && cp /usr/lib64/libunwind-x86_64.so.8 /build/runtime/lib/libunwind-x86_64.so.8 RUN yum install -y libidn libidn-devel && cp /usr/lib64/libidn.so.11 lib/libidn.so.11 RUN yum install -y git make openssl-devel RUN curl -O -L https://github.com/curl/curl/releases/download/curl-7_62_0/curl-7.62.0.tar.gz && tar -zxvf curl-7.62.0.tar.gz RUN curl -O -L https://github.com/openssl/openssl/archive/OpenSSL_1_0_2l.tar.gz && tar -zxvf OpenSSL_1_0_2l.tar.gz RUN git clone https://github.com/google/re2.git WORKDIR /build/runtime/re2/ ENV LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH ENV CXX=/opt/llvm-3.9.0/bin/clang++ LDFLAGS="-static-libstdc++" RUN make && make install RUN cp /usr/local/lib/libre2.so.0 /build/runtime/lib/libre2.so.0 WORKDIR /build/runtime/openssl-OpenSSL_1_0_2l RUN ./config --prefix=/opt/lib/ssl --openssldir=/opt/lib/ssl shared zlib RUN make && make install WORKDIR /build/runtime/curl-7.62.0/ ENV LD_LIBRARY_PATH=/opt/lib:/usr/lib64:$LD_LIBRARY_PATH RUN mkdir -p /build/runtime/lib/ssl && cp -r /opt/lib/ssl/lib /build/runtime/lib/ssl/lib RUN ./configure --prefix=$(/build/runtime) --with-ssl=/opt/lib/ssl && make && make install # libs RUN cp /lib/libcurl.so.4 /build/runtime/lib/libcurl.so.4 && cp /opt/lib/ssl/lib/libcrypto.so.1.0.0 /build/runtime/lib/libcrypto.so.1.0.0 && cp /opt/lib/ssl/lib/libssl.so.1.0.0 /build/runtime/lib/libssl.so.1.0.0 WORKDIR /build/runtime/ RUN rm OpenSSL_1_0_2l.tar.gz curl-7.62.0.tar.gz RUN mkdir -p /build/main WORKDIR /build/main ENV LD_LIBRARY_PATH=/opt/lib/ssl/lib:/usr/local/lib:$LD_LIBRARY_PATH ADD build.sbt /build/main/ ADD project/ /build/main/ CMD tail -f /dev/null -
Hello world in Scala Native Build code with docker ڊେ
FROM amazonlinux:latest ENV SCALA_VERSION="2.11.12" WORKDIR /build/ RUN yum install -y java-1.8.0 && \ yum install -y tar.x86_64 && \ yum install -y gzip gunzip RUN curl -O http://downloads.typesafe.com/scala/${SCALA_VERSION}/scala-${SCALA_VERSION}.tgz RUN tar -xzvf scala-${SCALA_VERSION}.tgz && \ rm -rf scala-${SCALA_VERSION}.tgz && \ echo "export SCALA_HOME=/home/ec2-user/scala-${SCALA_VERSION}" >> ~/.bashrc && \ echo "export PATH=$PATH:/home/ec2-user/scala-${SCALA_VERSION}/bin:/opt/llvm-3.9.0/bin" >> ~/.bashrc && \ source ~/.bashrc #SBT RUN curl https://bintray.com/sbt/rpm/rpm | tee /etc/yum.repos.d/bintray-sbt-rpm.repo RUN yum install -y sbt WORKDIR /build/runtime/ RUN yum install -y -q yum-utils RUN yum-config-manager --enable epel > /dev/null RUN yum -y update RUN yum -y group install "development tools" RUN yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm RUN echo $'[alonid-llvm-3.9.0] \n\ name=Copr repo for llvm-3.9.0 owned by alonid \n\ baseurl=https://copr-be.cloud.fedoraproject.org/results/alonid/llvm-3.9.0/epel-7-$basearch/ \n\ type=rpm-md \n\ skip_if_unavailable=True \n\ gpgcheck=1 \n\ gpgkey=https://copr-be.cloud.fedoraproject.org/results/alonid/llvm-3.9.0/pubkey.gpg \n\ repo_gpgcheck=0 \n\ enabled=1 \n\ enabled_metadata=1' >> /etc/yum.repos.d/epel.repo RUN yum install -y clang-3.9.0 RUN yum install -y llvm-3.9.0 llvm-3.9.0-devel RUN yum install -y zip which libunwind libunwind-devel python-pip jq libcurl-devel RUN mkdir -p /build/runtime/lib/ && cp /usr/lib64/libunwind.so /build/runtime/lib/libunwind.so.8 && cp /usr/lib64/libunwind-x86_64.so.8 /build/runtime/lib/libunwind-x86_64.so.8 RUN yum install -y libidn libidn-devel && cp /usr/lib64/libidn.so.11 lib/libidn.so.11 RUN yum install -y git make openssl-devel RUN curl -O -L https://github.com/curl/curl/releases/download/curl-7_62_0/curl-7.62.0.tar.gz && tar -zxvf curl-7.62.0.tar.gz RUN curl -O -L https://github.com/openssl/openssl/archive/OpenSSL_1_0_2l.tar.gz && tar -zxvf OpenSSL_1_0_2l.tar.gz RUN git clone https://github.com/google/re2.git WORKDIR /build/runtime/re2/ ENV LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH ENV CXX=/opt/llvm-3.9.0/bin/clang++ LDFLAGS="-static-libstdc++" RUN make && make install RUN cp /usr/local/lib/libre2.so.0 /build/runtime/lib/libre2.so.0 WORKDIR /build/runtime/openssl-OpenSSL_1_0_2l RUN ./config --prefix=/opt/lib/ssl --openssldir=/opt/lib/ssl shared zlib RUN make && make install WORKDIR /build/runtime/curl-7.62.0/ ENV LD_LIBRARY_PATH=/opt/lib:/usr/lib64:$LD_LIBRARY_PATH RUN mkdir -p /build/runtime/lib/ssl && cp -r /opt/lib/ssl/lib /build/runtime/lib/ssl/lib RUN ./configure --prefix=$(/build/runtime) --with-ssl=/opt/lib/ssl && make && make install # libs RUN cp /lib/libcurl.so.4 /build/runtime/lib/libcurl.so.4 && cp /opt/lib/ssl/lib/libcrypto.so.1.0.0 /build/runtime/lib/libcrypto.so.1.0.0 && cp /opt/lib/ssl/lib/libssl.so.1.0.0 /build/runtime/lib/libssl.so.1.0.0 WORKDIR /build/runtime/ RUN rm OpenSSL_1_0_2l.tar.gz curl-7.62.0.tar.gz RUN mkdir -p /build/main WORKDIR /build/main ENV LD_LIBRARY_PATH=/opt/lib/ssl/lib:/usr/local/lib:$LD_LIBRARY_PATH ADD build.sbt /build/main/ ADD project/ /build/main/ CMD tail -f /dev/null Oh, huge scala-on-lambda-snappy/scalanative/Dockerfile -
Hello world in Scala Native ‣ Build docker container &
build executable %PDLFSίϯςφͷϏϧυͱɺ࣮ߦՄೳϑΝΠϧͷϏϧυ # bootstrap $ cd scala-on-lambda-snappy/scalanative $ docker build -t native-builder-image . $ docker run --name native-builder -dt native-builder-image $ docker cp ./ native-builder:/build/main/ $ docker exec -it native-builder sbt 'project hello' nativeLink $ docker cp native-builder:/build/main/hello/target/scala-2.11/hello-out \ hello/bootstrap $ zip -Dj hello.zip ./hello/bootstrap # Deploy this zip # runtime layer $ docker cp native-builder:/build/runtime/ ./runtime/ $ cd ./runtime $ zip -r runtime.zip . # Publish this zip as Lambda Layer -
Hello world in Scala Native ‣ Let's run! 1BDLBHFTJ[F .BYNFNPSZVTBHF
$PMETUBSU %VSBUJPO NT 4DBMB+7. .# .# NT 4DBMBKT ,# .# NT 4DBMB/BUJWF .# .# NT -
Hello world in Scala Native ‣ Let's run! 1BDLBHFTJ[F .BYNFNPSZVTBHF
$PMETUBSU %VSBUJPO NT 4DBMB+7. .# .# NT 4DBMBKT ,# .# NT 4DBMB/BUJWF .# .# NT -
Hello world in Scala Native ‣ Let's run! 1BDLBHFTJ[F .BYNFNPSZVTBHF
$PMETUBSU %VSBUJPO NT 4DBMB+7. .# .# NT 4DBMBKT ,# .# NT 4DBMB/BUJWF .# .# NT -
Hello world in Scala Native ‣ Let's run! 1BDLBHFTJ[F .BYNFNPSZVTBHF
$PMETUBSU %VSBUJPO NT 4DBMB+7. .# .# NT 4DBMBKT ,# .# NT 4DBMB/BUJWF .# .# NT -
Hello world in Scala Native ‣ Let's run! 1BDLBHFTJ[F .BYNFNPSZVTBHF
$PMETUBSU %VSBUJPO NT 4DBMB+7. .# .# NT 4DBMBKT ,# .# NT 4DBMB/BUJWF .# .# NT Yay! ͬͨͶʂ -
Hello world in GraalVM native image Build is easier than
Scala Native a bit 4DBMB/BUJWFͷϏϧυΑΓগ͠؆୯ -
Hello world in GraalVM native image Build is easier than
Scala Native a bit ‣ Write Scala code 4DBMBͷίʔυΛॻ͘ -
Hello world in GraalVM native image Build is easier than
Scala Native a bit ‣ Write Scala code ‣ Build fat-jar with sbt-assembly TCUBTTFNCMZͰGBUKBSΛϏϧυ͢Δ -
Hello world in GraalVM native image Build is easier than
Scala Native a bit ‣ Write Scala code ‣ Build fat-jar with sbt-assembly ‣ Build the GraalVM docker container ‣ Community Edition, in this sample (SBBM7.ͷ%PDLFSίϯςφΛϏϧυ͢Δ -
Hello world in GraalVM native image Build is easier than
Scala Native a bit ‣ Write Scala code ‣ Build fat-jar with sbt-assembly ‣ Build the GraalVM docker container ‣ Community Edition, in this sample ‣ Build executable from jar with the docker and deploy %PDLFSΛͬͯKBS͔Β࣮ߦՄೳϑΝΠϧΛϏϧυ͠ɺσϓϩΠ͢Δ -
Hello world in GraalVM native image ‣ build.sbt // build.sbt
lazy val commonSettings = Seq( scalaVersion := "2.12.8" ) lazy val hello = (project in file("hello")) .settings( commonSettings, libraryDependencies ++= Seq( "com.softwaremill.sttp" %% "core" % "1.5.11", "io.argonaut" %% "argonaut" % "6.2.2" ) ) -
Hello world in GraalVM native image ‣ Almost the same
as Scala Native's Hello world 4DBMB/BUJWFͷ)FMMPXPSMEͱ΄΅ಉ͡ import argonaut._; import Argonaut._; import com.softwaremill.sttp._ object Hello { implicit val backend = HttpURLConnectionBackend() implicit val requestCodec = casecodec1(Request.apply, Request.unapply)("name") def main(args: Array[String]): Unit = { val runtimeApi = System.getenv("AWS_LAMBDA_RUNTIME_API") while (true) { val next = sttp.get(uri"http://$runtimeApi/2018-06-01/runtime/invocation/next").send() val requestId = next.headers("lambda-runtime-aws-request-id").head val request = next.unsafeBody.decodeOption[Request].get val responseJson = Json("message" -> jString(s"Hello, ${request.name}!")).nospaces sttp.post(uri"http://$runtimeApi/2018-06-01/runtime/invocation/$requestId/response") .body(responseJson).send() } } } case class Request(name: String) -
Hello world in GraalVM native image ‣ First, build fat-jar
‣ Then, build native image from jar with GraalVM ·ͣGBUKBSΛϏϧυ ࣍ʹ(SBBM7.ͰKBS͔ΒOBUJWFJNBHFΛϏϧυ $ cd scala-on-lambda-snappy/graalvm # build fat-jar $ sbt 'project hello' assembly # build native image $ docker build -t graal-builder-image . $ docker run --name graal-builder -dt graal-builder-image $ docker cp ./hello/target/scala-2.12/hello-assembly-0.1-SNAPSHOT.jar \ graal-builder:/hello.jar $ docker exec -it graal-builder native-image -jar hello.jar \ -H:EnableURLProtocols=http,https --no-server $ docker cp graal-builder:/hello ./hello/bootstrap $ zip -Dj hello.zip ./hello/bootstrap -
Hello world in GraalVM native image ‣ Let's run! 1BDLBHFTJ[F
.BYNFNPSZVTBHF $PMETUBSU %VSBUJPO NT 4DBMB+7. .# .# NT 4DBMBKT ,# .# NT 4DBMB/BUJWF .# .# NT (SBBM7. .# .# NT -
Hello world in GraalVM native image ‣ Let's run! 1BDLBHFTJ[F
.BYNFNPSZVTBHF $PMETUBSU %VSBUJPO NT 4DBMB+7. .# .# NT 4DBMBKT ,# .# NT 4DBMB/BUJWF .# .# NT (SBBM7. .# .# NT -
Hello world in GraalVM native image ‣ Let's run! 1BDLBHFTJ[F
.BYNFNPSZVTBHF $PMETUBSU %VSBUJPO NT 4DBMB+7. .# .# NT 4DBMBKT ,# .# NT 4DBMB/BUJWF .# .# NT (SBBM7. .# .# NT -
Hello world in GraalVM native image ‣ Let's run! 1BDLBHFTJ[F
.BYNFNPSZVTBHF $PMETUBSU %VSBUJPO NT 4DBMB+7. .# .# NT 4DBMBKT ,# .# NT 4DBMB/BUJWF .# .# NT (SBBM7. .# .# NT -
Hello world in GraalVM native image ‣ Let's run! 1BDLBHFTJ[F
.BYNFNPSZVTBHF $PMETUBSU %VSBUJPO NT 4DBMB+7. .# .# NT 4DBMBKT ,# .# NT 4DBMB/BUJWF .# .# NT (SBBM7. .# .# NT Good! ͍͍Ͷʂ -
AWS integration with Scala.js ‣ Use AWS SDK for JavaScript
‣ There is some tips to use the SDK type safely ‣ In this example, use DynamoDB Document Client "844%,GPS+BWB4DSJQUΛ͏͕ɺܕ҆શʹѻ͏ʹগʑίπ͕͍Δ ͜ͷྫͰ%ZOBNP%#ͷ%PDVNFOU$MJFOUΛ͏ -
Querying DynamoDB with Scala.js ‣ AWS.scala // AWS.scala @JSImport("aws-sdk", JSImport.Namespace)
@js.native object AWS extends js.Object { @js.native object DynamoDB extends js.Object { @js.native class DocumentClient extends js.Object { def query( param: js.Dictionary[String], callback:js.Function2[js.Object, js.Object, js.Any] ): Unit = js.native } } } -
Querying DynamoDB with Scala.js ‣ Import AWS SDK for JavaScript
// AWS.scala @JSImport("aws-sdk", JSImport.Namespace) @js.native object AWS extends js.Object { @js.native object DynamoDB extends js.Object { @js.native class DocumentClient extends js.Object { def query( param: js.Dictionary[String], callback:js.Function2[js.Object, js.Object, js.Any] ): Unit = js.native } } } "844%,GPS+BWB4DSJQUͷΠϯϙʔτ -
Querying DynamoDB with Scala.js ‣ Facade for DocumentClient to handle
type safely // AWS.scala @JSImport("aws-sdk", JSImport.Namespace) @js.native object AWS extends js.Object { @js.native object DynamoDB extends js.Object { @js.native class DocumentClient extends js.Object { def query( param: js.Dictionary[String], callback:js.Function2[js.Object, js.Object, js.Any] ): Unit = js.native } } } %PDVNFOU$MJFOUΛܕ҆શʹѻ͏ͨΊͷϑΝαʔυ -
Querying DynamoDB with Scala.js ‣ DynamoAccessor.scala object DynamoAccessor { val
documentClient = new AWS.DynamoDB.DocumentClient() @JSExportTopLevel("handleRequest") def handleRequest(req: Request, cx: js.Object, callback: Callback): Unit = { val param = js.Dynamic.literal( "TableName" -> "PriceHistory", "KeyConditionExpression" -> "#ji = :jobid", "ExpressionAttributeNames" -> js.Dynamic.literal("#ji" -> "JobId"), "ExpressionAttributeValues" -> js.Dynamic.literal(":jobid" -> req.jobId) ) documentClient.query( param, (err: js.Object, data: js.Object) => callback(null, data)) } } -
Querying DynamoDB with Scala.js ‣ Initialize DocumentClient object DynamoAccessor {
val documentClient = new AWS.DynamoDB.DocumentClient() @JSExportTopLevel("handleRequest") def handleRequest(req: Request, cx: js.Object, callback: Callback): Unit = { val param = js.Dynamic.literal( "TableName" -> "PriceHistory", "KeyConditionExpression" -> "#ji = :jobid", "ExpressionAttributeNames" -> js.Dynamic.literal("#ji" -> "JobId"), "ExpressionAttributeValues" -> js.Dynamic.literal(":jobid" -> req.jobId) ) documentClient.query( param, (err: js.Object, data: js.Object) => callback(null, data)) } } %PDVNFOU$MJFOUͷΠχγϟϥΠζ -
Querying DynamoDB with Scala.js ‣ Prepare param (using Dynamic, so
not type safe...) object DynamoAccessor { val documentClient = new AWS.DynamoDB.DocumentClient() @JSExportTopLevel("handleRequest") def handleRequest(req: Request, cx: js.Object, callback: Callback): Unit = { val param = js.Dynamic.literal( "TableName" -> "PriceHistory", "KeyConditionExpression" -> "#ji = :jobid", "ExpressionAttributeNames" -> js.Dynamic.literal("#ji" -> "JobId"), "ExpressionAttributeValues" -> js.Dynamic.literal(":jobid" -> req.jobId) ) documentClient.query( param, (err: js.Object, data: js.Object) => callback(null, data)) } } ύϥϝʔλͷ༻ҙ %ZOBNJDΛ͍ͬͯΔͷͰܕ҆શͰͳ͍ -
Querying DynamoDB with Scala.js ‣ To make it type safe,
define type like the Request trait object DynamoAccessor { val documentClient = new AWS.DynamoDB.DocumentClient() @JSExportTopLevel("handleRequest") def handleRequest(req: Request, cx: js.Object, callback: Callback): Unit = { val param = js.Dynamic.literal( "TableName" -> "PriceHistory", "KeyConditionExpression" -> "#ji = :jobid", "ExpressionAttributeNames" -> js.Dynamic.literal("#ji" -> "JobId"), "ExpressionAttributeValues" -> js.Dynamic.literal(":jobid" -> req.jobId) ) documentClient.query( param, (err: js.Object, data: js.Object) => callback(null, data)) } } ܕ҆શʹ͢ΔͳΒɺ3FRVFTUτϨΠτͷΑ͏ʹܕΛఆٛ͢Δ -
Querying DynamoDB with Scala.js ‣ Submit a query with the
client object DynamoAccessor { val documentClient = new AWS.DynamoDB.DocumentClient() @JSExportTopLevel("handleRequest") def handleRequest(req: Request, cx: js.Object, callback: Callback): Unit = { val param = js.Dynamic.literal( "TableName" -> "PriceHistory", "KeyConditionExpression" -> "#ji = :jobid", "ExpressionAttributeNames" -> js.Dynamic.literal("#ji" -> "JobId"), "ExpressionAttributeValues" -> js.Dynamic.literal(":jobid" -> req.jobId) ) documentClient.query( param, (err: js.Object, data: js.Object) => callback(null, data)) } } ΫϥΠΞϯτΛͬͯΫΤϦΛ͛Δ -
Facade library for aws-sdk ‣ exoego/aws-sdk-scalajs-facade https://github.com/exoego/aws-sdk-scalajs-facade ‣ Scala.js
type facade for aws-sdk-js ‣ A fork of balshor/aws-sdk-scalajs-facade ‣ It's also good for learning how to write facade type by yourself FYPFHPBXTTELTDBMBKTGBDBEFͱ͍͏ϑΝαʔυϥΠϒϥϦ͕͋Δ ϑΝαʔυͷॻ͖ํΛֶͿͷʹΑͦ͞͏ -
AWS with Scala Native? ‣ Currently, there's no Scala Native
library of AWS SDK ‣ So need to implement a client which invokes AWS REST API ‣ cf. rusoto: A third-party AWS SDK for Rust https://rusoto.org 4DBMB/BUJWF൛ͷ"844%,GPS+BWBݱঢ়ͳ͍ ͠"84αʔϏεΛ͍͍ͨ߹ɺΫϥΠΞϯτΛ࣮͢Δඞཁ͕͋Δ -
AWS with GraalVM? ‣ GraalVM is able to build native
images from Java, however... (SBBM7.+BWB͔ΒOBUJWFJNBHFΛϏϧυͰ͖Δ͕ -
AWS with GraalVM? ‣ GraalVM is able to build native
images from Java, however... ‣ There is some limitations of SubstrateVM 4VCTUSBUF7.ͷ͍͔ͭ͘ͷ੍ݶ͕͋Δ -
AWS with GraalVM? ‣ GraalVM is able to build native
images from Java, however... ‣ There is some limitations of SubstrateVM ‣ AOT compilation doesn't support Dynamic class loading https://github.com/oracle/graal/blob/master/substratevm/LIMITATIONS.md ྫ͑ɺDMBTTͷಈతͳMPBEJOHVOMPBEJOHαϙʔτ͞Ε͍ͯͳ͍ -
AWS with GraalVM? ‣ GraalVM is able to build native
images from Java, however... ‣ There is some limitations of SubstrateVM ‣ AOT compilation doesn't support Dynamic class loading https://github.com/oracle/graal/blob/master/substratevm/LIMITATIONS.md ‣ AWS SDK for Java uses commons-logging which uses ClassLoader... "844%,GPS+BWBͷDPNNPOTMPHHJOH͕$MBTT-PBEFSΛ͍ͬͯΔ -
AWS with GraalVM? ‣ GraalVM is able to build native
images from Java, however... ‣ There is some limitations of SubstrateVM ‣ AOT compilation doesn't support Dynamic class loading https://github.com/oracle/graal/blob/master/substratevm/LIMITATIONS.md ‣ AWS SDK for Java uses commons-logging which uses ClassLoader... "844%,GPS+BWBͷDPNNPOTMPHHJOH͕$MBTT-PBEFSΛ͍ͬͯΔ -
Kuromoji ‣ atilika / kuromoji https://github.com/atilika/kuromoji ‣ A Japanese morphological
analyzer written in Java ‣ Its name is derived from "ࠇจࣈ" ‣ A traditional small wooden knife to cut Japanese sweets in tea ceremony ,VSPNPKJ+BWBͷຊޠܗଶૉղੳϥΠϒϥϦ ໊લ՛ࢠΛΓ͚Δʮࠇจࣈʯʹ༝དྷ͢ΔʢΒ͍͠ʣ -
Kuromoji ‣ atilika / kuromoji https://github.com/atilika/kuromoji ‣ A Japanese morphological
analyzer written in Java ‣ Its name is derived from "ࠇจࣈ" ‣ A traditional small wooden knife to cut Japanese sweets in tea ceremony ‣ Let's create a morphological analysis API with this! ؆୯ͳܗଶૉղੳ"1*Λ࡞ͬͯΈΑ͏ʂ -
Kuromoji with GraalVM ‣ Kuromoji.scala object Kuromoji { implicit val
backend = HttpURLConnectionBackend() def main(args: Array[String]): Unit = { val runtime = System.getenv("AWS_LAMBDA_RUNTIME_API") val tokenizer = new Tokenizer() while (true) { val req = sttp.get(uri"http://$runtime/2018-06-01/runtime/invocation/next").send() val requestId = req.headers("lambda-runtime-aws-request-id").head val text = Parse.parseWith(req.unsafeBody, _.field("text").flatMap(_.string).get, identity) val results = tokenizer.tokenize(text).asScala.map { token => jString(s"${token.getSurface}: ${token.getAllFeatures}") } sttp.post(uri"http://$runtime/2018-06-01/runtime/invocation/$requestId/response") .body(Json.array(results :_*).nospaces).send() } } } -
Kuromoji with GraalVM ‣ Parse text with Kuromoji & tokenize
object Kuromoji { implicit val backend = HttpURLConnectionBackend() def main(args: Array[String]): Unit = { val runtime = System.getenv("AWS_LAMBDA_RUNTIME_API") val tokenizer = new Tokenizer() while (true) { val req = sttp.get(uri"http://$runtime/2018-06-01/runtime/invocation/next").send() val requestId = req.headers("lambda-runtime-aws-request-id").head val text = Parse.parseWith(req.unsafeBody, _.field("text").flatMap(_.string).get, identity) val results = tokenizer.tokenize(text).asScala.map { token => jString(s"${token.getSurface}: ${token.getAllFeatures}") } sttp.post(uri"http://$runtime/2018-06-01/runtime/invocation/$requestId/response") .body(Json.array(results :_*).nospaces).send() } } } ,VSPNPKJΛͬͯςΩετΛύʔε͠ɺτʔΫϯԽ͢Δ -
Other examples in repo ‣ Takeuchi function ‣ As known
as "Tarai-mawashi Function" ‣ High load function; when takeuchi(2n, n, 0), O(n^n) ‣ Slack notification of S3 put with Scala Native ͦͷଞͷαϯϓϧ https://github.com/todokr/scala-on-lambda-snappy -
Scala.js ‣ Pros ‣ Very small code size (hello.zip: 4KB)
‣ Integration with AWS services ‣ Cons ‣ Need to write some boilerplaty facade for type safe ‣ Need to know Node.js API, ecosystem, manners, etc... ‣ I use this for ‣ DynamoDB accessor ॴίʔυαΠζͷখ͞͞ɺ"84αʔϏεͱͷ࿈ܞ ॴϘΠϥʔϓϨʔτతϑΝαʔυ /PEFKTͷ͕ࣝඞཁ -
Scala Native ‣ Pros ‣ Relatively small package size (hello.zip:
1.3M) ‣ Cons ‣ Tricky to build for Lambda ‣ Need to check if the library is supported for Scala Native ॴൺֱతখ͍͞ύοέʔδαΠζ ॴ-BNCEB͚ͷϏϧυʹίπ͕͍Δɺ͍͍ͨϥΠϒϥϦ͕ 4DBMB/BUJWFʹରԠ͍ͯ͠Δ͔֬ೝ͢Δඞཁ͕͋Δ -
Native image built by GraalVM ‣ Pros ‣ Easy to
use Scala/Java libraries ‣ Cons ‣ Relatively large package size (hello.zip: 7.3MB) ‣ Tricky to build a bit ‣ Some limitation of AOT compilation ॴ4DBMB+BWBͷϥΠϒϥϦΛ؆୯ʹ͑Δ ॴύοέʔδ͕ͦͦ͜͜େ͖͍ Ϗϧυʹগʑίπ͕͍Δ "05ίϯύΠϧͷ੍ݶ͕͋Δ -
My case: EC Discount Notifier cron Cloudwatch Events Crawler Price
history Amazon etc. Notifier ࢲͷར༻έʔε&$ͷԼ͛ใΛ௨ -
My case: EC Discount Notifier cron Cloudwatch Events Crawler Price
history Amazon etc. Notifier Ϋϩʔϥʔ(SBBM7.OBUJWFJNBHF +TPVQ +BWBϥΠϒϥϦ Λ͑Δ͜ͱɺจࣈίʔυͷѻ͍Ͱ༗ར GraalVM native-image - Using Jsoup (Java library) - Charset -
My case: EC Discount Notifier cron Cloudwatch Events Crawler Price
history Amazon etc. Notifier /PUJpFS4DBMB+4Y/PEFKTϥϯλΠϜ 4%,͕͑ΔͷͰɺ%ZOBNP%#ͷΞΫηε͕༰қ GraalVM native-image - Using Jsoup (Java library) - Charset ScalaJS x Node.js -Accessing DynamoDB -
Java improvement for the container age ‣ Java continue to
evolve! +BWBਐԽ͠ଓ͚͍ͯΔ -
Java improvement for the container age ‣ Java continue to
evolve! ‣ jlink (Java9) ‣ A command-line tool to create customized small JRE খ͞ͳ+3&Λ࡞ΔͨΊͷKMJOL -
Java improvement for the container age ‣ Java continue to
evolve! ‣ jlink (Java9) ‣ A command-line tool to create customized small JRE ‣ Class-Data Sharing (JEP 310, 341) ‣ storing the preprocessed class metadata to file ‣ loading classes from CDS file improves startup performance ىಈΛߴԽ͢Δ$MBTT%BUB4IBSJOH -
‣ Run Scala on Lambda without JVM is not so
simple 4DBMBΛ+7.ͳ͠Ͱಈ͔͢ͷͦΜͳʹγϯϓϧͰͳ͍ -
‣ Run Scala on Lambda without JVM is not so
simple ‣ However, it's powerful on some special occasion ͔͠͠ɺಛผͳঢ়گԼͰͱͯύϫϑϧ -
‣ Run Scala on Lambda without JVM is not so
simple ‣ However, it's powerful on some special occasion ‣ So, we should consider the trade-off ͳͷͰɺࢲͨͪτϨʔυΦϑΛߟ͑Δඞཁ͕͋Δ -
‣ Run Scala on Lambda without JVM is not so
simple ‣ However, it's powerful on some special occasion ‣ So, we should consider the trade-off ‣ Need to integrate with other AWS service => Scala.js with Node.js runtime "84αʔϏεͱͷ࿈ܞ͕ඞཁ/PEFKTϥϯλΠϜͰ4DBMBKT -
‣ Run Scala on Lambda without JVM is not so
simple ‣ However, it's powerful on some special occasion ‣ So, we should consider the trade-off ‣ Need to integrate with other AWS service => Scala.js with Node.js runtime ‣ Need to use Java library & run snappy => Native Image with GraalVM +BWBͷϥΠϒϥϦΛ͍ɺαΫαΫಈ͔͢ඞཁ͕͋Δ (SBBM7.ͰOBUJWFJNBHF -
‣ Run Scala on Lambda without JVM is not so
simple ‣ However, it's powerful on some special occasion ‣ So, we should consider the trade-off ‣ Need to integrate with other AWS service => Scala.js with Node.js runtime ‣ Need to use Java library & run snappy => Native Image with GraalVM ‣ Invoked constantly and throughput is not so important => Just use Java8 runtime ఆظతͳݺͼग़͕͋͠Γɺεϧʔϓοτ͕ॏཁͰͳ͍ ૉʹ+BWBϥϯλΠϜΛ͓͏ -
‣ Run Scala on Lambda without JVM is not so
simple ‣ However, it's powerful on some special occasion ‣ So, we should consider the trade-off ‣ Need to integrate with other AWS service => Scala.js with Node.js runtime ‣ Need to use Java library & run snappy => Native Image with GraalVM ‣ Invoked constantly and throughput is not so important => Just use Java8 runtime ‣ _ => Give up and use other runtime such as Python or Node.js ͦΕҎ֎ͳΒɺఘΊͯ1ZUIPO/PEFKTͳͲଞͷϥϯλΠϜΛ͓͏ -
Thank you! Contact(via) match { case GitHub => "@todokr" case
Twitter => "@todokr" case Facebook => "shusnuke.tadokoro" case _ => Dynamic.talk } ͋Γ͕ͱ͏͍͟͝·ͨ͠ʂ