Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Scala: State of the Union

Scala: State of the Union

In this keynote, Kevin Webber, Enterprise Architect at Lightbend, will discuss the past, present, and future of Scala; where it has been and where it’s going. Yes, this talk will focus on the new language features introduced in 2.12, but also dive into an equally important aspect of Scala in 2016: the incredible community that’s grown up around Scala over the years. Kevin will frame the talk around his own journey to Scala and how both the language and the community made him a better developer and architect.

Kevin Webber

August 05, 2016
Tweet

More Decks by Kevin Webber

Other Decks in Technology

Transcript

  1. A bit about me » Enterprise Architect at Lightbend »

    Started exploring Scala in 2008 » Former Lightbend customer (2012-2014)
  2. Agenda » Where are we today? » Spoiler: 2.11.8 »

    Where are we going? » 2.12, 2.13, and beyond (aka, dotty)
  3. Agenda » Where are we today? » Spoiler: 2.11.8 »

    Where are we going? » 2.12, 2.13, and beyond (aka, dotty) » Where is the industry focused? » Scala is not an island » What have I learned over five years?
  4. Scala is active! » Significant contributions by » open-source community

    » partners (EPFL, Lightbend, Scala Center) » Releases average 2x per year
  5. Companies driving growth? » Apple » Coursera » IBM →

    Big Data University » Intel » Mesosphere
  6. Business drivers? » Data is ruling the world » Metal

    is abstracted » Functional programming rules this space » Object-oriented programming is a familiar path
  7. » Kafka (Confluent) » Distributed, publish-subscribe messaging system » Spark

    (Apache) » General-purpose cluster computing » Akka (Lightbend) » Distributed messaging toolkit
  8. Akka Streams » Higher-level abstraction on top of Akka »

    Per-event processing of streams » Back-pressure enabled by default
  9. Why Scala? Example "one-liner" to implement per-event stream processing with

    Akka Streams and Kafka implicit val actorSystem = ActorSystem("ReactiveKafka") implicit val materializer = ActorMaterializer() val kafka = new ReactiveKafka(host = "localhost:9092", zooKeeperHost = "localhost:2181") val publisher = kafka.consume("lowercaseStrings", "groupName", new StringDecoder()) val subscriber = kafka.publish("uppercaseStrings", "groupName", new StringEncoder()) // consume lowercase strings from kafka and publish them transformed to uppercase Source(publisher).map(_.toUpperCase).to(Sink(subscriber)).run()
  10. val g = RunnableGraph.fromGraph(GraphDSL.create() { implicit builder => import GraphDSL.Implicits._

    // Source val A: Outlet[String] = builder.add(Source.fromIterator(() => flightDelayLines)).out // Flows val B: FlowShape[String, FlightEvent] = builder.add(csvToFlightEvent) val C: FlowShape[FlightEvent, FlightDelayRecord] = builder.add(filterAndConvert) val D: UniformFanOutShape[FlightDelayRecord, FlightDelayRecord] = builder.add(Broadcast[FlightDelayRecord](2)) val F: FlowShape[FlightDelayRecord, (String, Int, Int)] = builder.add(averageCarrierDelay) // Sinks val E: Inlet[Any] = builder.add(Sink.ignore).in val G: Inlet[Any] = builder.add(Sink.foreach(averageSink)).in // Graph A ~> B ~> C ~> D E <~ D G <~ F <~ D ClosedShape }) g.run()
  11. Fast Data Architecture » Akka Streams » < 1 second

    latency per- event processing » Spark Streaming » 1-10+ second latency for mini-batch event processing » Kafka captures the complete event stream
  12. Big picture » An ecosystem is growing around Scala for:

    » Cloud computing » Stream processing » Fast data, big data, fast big data » Microservices » Scala is a natural fit in these spaces
  13. Notable libraries » Argonaut → purely functional JSON » Cats

    → foundation for pure, typeful libraries » ScalaCheck → property-based testing » ScalaTest → test Scala, Java, and Scala.js » scalaz → purely functional data structures » Shapeless → generic programming library » Specs → executable software specifications
  14. Educational opportunities » Carnegie-Mellon University » EPFL » Harvard University

    » Stanford University » University of Texas - Austin » University of Washington » West Point
  15. Scala Center » Open source foundation for Scala » Engage

    with developers in the open-source community to improve the language » Modularization (more to come in 2.13) » Smaller core to standardize & stabilize
  16. Scala Center » Open source foundation for Scala » Engage

    with developers in the open-source community to improve the language » Modularization (more to come in 2.13) » Smaller core to standardize & stabilize » Help developers learn the language
  17. Scala 2.1x » 2.12 ("compiler release") » Java 8 (λ,

    SAM, default) » New backend & optimizer » Q2 2016
  18. Scala 2.1x » 2.12 ("compiler release") » Java 8 (λ,

    SAM, default) » New backend & optimizer » Q2 2016 » 2.13 ("modularization") » Focused on libraries, collections, and Spark
  19. Scala.js 0.6.9 » Bringing Scala even closer to the JavaScript

    community » Current version is mostly about Scala 2.12 support and faster compilation » Closer and closer to production-grade
  20. Scala Native » Low-level primitives » Pointers, structs, etc »

    Interop with C » extern objects, no runtime overhead » Compiled ahead-of-time via LLVM » Instant startup vs JIT
  21. dotty » Currently a platform to try out new language

    concepts and compiler technologies
  22. dotty » Currently a platform to try out new language

    concepts and compiler technologies » Is the next Scala
  23. dotty » Currently a platform to try out new language

    concepts and compiler technologies » Is the next Scala » Focus is mainly on simplification
  24. dotty » Currently a platform to try out new language

    concepts and compiler technologies » Is the next Scala » Focus is mainly on simplification » Add new features & stay approachable by dropping other features » macros, general type projection, early initializers, existential types, etc
  25. 2. You don't have to know everything ... » That

    time I had a job interview with Martin
  26. ... but it can't hurt to try! » Buy this

    book and master: » Higher-order functions, currying, partial functions, functional data structures, handling errors without exceptions, strictness and laziness, purely functional state ...
  27. 3. Learn to explain the value functional, immutable, asynchronous, non-blocking,

    real-time, highly-available, loosely coupled, scalable, fault-tolerant, concurrent, low latency, high throughput... How do you map the properties of Scala and functional programming to business value?
  28. 4. Encourage, teach, mentor » We need developers of all

    experience levels, academic backgrounds, and professional backgrounds