Upgrade to Pro — share decks privately, control downloads, hide ads and more …

What's ahead for Elixir: v1.2 and GenRouter

What's ahead for Elixir: v1.2 and GenRouter

With Elixir v1.1 just released, José Valim explains what is ahead for Elixir 1.2 and discuss what we may see in Elixir v1.3 and later.

Plataformatec

October 10, 2015
Tweet

More Decks by Plataformatec

Other Decks in Programming

Transcript

  1. Elixir v1.1 • September/2015 • >295 contributors • ~ 1000

    packages on hex.pm • > 6.5million downloads
  2. Application.fetch_env!/2 Keyword.get_and_update/3 Keyword.get_lazy/3 Keyword.pop_lazy/3 Keyword.put_new_lazy/3 Map.get_and_update/3 Map.get_lazy/3 Map.pop_lazy/3 Map.put_new_lazy/3 Enum.dedup/1

    Enum.dedup_by/2 Enum.min_max/1 Enum.min_max_by/2 Enum.random/1 Enum.reduce_while/3 Enum.reverse_slice/3 Enum.take_random/2 More functions File.lstat/1 File.lstat!/1 File.rename/2 Integer.digits/2 Integer.undigits/2 GenServer.whereis/1 Process.hibernate/3 Stream.dedup/1 Stream.dedup_by/2 Stream.transform/4 String.jaro_distance/2 String.splitter/3 StringIO.flush/1 Task.yield/2 Task.shutdown/2 Tuple.append/2 URI.to_string/1
  3. Enum iex> Enum.dedup([1, 1, 2, 2, 1, 1]) [1, 2,

    1] iex> Enum.random 1..100 51 iex> Enum.take_random 1..100, 3 [14, 3, 72]
  4. iex> String.jaro_distance "helo", "help" 0.8333333333334 String $ mix helo **

    (Mix) The task "helo" could not be found. Did you mean "help"?
  5. # v1.0+ iex> task = Task.async(fn -> ... end) %Task{}

    iex> Task.await(task, 5_000) # v1.1+ iex> Task.yield(task, 5_000) {:ok, result} | nil iex> Task.shutdown(task, :brutal_kill) {:ok, result} | nil Task
  6. Capture Log # v1.0+ import ExUnit.CaptureIO capture_io fn -> IO.puts

    "hello" end # v1.1+ import ExUnit.CaptureLog capture_log fn -> Logger.info "hello" end
  7. # Per test @tag :capture_log test "something with logging" do

    ... end # Per suite ExUnit.configure capture_log: true Capture Log
  8. Capture Log 1) test something with logging test/plug/error.exs:59 ** (RuntimeError)

    oops stacktrace: test/plug/error.exs:59 The following output was logged: 20:27:52.443 [info] log message
  9. Not implemented tests # @tag :not_implemented test "eventually will do

    X" $ mix test --exclude not_implemented $ mix test --only not_implemented
  10. • stacktrace depth is now configurable and has been increased

    to 20 • More detailed errors • Ability to skip tests • Proper line number in doctests failures More improvements
  11. mix profile.fprof # CNT ACC (ms) OWN (ms) Total 200279

    1972.188 1964.579 Test.run/0 1 1972.166 0.007 Test.do_something/1 3 1972.131 0.040 Test.bottleneck/0 1 1599.490 0.007 ... $ mix profile.fprof my_script.exs
  12. Build Embedded (false) /_build ├── dev │ └── lib │

    ├── cowboy │ │ └── ebin -> ../../../../deps/cowboy/ebin │ ├── cowlib │ │ ├── ebin -> ../../../../deps/cowlib/ebin │ ├── phoenix │ │ ├── ebin │ │ │ ├── Elixir.Access.Phoenix.Socket.beam
  13. Build embedded (true) • Packages your whole application into _build

    (no symlinks) • Enables protocol consolidation • No longer automatically compiles
  14. • "mix compile" tracks only compile-time dependencies for Elixir •

    "recompile()" inside IEx • Use the safer https protocol instead of git for :github dependencies More improvements
  15. Elixir v1.1 # v1.1 iex> "Hello \u2661" "Hello ♡" #

    v1.0 iex> "Hello \x{2661}” "Hello ♡" SOFT DEPRECATED
  16. Elixir v1.1 # v1.0 defmodule Calculator do use Behaviour defcallback

    add(number, number) :: number end SOFT DEPRECATED # v1.1 defmodule Calculator do @callback add(number, number) :: number end
  17. Elixir v1.1 defmodule Calculator do @typedoc "A complex number" @opaque

    complex :: ... @doc "Adds two numbers" @callback add(number, number) :: ... end Code.get_docs(Calculator, :type_docs)
  18. • opts[key] still works • Access defines a subset of

    the Dict behaviour • The bottleneck does not exist in prod • Common hot-paths have been inlined Access
  19. Migration • Elixir v1.0, v1.1
 Support Erlang 17 & 18+


    • Elixir v1.2
 Supports Erlang 18+
  20. Dicts in Elixir v1.1 • Keyword, Map, HashDict • Unified

    via the Dict API • Too many options. Often confusing.
  21. Dicts in Elixir v1.2 • Keyword lists - used as

    options, allow duplicate keys, user ordered • Maps - pattern matching, fast & scalable, term ordered
  22. Dict & Set • HashDict will be soft deprecated
 Use

    Map • HashSet will be soft deprecated
 Use MapSet • Dict and Set will be soft deprecated
  23. Collections sink (collectable) widgets |> Enum.filter(fn b -> b.color ==

    :red end) |> Enum.map(fn b -> {b.title, b.height} end) |> Enum.into(%{}) source (enumerable)
  24. Collections CSV.parse(path) |> Enum.filter(fn b -> b.color == :red end)

    |> Enum.map(fn b -> {b.title, b.height} end) |> Enum.into(IO.stream(:stdio, :inspect))
  25. Collections + Laziness CSV.parse(path) |> Stream.filter(fn b -> b.color ==

    :red end) |> Stream.map(fn b -> {b.title, b.height} end) |> Stream.into(IO.stream(:stdio, :inspect)) |> Stream.run()
  26. Pipeline parallelism CSV.parse(path) |> Stream.async() |> Stream.filter(fn b -> b.color

    == :red end) |> Stream.map(fn b -> {b.title, b.height} end) |> Stream.into(IO.stream(:stdio, :inspect)) |> Stream.run()
  27. Pipeline parallelism CSV.parse(path) |> ... |> Stream.async() |> ... |>

    Stream.async() |> ... |> Stream.async() |> ... |> Stream.run()
  28. Pipeline parallelism • Each async stage is a process •

    What happens if one of them crash? • How to provide back-pressure?
  29. Pipeline parallelism • How to make streams supervised processes? •

    Pipeline parallelism isn’t even a good strategy
  30. Some Use Cases • A GenEvent replacement that is process-based

    • A way to load-balance jobs across a pool of processes
  31. Demand-driven • It is a message contract • It pushes

    back-pressure to the boundary • GenRouter is one impl of this contract • Inspired by Akka Streams
  32. Supervised TCP Acceptor # Source {:ok, acceptor} = GenRouter.TCPAcceptor.start_link(...) #

    Sink {:ok, supervisor} = GenRouter.Supervisor.start_link(...) # Source <-> Sink GenRouter.subscribe(supervisor, to: acceptor)
  33. The path forward • Define the demand-driven message contract •

    Implement GenRouter and related abstractions • Integrate with streams