Lessons Learned From an Elixir OTP Project

You started an Elixir project, coming from another language, and you are curious about how everything fits, what are the most common problems encountered, and how is the best way to solve them.

Questions such as: How to organize your tests in a concurrent application? What are the pitfalls when writing process tests? How can we organize our code? How can contexts help us? Which smells can help us realize it's time to refactor our code? And how do we use the language tooling in our favor?

Transcript

1. Lessons Learned From an Elixir/OTP Project @amandasposito
2. •amandasposito.com •speakerdeck.com/amandasposito •linkedin.com/in/amandasposito
7. How was it like to start in a new language?
8. Normally, Elixir is not the first language we learn
9. Most people come from Object-Oriented languages
10. Where do I start?
11. How do I organize things?
12. Is this code really complex or is it me that doesn't know FP?
14. A lot of new stuff
15. A new project, a new language, a new paradigm
16. Even though anxiety may hit harder
18. You started your new project
19. How to use the language tooling in our favor?
20. Elixir has a bunch of cool stuff to help us in our journey
21. Chances are you will deal with databases in your project
22. One of the things that may be different from other languages
24. In Object-Oriented languages we have ORMs
25. Implicit can be dangerous sometimes
26. Ecto is explicit
27. This is a big change in the way we think
28. You have to take most of the decisions dealing with databases
29. Instead of accessing the data using Objects
30. We now use Repo
31. We write things more like SQL
32. For example: Preloads
33. Will I need to access this association?
34. Repo.all from c in Course, preload: [:users]
35. SELECT * FROM Courses; SELECT * FROM Users WHERE course_id = ? SELECT * FROM Users WHERE course_id = ? SELECT * FROM Users WHERE course_id = ? SELECT * FROM Users WHERE course_id = ? SELECT * FROM Users WHERE course_id = ? SELECT * FROM Users WHERE course_id = ? SELECT * FROM Users WHERE course_id = ? SELECT * FROM Users WHERE course_id = ?
36. SELECT * FROM Courses; SELECT * FROM Users WHERE course_id = ?
37. Spend time learning all the cool features Ecto has
38. •Repo •Changesets •Schemas •Associations •Ecto.Multi •etc.
39. Sometimes depending on how many access to the database you have
40. Or how much data you are dealing with
41. The database may be a bottleneck
42. In some cases what we can do is to cache some data to help us
43. ETS Tables
44. –Erlang Documentation “These provide the ability to store very large quantities of data in an Erlang runtime system, and to have constant access time to the data.”
45. A common use case for ETS tables is to store cache
46. However, there are some things to take into consideration
47. When we talk about cache in memory Redis usually comes to mind
48. This ETS kinda looks like Redis, and Redis I know how to use, so why not use this one for free?
50. ETS Tables are like a Hash straight into memory
51. It does not have many optimization options
52. They consume the memory available for the application
53. It doesn't offer support to distribution
54. With these constraints, is ETS what you need?
55. Take into account the number of items you will handle
56. There was this one time where we ended up creating an ETS with 76GB of memory consumption
58. https://moz.com/devblog/moz-analytics-db-free
59. https://moz.com/devblog/moz-analytics-db-free
61. Doctest
62. defmodule MyModuleTest do use ExUnit.Case, async: true doctest MyModule end
63. @doc """ Sums two numbers iex> MyModule.sum(2, 2) 4 """ def sum(a, b) do a + b end
64. What about tests?
65. Doctest != Test
66. So your application is talking with the outside world, what now?
68. How do we mock or stub a request?
69. Bypass
70. Bypass
71. setup do bypass = Bypass.open {:ok, bypass: bypass} end
72. test "fetch/1 returns and formats tweets", %{bypass: bypass} do response = Jason.encode!([%{"text" => "Elixir Brasil 2019"}]) Bypass.expect(bypass, fn conn -> assert "/1.1/search/tweets.json" == conn.request_path assert "GET" == conn.method Plug.Conn.resp(conn, 200, response) end) tweets = TwitterClient.fetch("http://localhost:#{bypass.port}") assert tweets == [%{"text" => "Elixir Brasil 2019"}] end
73. test "fetch/1 returns and formats tweets", %{bypass: bypass} do response = Jason.encode!([%{"text" => "Elixir Brasil 2019"}]) Bypass.expect(bypass, fn conn -> assert "/1.1/search/tweets.json" == conn.request_path assert "GET" == conn.method Plug.Conn.resp(conn, 200, response) end) tweets = TwitterClient.fetch("http://localhost:#{bypass.port}") assert tweets == [%{"text" => "Elixir Conf 2019"}] end
74. def fetch(url \\ "https://api.twitter.com") do {:ok, response} = HTTPoison.get("#{url}/1.1/search/tweets.json") Jason.decode!(response.body) end
75. When to use Bypass?
76. Code that needs to make an HTTP request
78. Mox
79. Timeline TwitterClient
80. test "messages/0 lists all messages from the timeline" do TwitterMock |> expect(:fetch, fn -> [%{"text" => "Olá mundo"}] end) assert Timeline.messages() == {:ok, 1} end https://github.com/amandasposito/mox_example
81. http://blog.plataformatec.com.br/2015/10/mocks-and-explicit-contracts
82. How to test processes?
83. GenServer
84. It is a mindset change about the way we deal with the state
85. We don't eliminate the state, we control it
86. Functional paradigm helps you turn the state more explicit
87. defmodule NewBank.Counter do use GenServer # Client def start_link(opts) do GenServer.start_link(__MODULE__, opts, name: Keyword.get(opts, :name)) end def increment() do GenServer.cast(__MODULE__, :increment) end def count() do GenServer.call(__MODULE__, :fetch) end end
88. defmodule NewBank.CounterServer do ... # Server @impl true def handle_cast(:increment, _from) do default = 1 value = :ets.update_counter(:counter_table, "counter_key", 1, default) {:reply, :ok, value} end @impl true def handle_call(:fetch, _from, counter) do {:reply, :ets.lookup(:counter_table, "counter_key"), counter} end end
89. How do I test it now?
90. To test a GenServer Callback Is Not a Good Practice
91. ✉
92. GenServer.cast
93. We don't wait for an answer
94. GenServer.call
95. We expect an answer
96. test "increments and returns the counter current number" do start_supervised!(CounterServer) CounterServer.increment() assert CounterServer.count() == 1 end
97. defmodule NewBank.CounterServer do ... # Server @impl true def handle_cast(:increment, _from) do value = Counter.really_complex |> Counter.new_feature |> Counter.we_need_to_implement |> Counter.increment() {:reply, :ok, value} end @impl true def handle_call(:fetch, _from, counter) do value = Counter.different_way |> Counter.fetch {:reply, value, counter} end end
98. What are the most common problems?
99. •Everything I've learned in OOP, I'm going to throw away in functional programming? •How do I organize my code? •Do all the problems I had in OOP disappear in functional programming? •What about this Context? How do I use it?
100. Chances are the code will be less complex
101. But the problems still exist
102. Many of the problems we see in OOP can be seen in FP
103. •Long functions •Functions that are hard to test •Simple changes need to be done in many places •Feature Envy •Context with too many lines •Tight coupling
104. https://youtu.be/eldYot7uxUc
105. We still want to
106. Minimize the number of modules affected by a change
107. To have a reliable interface contract
108. High-level policy to be independent of low-level details
110. How can we organize our code?
111. Contexts
112. How does it work?
113. Where do I put my code now?
114. What should be its responsibilities?
115. Contexts are boundaries between your application modules
117. defmodule NewBank.CreditCard do @moduledoc """ The CreditCard context. """ def create do ... end end
118. defmodule NewBank.CreditCard do @moduledoc """ The CreditCard context. """ def create do ... end def fetch do ... end def update do ... end def delete do ... end def lists do ... end def fetch_available_limit do .. end end
119. As time goes by
120. def fetch_transactions do .. end def count_transactions do ... end def total_transactions_by_user do ... end defp confirmations do ... end def list_top_credit_cards do ... end def fetch_pending_transactions do ... end def count_pending_transactions do ... end defp join_association(query, [{association, nested_preload}]) do ... end defp page_transactions do ... end defp where_transaction_has_multiple_disputes do ... end
121. The need to interact with other schemas increase
122. Contexts can get bigger than they should
123. Move orthogonal functionality out of the Context
124. def create_reward do ... end def update_reward do .. end def delete_reward do ... end
125. Move queries closer to their schema
126. def fetch_transactions do .. end def fetch_pending_transactions do ... end defp join_association(query, [{association, nested_preload}]) do ... end defp page_transactions do ... end defp where_transaction_has_multiple_disputes do ... end
128. Thank you! amandasposito.com