Testing Oban Jobs From the Inside Out

Background jobs with Oban are integral to the business side of many applications. Knowing that your jobs function correctly in isolation, and when composed with other jobs, and when integrated with the rest of the system is essential for peace of mind. Fortunately, Oban provides all the tools necessary to test jobs at any level of isolation.

Together we’ll explore how to make optimal use of Oban’s robust testing functionality for unit, integration, and acceptance tests. Come learn how to test Oban from the inside out—from asserting that workers return correct values to assuring every job in a queue is executed as it would be in production.

Transcript

1. Testing Oban Jobs from the Inside Out
2. Parker Selbert @sorentwo Dscout / Soren Hiring!!! Not Hiring…
3. Oban is a persistent background job system with layered components. (queues, workers, jobs, plugins, telemetry)
4. Automated testing is an essential component of any reliable application. (at least those that are long-lived and actively developed 😉)
5. We need to verify that all moving parts are working together as we expect.
6. Database Application Queue Queue Queue Worker Job Job Job Job Job Job Job Job Worker Worker Context Controller
7. Where you focus test effort depends on performance, isolation, and boundaries.
8. Unit Integration Acceptance Recommended Testing Pyramid
9. Background jobs are always about side effects.
10. Great tests are async, isolated within transactions.
11. Testing Setup
12. Play nicely with Ecto’s SQL.Sandbox for isolated testing.
13. config :my_app, Oban, repo: MyApp.Repo, plugins: [Oban.Plugins.Pruner, Oban.Plugins.Stager], queues: [ default: 10, events: 20, emails: 5 ] config.exs
14. config :my_app, Oban, plugins: false, queues: false test.exs
15. Use Oban.Testing for unit, integration, and acceptance tests.
16. defmodule MyApp.Case do use ExUnit.CaseTemplate using do quote do use Oban.Testing, repo: MyApp.Repo end end setup do :ok = Ecto.Adapters.SQL.Sandbox.mode(MyApp.Repo, :manual) :ok = Ecto.Adapters.SQL.Sandbox.checkout(MyApp.Repo) end end MyApp.Repo.start_link() ExUnit.start() test_helper.exs
17. Now we’re ready for some Testing!
18. Unit Testing
19. Application Worker Worker Worker Worker Database Unit
20. @doc """ Construct a job and execute it with a worker module. """ @spec perform_job( worker :: Worker.t(), args :: term(), opts :: [Job.option() | {:repo, module()}] ) :: Worker.result() oban/testing.ex
21. defmodule MyApp.ActivationWorkerTest do use MyApp.Case, async: true test "activating a new account" do account = MyApp.Account.create(name: “ElixirConf”, email: “parker@example.com”) {:ok, account} = perform_job(MyApp.ActivationWorker, %{id: account.id}) assert account.active? end end activation_worker_test.exs
22. defmodule MyApp.ActivationWorkerTest do use MyApp.Case, async: true test "activating a new account" do account = MyApp.Account.create(name: “ElixirConf”, email: “parker@example.com") {:ok, account} = perform_job(MyApp.ActivationWorker, %{id: account.id}) assert account.active? end end activation_worker_test.exs
23. 1) test activating a new account (MyApp.AccountWorkerTest) Expected worker to be a module that implements the Oban.Worker behaviour, got: MyApp.AccountWorker code: {:ok, account} = perform_job(MyApp.AccountWorker, %{id: account.id})
24. defmodule MyApp.ActivationWorker do use Oban.Worker @impl Worker def perform(%Job{args: %{id: account_id}}) do MyApp.Account.activate(account_id) end end activation_worker.ex
25. 1) test activating a new account (MyApp.AccountWorkerTest) ** (FunctionClauseError) no function clause matching in MyApp.AccountWorker.perform/1 The following arguments were given to MyApp.AccountWorker.perform/1: # 1 %Oban.Job{args: %{"id" => 1}, attempt: 1, worker: "MyApp.AccountWorker"} code: {:ok, account} = perform_job(MyApp.AccountWorker, %{id: account.id})
26. defmodule MyApp.ActivationWorker do use Oban.Worker @impl Worker def perform(%Job{args: %{"id" => account_id}}) do MyApp.Account.activate(account_id) end end activation_worker.ex
27. 1) test activating a new account (MyApp.AccountWorkerTest) Expected result to be one of - `:ok` - `:discard` - `{:ok, value}` - `{:error, reason}` - `{:discard, reason}` - `{:snooze, duration} Instead received: {:activated, %MyApp.Account{active?: true, id: 1, name: nil}} code: {:ok, account} = perform_job(MyApp.AccountWorker, %{id: account.id})
28. defmodule MyApp.ActivationWorker do use Oban.Worker @impl Worker def perform(%Job{args: %{id: account_id}}) do with {:activated, account} <- MyApp.Account.activate(account_id) do {:ok, account} end end end activation_worker.ex
29. . Finished in 0.3 seconds (0.2s on load, 0.08s async, 0.00s sync) 1 test, 0 failures
30. defmodule MyApp.ActivationWorkerTest do use MyApp.Case, async: true test "activating a new account" do account = MyApp.Account.create(name: “ElixirConf”, email: “parker@example.com”) {:ok, account} = perform_job(MyApp.ActivationWorker, %{id: account.id}, priority: 9) assert account.active? end end activation_worker_test.exs
31. 1) test activating a new account (MyApp.ActivationWorkerTest) Expected args and opts to build a valid job, got validation errors: priority: must be less than 4 code: {:ok, account} = perform_job(MyApp.ActivationWorker, %{id: id}, priority: 9)
32. defmodule MyApp.ActivationWorkerTest do use MyApp.Case, async: true test "activating a new account" do account = MyApp.Account.create(name: “ElixirConf”, email: “parker@example.com”) {:ok, account} = perform_job(MyApp.ActivationWorker, %{id: account.id}, priority: 3) assert account.active? end end activation_worker_test.exs
33. . Finished in 0.3 seconds (0.2s on load, 0.08s async, 0.00s sync) 1 test, 0 failures
34. Use perform_job/1 to: • Ensure Worker Behavior • Check String Keys in perform/1 • Verify Return Values • Encode/Decode Args to JSON • Validate Job Opts • Exercise a Custom new/1 Callback
35. Integration Testing
36. Application Worker Worker Worker Worker Database Integration
37. @doc """ Assert that a job with particular options has been enqueued. """ @spec assert_enqueued(opts :: Keyword.t()) :: true @doc """ Refute that a job with particular options has been enqueued. """ @spec refute_enqueued(opts :: Keyword.t()) :: false oban/testing.ex
38. defmodule MyApp.AccountTest do use MyApp.Case, async: true test "scheduling activation upon sign up" do {:ok, account} = MyApp.Account.sign_up(name: “ElixirConf”, email: “parker@example.com") assert_enqueued worker: MyApp.ActivationWorker, args: %{id: account.id}, queue: :default end end account_test.exs
39. defmodule MyApp.AccountTest do use MyApp.Case, async: true test "scheduling activation upon sign up" do {:ok, account} = MyApp.Account.sign_up(name: “ElixirConf”, email: “parker@example.com") assert_enqueued worker: MyApp.ActivationWorker, args: %{id: account.id}, queue: :default end end account_test.exs
40. 1) test scheduling activation upon sign up (MyApp.AccountTest) Expected a job matching: %{args: %{id: 4}, queue: :default, worker: MyApp.ActivationWorker} to be enqueued in the "public" schema. Instead found: [] code: assert_enqueued worker: MyApp.ActivationWorker,
41. defmodule MyApp.Account do def sign_up(args) do changeset = changeset(args) with {:ok, account} <- MyApp.Repo.insert(changeset) do %{id: account.id} |> MyApp.ActivationWorker.new() |> Oban.insert() {:ok, account} end end account.ex
42. . Finished in 0.1 seconds (0.1s async, 0.00s sync) 1 test, 0 failures
43. def sign_up(args) do changeset = changeset(args) with {:ok, account} <- MyApp.Repo.insert(changeset) do Task.start(fn -> Process.sleep(100) %{id: account.id} |> MyApp.ActivationWorker.new() |> Oban.insert() end) {:ok, account} end end account.ex
44. 1) test scheduling activation upon sign up (MyApp.AccountTest) Expected a job matching: %{args: %{id: 4}, queue: :default, worker: MyApp.ActivationWorker} to be enqueued in the "public" schema. Instead found: [] code: assert_enqueued worker: MyApp.ActivationWorker,
45. test "scheduling activation upon sign up" do {:ok, account} = MyApp.Account.sign_up(name: "ElixirConf", email: "parker@example.com") expected_opts = [ worker: MyApp.ActivationWorker, args: %{id: account.id}, queue: :default ] assert_enqueued expected_opts, 150 end account_test.exs
46. . Finished in 0.1 seconds (0.1s async, 0.00s sync) 1 test, 0 failures
47. test "skipping activation on failed sign up" do {:error, _changeset} = MyApp.Account.sign_up(name: "", email: "") refute_enqueued [worker: MyApp.ActivationWorker], 150 end account_test.exs
48. . Finished in 0.1 seconds (0.1s async, 0.00s sync) 2 tests, 0 failures
49. Use assert_enqueued/* and refute_enqueued/* to: • Assert Jobs in the Database • Refute Jobs in the Database • Query by Worker/Args/Meta/Queue/ScheduledAt/Etc. • Compensate for Async Activity with Timeouts
50. Acceptance Testing
51. Application Database Worker Worker Worker Worker Acceptance
52. @type drain_option :: {:queue, queue_name()} | {:with_limit, pos_integer()} | {:with_recursion, boolean()} | {:with_safety, boolean()} | {:with_scheduled, boolean()}
 @type drain_result :: %{ failure: non_neg_integer(), snoozed: non_neg_integer(), success: non_neg_integer() } @doc """ Synchronously execute all available jobs in a queue. """ @spec drain_queue([drain_option()]) :: drain_result() oban.ex
53. Wait… That’s not in the Oban.Testing module!
54. test "delivering a follow-up welcome email after sign up" do {:ok, account} = MyApp.Account.sign_up(name: "ElixirConf", email: "parker@example.com") assert %{success: 1} = Oban.drain_queue(queue: :emails) assert_email_sent MyApp.AccountEmail.welcome(account) end account_test.exs
55. test "delivering a follow-up welcome email after sign up" do {:ok, account} = MyApp.Account.sign_up(name: "ElixirConf", email: "parker@example.com") assert %{success: 1} = Oban.drain_queue(queue: :emails) assert_email_sent MyApp.AccountEmail.welcome(account) end account_test.exs
56. test "delivering a follow-up welcome email after sign up" do {:ok, account} = MyApp.Account.sign_up(name: "ElixirConf", email: "parker@example.com") assert %{success: 1} = Oban.drain_queue(queue: :emails) assert_email_sent MyApp.AccountEmail.welcome(account) end account_test.exs
57. 1) test delivering a follow-up welcome email after sign up (MyApp.AccountTest) match (=) failed code: assert %{success: 1} = Oban.drain_queue(queue: :emails) left: %{success: 1} right: %{success: 0, failure: 0, snoozed: 0}
58. def sign_up(args) do changeset = changeset(args) with {:ok, account} <- MyApp.Repo.insert(changeset) do Oban.insert_all([ MyApp.ActivationWorker.new(%{id: account.id}), MyApp.WelcomeWorker.new(%{id: account.id}, schedule_in: {1, :hour}) ]) {:ok, account} end end account.ex
59. defmodule MyApp.WelcomeWorker do use Oban.Worker, queue: :emails @impl Worker def perform(%Job{args: %{"id" => account_id}}) do {:ok, account} = MyApp.Repo.get(MyApp.Account, account_id) account |> MyApp.AccountEmail.welcome() |> MyApp.Mailer.deliver() end end welcome_worker.ex
60. 1) test delivering a follow-up welcome email after sign up (MyApp.AccountTest) match (=) failed code: assert %{success: 1} = Oban.drain_queue(queue: :emails) left: %{success: 1} right: %{success: 0, failure: 0, snoozed: 0}
61. test "delivering a follow-up welcome email after sign up" do {:ok, account} = MyApp.Account.sign_up(name: "ElixirConf", email: "parker@example.com") assert %{success: 1} = Oban.drain_queue(queue: :emails, with_scheduled: true) assert_email_sent MyApp.AccountEmail.welcome(account) end account_test.exs
62. 1) test delivering a follow-up welcome email after sign up (MyApp.AccountTest) match (=) failed code: assert %{success: 1} = Oban.drain_queue(queue: :emails, with_scheduled: true) left: %{success: 1} right: %{success: 0, failure: 1, snoozed: 0}
63. test "delivering a follow-up welcome email after sign up" do {:ok, account} = MyApp.Account.sign_up(name: "ElixirConf", email: "parker@example.com") assert %{success: 1} = Oban.drain_queue(queue: :emails, with_safety: false, with_scheduled: true) assert_email_sent MyApp.AccountEmail.welcome(account) end account_test.exs
64. 1) test delivering a follow-up welcome email after sign up (MyApp.AccountTest) ** (MatchError) no match of right hand side value: %MyApp.Account{active: false, email: "parker@example.com", id: 16, name: "ElixirConf"} code: Oban.drain_queue(queue: :emails, with_scheduled: true, with_safety: false)
65. defmodule MyApp.WelcomeWorker do use Oban.Worker, queue: :emails @impl Worker def perform(%Job{args: %{"id" => account_id}}) do {:ok, account} = MyApp.Repo.get(MyApp.Account, account_id) account |> MyApp.AccountEmail.welcome() |> MyApp.Mailer.deliver() end end welcome_worker.ex
66. defmodule MyApp.WelcomeWorker do use Oban.Worker, queue: :emails @impl Worker def perform(%Job{args: %{"id" => account_id}}) do account = MyApp.Repo.get(MyApp.Account, account_id) account |> MyApp.AccountEmail.welcome() |> MyApp.Mailer.deliver() end end welcome_worker.ex
67. . Finished in 0.2 seconds (0.2s async, 0.00s sync) 3 tests, 0 failures, 0 excluded
68. test "starting our welcome drip after sign up" do {:ok, _account} = MyApp.Account.sign_up(name: "ElixirConf", email: "parker@example.com") assert %{success: 4, failure: 0, snoozed: 0} = Oban.drain_queue(queue: :emails, with_scheduled: true, with_safety: false) end account_test.exs
69. defmodule MyApp.ActivationWorker do use Oban.Worker @impl Worker def perform(%Job{args: %{"id" => account_id} = args}) do with {:activated, account} <- MyApp.Account.activate(account_id) do Oban.insert_all([ MyApp.HelloWorker.new(args, schedule_in: {1, :hour}), MyApp.IntroWorker.new(args, schedule_in: {1, :day}), MyApp.NudgeWorker.new(args, schedule_in: {1, :week}) ]) {:ok, account} end end end activation_worker.ex
70. 1) test starting our welcome drip after sign up (MyApp.AccountTest) match (=) failed code: assert %{success: 4, failure: 0, snoozed: 0} = Oban.drain_queue(queue: :emails, with_scheduled: true, with_safety: false) left: %{failure: 0, snoozed: 0, success: 4} right: %{failure: 0, snoozed: 0, success: 1}
71. test "starting our welcome drip after sign up" do {:ok, _account} = MyApp.Account.sign_up(name: "ElixirConf", email: "parker@example.com") assert %{success: 4, failure: 0, snoozed: 0} = Oban.drain_queue( queue: :emails, with_scheduled: true, with_safety: false, with_recursion: true ) end account_test.exs
72. . Finished in 0.2 seconds (0.2s async, 0.00s sync) 4 tests, 0 failures, 0 excluded
73. Use drain_queue/* to: • Execute All Jobs in a Particular Queue • Make Scheduled Jobs Immediately Available • Surface Job Errors and Exceptions • Recursively Execute All Jobs
74. Inline Testing
75. If you absolutely must run jobs normally during your tests. (for browser testing or live view)
76. config :my_app, Oban, name: Oban.Disabled, plugins: false, queues: false test.exs
77. setup do start_supervised_oban!(queues: [default: 5, emails: 5]) :ok end any_test.exs
78. @doc """ Set up a supvervised version of Oban with the provided options. """ def start_supervised_oban!(opts) do opts = opts |> Keyword.put_new(:name, Oban) |> Keyword.put_new(:repo, Repo) |> Keyword.put_new(:plugins, [{Repeater, interval: 25}]) |> Keyword.put_new(:poll_interval, :timer.minutes(10)) |> Keyword.put_new(:shutdown_grace_period, 25) start_supervised!({Oban, opts}, id: opts[:name]) end test_helper.exs
79. @doc """ Set up a supvervised version of Oban with the provided options. """ def start_supervised_oban!(opts) do opts = opts |> Keyword.put_new(:name, Oban) |> Keyword.put_new(:repo, Repo) |> Keyword.put_new(:plugins, [{Repeater, interval: 25}]) |> Keyword.put_new(:poll_interval, :timer.minutes(10)) |> Keyword.put_new(:shutdown_grace_period, 25) start_supervised!({Oban, opts}, id: opts[:name]) end test_helper.exs
80. @doc """ Set up a supvervised version of Oban with the provided options. """ def start_supervised_oban!(opts) do opts = opts |> Keyword.put_new(:name, Oban) |> Keyword.put_new(:repo, Repo) |> Keyword.put_new(:plugins, [{Repeater, interval: 25}]) |> Keyword.put_new(:poll_interval, :timer.minutes(10)) |> Keyword.put_new(:shutdown_grace_period, 25) start_supervised!({Oban, opts}, id: opts[:name]) end test_helper.exs
81. @doc """ Set up a supvervised version of Oban with the provided options. """ def start_supervised_oban!(opts) do opts = opts |> Keyword.put_new(:name, Oban) |> Keyword.put_new(:repo, Repo) |> Keyword.put_new(:plugins, [{Repeater, interval: 25}]) |> Keyword.put_new(:poll_interval, :timer.minutes(10)) |> Keyword.put_new(:shutdown_grace_period, 25) start_supervised!({Oban, opts}, id: opts[:name]) end test_helper.exs
82. Use start_supervised_oban!/1 to: • Run Jobs Normally in the Background • …Without Scheduling • …Without Isolation • …Without Control
83. Write tests for your Oban jobs! Whether it’s from the inside-out or the outside-in, that’s up to you.
84. 💛 Thanks For Listening