Upgrade to Pro — share decks privately, control downloads, hide ads and more …

[RubyConf 2022] Weaving & seaming mocks

[RubyConf 2022] Weaving & seaming mocks

Video: https://www.youtube.com/watch?v=ZFSS8_pFemc

https://www.rubyconfmini.com/program#Weaving-and-seaming-mocks

To mock or not mock is an important question, but let's leave it apart and admit that we, Rubyists, use mocks in our tests.

Mocking is a powerful technique, but even when used responsibly, it could lead to false positives in our tests (thus, bugs leaking to production): fake objects could diverge from their real counterparts.

In this talk, I'd like to discuss various approaches to keeping mocks in line with the actual implementation and present a brand new idea based on mock fixtures and contracts.

Vladimir Dementyev

November 16, 2022
Tweet

More Decks by Vladimir Dementyev

Other Decks in Programming

Transcript

  1. 2
    Ruby Devs ! Tests

    View full-size slide

  2. How do you
    write tests?

    View full-size slide

  3. palkan_tula
    palkan
    4
    Martin Fowler, Mocks Aren't Stubs

    View full-size slide

  4. palkan_tula
    palkan
    5
    Classical
    def search(q)
    query = Query.build(q)
    User.find_by_sql(query.to_sql)
    end

    View full-size slide

  5. palkan_tula
    palkan
    5
    Classical
    def search(q)
    query = Query.build(q)
    User.find_by_sql(query.to_sql)
    end
    user = create(:user, name: "Vova")
    expect(search("name=Vova")).to eq([user])

    View full-size slide

  6. palkan_tula
    palkan
    Mockist
    5
    def search(q)
    query = Query.build(q)
    User.find_by_sql(query.to_sql)
    end
    expect(Query)
    .to receive(:build).with("name=x")
    .and_return(double(to_sql: :cond))
    expect(User).to receive(:find_by_sql).with(:cond)
    .and_return(:users)
    expected(search("user=x")).to eq :users

    View full-size slide

  7. palkan_tula
    palkan
    To mock or not to mock?
    Emily Samp @ RubyConf, 2021
    6

    View full-size slide

  8. palkan_tula
    palkan
    Classical
    7

    View full-size slide

  9. palkan_tula
    palkan
    8
    Mockist

    View full-size slide

  10. palkan_tula
    palkan
    9
    Mocklassicism

    View full-size slide

  11. palkan_tula
    palkan
    False positive
    10
    def search(q)
    query = Query.build(q)
    User.find_by_sql(query.to_sql)
    end
    expect(Query)
    .to receive(:build).with("name=x")
    .and_return(double(to_sql: :cond))
    expect(User).to receive(:find_by_sql).with(:cond)
    .and_return(:users)
    expected(search("user=x")).to eq :users
    module Query
    - def self.build(query_str)
    - query_hash = Hash[query.split("=")]
    + def self.build(query_hash)
    # query has to sql
    end

    View full-size slide

  12. palkan_tula
    palkan
    How to avoid false positives?
    11

    View full-size slide

  13. palkan_tula
    palkan
    11
    How to put seams?
    ?

    View full-size slide

  14. palkan_tula
    palkan
    github.com/palkan
    12

    View full-size slide

  15. Keeping mocks
    in line with real
    objects
    16

    View full-size slide

  16. palkan_tula
    palkan
    17
    Case study: anyway_config
    class RubyConfConfig < Anyway::Config
    attr_config :city, :date
    coerce_types date: :date
    end
    github.com/palkan/anyway_config

    View full-size slide

  17. 18
    $ RUBYCONF_CITY Providence RUBYCONF_DATE 2022 11 15 \
    ruby -r rubyconf_conf.rb -e "pp RubyConfConfig.new"
    #config_name="rubyconf"
    env_prefix="RUBYCONF"
    values:
    city => "Providence" (type=env key=RUBYCONF_CITY),
    date =>
    # (type=env key=RUBYCONF_DATE)>
    #config_name="rubyconf"
    env_prefix="RUBYCONF"
    values:
    city => "Providence" (type=env key=RUBYCONF_CITY),
    date =>
    # (type=env key=RUBYCONF_DATE)>

    View full-size slide

  18. palkan_tula
    palkan
    19
    ENV parser
    class Anyway::Env
    def fetch(prefix)
    # Scans ENV and parses matching values
    some_hash
    end
    def fetch_with_trace(prefix)
    [fetch(prefix), traces[prefix]]
    end
    end

    View full-size slide

  19. palkan_tula
    palkan
    20
    ENV loader
    class Anyway::Loaders::Env
    def call(env_prefix:, **_options)
    env = Anyway::Env.new
    env.fetch_with_trace(env_prefix)
    .then do |(data, trace)|
    Tracing.current_trace&.merge!(trace)
    data
    end
    end
    end

    View full-size slide

  20. 21
    $ bundle exec rspec --tags env
    Run options:
    include {:env=>true}
    Randomized with seed 26894
    ........
    Finished in 0.00641 seconds (files took 0.27842 seconds to load)
    8 examples, 0 failures
    Coverage is at 100.0%.
    Coverage report sent to Coveralls.

    View full-size slide

  21. palkan_tula
    palkan
    22
    ENV loader
    describe Anyway::Loaders::Env do
    subject { described_class.call(env_prefix: "TESTO") }
    it "loads data from env" do
    with_env(
    "TESTO_A" => "x",
    "TESTO_DATA__KEY" => "value"
    ) do
    expect(subject)
    .to eq({"a" => "x", "data" => { "key" => "value"}})
    end
    end
    end
    Aren't we testing ENV
    parser here, not loader?

    View full-size slide

  22. palkan_tula
    palkan
    describe Anyway::Loaders::Env do
    let(:env) { double("env") }
    let(:data) { {"a" => "x", "data" => {"key" => "value"}} }
    subject { described_class.call(env_prefix: "TESTO") }
    before do
    allow(::Anyway::Env).to receive(:new).and_return(env)
    allow(env).to receive(:fetch_with_trace).and_return([data, nil])
    end
    it "loads data from Anyway::Env" do
    expect(subject)
    .to eq({"a" => "x", "data" => {"key" => "value"}})
    end
    end
    describe Anyway::Loaders::Env do
    let(:env) { double("env") }
    let(:data) { {"a" => "x", "data" => {"key" => "value"}} }
    subject { described_class.call(env_prefix: "TESTO") }
    before do
    allow(::Anyway::Env).to receive(:new).and_return(env)
    allow(env).to receive(:fetch_with_trace).and_return([data, nil])
    end
    it "loads data from Anyway::Env" do
    expect(subject)
    .to eq({"a" => "x", "data" => {"key" => "value"}})
    end
    end
    23
    Double-factoring

    View full-size slide

  23. palkan_tula
    palkan
    describe Anyway::Loaders::Env do
    let(:env) { double("env") }
    let(:data) { {"a" => "x", "data" => {"key" => "value"}} }
    subject { described_class.call(env_prefix: "TESTO") }
    before do
    allow(::Anyway::Env).to receive(:new).and_return(env)
    allow(env).to receive(:fetch_with_trace).and_return([data
    end
    it "loads data from Anyway::Env" do
    expect(subject)
    describe Anyway::Loaders::Env do
    let(:env) { double("env") }
    let(:data) { {"a" => "x", "data" => {"key" => "value"}} }
    subject { described_class.call(env_prefix: "TESTO") }
    before do
    allow(::Anyway::Env).to receive(:new).and_return(env)
    allow(env).to receive(:fetch_with_trace).and_return([data
    end
    it "loads data from Anyway::Env" do
    expect(subject)
    23
    Double-factoring

    View full-size slide

  24. Case #1.
    Undefined method
    24

    View full-size slide

  25. palkan_tula
    palkan
    25
    Refactor
    class Anyway::Env
    - def fetch(prefix)
    + def fetch(prefix, include_trace = false)
    @@ ...
    - def fetch_with_trace(prefix)
    - [fetch(prefix), traces[prefix]]
    - end
    end

    View full-size slide

  26. palkan_tula
    palkan
    26
    describe Anyway::Loaders::Env do
    let(:env) { double("env") }
    before do
    allow(::Anyway::Env)
    .to receive(:new).and_return(env)
    allow(env)
    .to receive(:fetch_with_trace)
    .and_return([data, nil])
    end
    # ...
    end
    Refactor

    View full-size slide

  27. 27
    $ bundle exec rspec --tags env
    Run options:
    include {:env=>true}
    Randomized with seed 26894
    ........
    Finished in 0.00594 seconds (files took 0.21516 seconds to load)
    8 examples, 0 failures
    Coverage is at 100.0%.
    Coverage report sent to Coveralls.

    View full-size slide

  28. 28
    $ RUBYCONF_CITY Providence RUBYCONF_DATE 2022 11 15 \
    ruby -r rubyconf_conf.rb -e "pp RubyConfConfig.new"
    #config_name="rubyconf"
    env_prefix="RUBYCONF"
    values:
    city => "Providence" (type=env key=RUBYCONF_CITY),
    date =>
    # (type=env key=RUBYCONF_DATE)>
    anyway_config/lib/anyway/loaders/env.rb:11:in `call': undefined
    method `fetch_with_trace' for #@type_cast=Anyway::NoCast, @data={}, @traces={}> (NoMethodError)
    env.fetch_with_trace(env_prefix).then do |(conf, trace)|
    ^^^^^^^^^^^^^^^^^

    View full-size slide

  29. palkan_tula
    palkan
    Double trouble
    — Tests are green ✅
    — Coverage 100% #
    — Code doesn't work ❌
    29

    View full-size slide

  30. palkan_tula
    palkan
    31
    github.com/rspec/rspec-mocks/issues/227

    View full-size slide

  31. palkan_tula
    palkan
    32
    github.com/xaviershay/rspec-fire

    View full-size slide

  32. palkan_tula
    palkan
    33
    describe Anyway::Loaders::Env do
    - let(:env) { double("env") }
    - let(:env) { instance_double("Anyway::Env") }
    Refactor

    View full-size slide

  33. 34
    $ bundle exec rspec --tags env
    1) Anyway::Loaders::Env loads data from Anyway::Env
    Failure/Error: allow(env_double).to
    receive(:fetch_with_trace).and_return([env, nil])
    the Anyway::Env class does not implement the instance
    method: fetch_with_trace
    # ./spec/loaders/env_spec.rb:22:in `block (2 levels) in
    '

    View full-size slide

  34. palkan_tula
    palkan
    35
    double
    No strings attached

    View full-size slide

  35. palkan_tula
    palkan
    verified double
    35
    double
    No strings attached Method existence

    View full-size slide

  36. Case #2.
    Incorrect method
    signature
    36

    View full-size slide

  37. palkan_tula
    palkan
    37
    Refactor
    class Anyway::Env
    - def fetch(prefix, include_trace = false)
    + def fetch(prefix, include_trace: false)
    @@ ...

    View full-size slide

  38. palkan_tula
    palkan
    37
    Refactor
    class Anyway::Env
    - def fetch(prefix, include_trace = false)
    + def fetch(prefix, include_trace: false)
    @@ ...
    describe Anyway::Loaders::Env do
    let(:env) { instance_double("env") }
    before do
    expect(env)
    .to receive(:fetch)
    .with("TESTO", true)
    .and_return([data, nil])
    end
    # ...
    end

    View full-size slide

  39. 38
    $ bundle exec rspec --tags env
    1) Anyway::Loaders::Env loads data from Anyway::Env
    Failure/Error:
    env.fetch(env_prefix, true)
    ArgumentError:
    Wrong number of arguments. Expected 1, got 2.
    # ./lib/anyway/loaders/env.rb:11:in `call'
    # ./lib/anyway/loaders/base.rb:10:in `call'
    # ./spec/loaders/env_spec.rb:18:in

    View full-size slide

  40. palkan_tula
    palkan
    39
    class MethodSignatureVerifier
    def initialize(signature, args=[])
    # ...
    end
    def valid?
    missing_kw_args.empty? &&
    invalid_kw_args.empty? &&
    valid_non_kw_args? &&
    arbitrary_kw_args? &&
    unlimited_args?
    end
    end
    rspec/support/method_signature_verifier.rb

    View full-size slide

  41. palkan_tula
    palkan
    39
    rspec/support/method_signature_verifier.rb
    class MethodSignature
    def initialize(method)
    # ...
    classify_parameters
    end
    def classify_parameters
    @method.parameters.each do |(type, name)|
    # ...
    end
    end
    end

    View full-size slide

  42. palkan_tula
    palkan
    class MethodSignature
    def initialize(method)
    # ...
    classify_parameters
    end
    def classify_parameters
    @method.parameters.each do |(type, name)|
    # ...
    end
    end
    end
    40
    rspec/support/method_signature_verifier.rb

    View full-size slide

  43. palkan_tula
    palkan
    41
    Method#parameters
    method_obj = Anyway::Env.instance_method(:fetch)
    method_obj.parameters
    #=> [
    [:req, :prefix],
    [:key, :include_trace]
    ]

    View full-size slide

  44. palkan_tula
    palkan
    42
    double verified double
    No strings attached Method existence
    Method parameters
    Method signature?

    View full-size slide

  45. palkan_tula
    palkan
    Method signature
    — Parameters shape
    — Argument types
    — Return value type
    43

    View full-size slide

  46. palkan_tula
    palkan
    44
    Refactor
    class Anyway::Env
    + Parsed = Struct.new(:data, :trace)
    def fetch(prefix, include_trace: false)
    @@ ...
    - [data, trace]
    + Parsed.new(data, trace)
    end

    View full-size slide

  47. palkan_tula
    palkan
    44
    Refactor
    class Anyway::Env
    + Parsed = Struct.new(:data, :trace)
    def fetch(prefix, include_trace: false)
    @@ ...
    - [data, trace]
    + Parsed.new(data, trace)
    end
    describe Anyway::Loaders::Env do
    let(:env) { instance_double("env") }
    before do
    expect(env)
    .to receive(:fetch)
    .with("TESTO", include_trace: true)
    .and_return([data, nil])
    end
    # ...
    end

    View full-size slide

  48. 45
    $ bundle exec rspec --tags env
    Run options:
    include {:env=>true}
    Randomized with seed 43108
    ........
    Finished in 0.0234 seconds (files took 0.80197 seconds to load)
    7 examples, 0 failures

    View full-size slide

  49. 46
    $ RUBYCONF_CITY Providence RUBYCONF_DATE 2022 11 15 \
    ruby -r rubyconf_conf.rb -e "pp RubyConfConfig.new"
    #config_name="rubyconf"
    env_prefix="RUBYCONF"
    values:
    city => "Providence" (type=env key=RUBYCONF_CITY),
    date =>
    # (type=env key=RUBYCONF_DATE)>
    anyway_config/lib/anyway/tracing.rb:59:in `merge!': undefined
    method `trace?' for nil:NilClass (NoMethodError)
    from anyway_config/lib/anyway/loaders/env.rb:12:in `block
    in call'
    from :124:in `then'
    from anyway_config/lib/anyway/loaders/env.rb:11:in `call'

    View full-size slide

  50. palkan_tula
    palkan
    “Test doubles are sweet for isolating your unit
    tests, but we lost something in the
    translation from typed languages. Ruby
    doesn't have a compiler that can verify the
    contracts being mocked out are indeed legit.”
    –rspec-fire's Readme
    47

    View full-size slide

  51. Mocks vs.
    types
    48

    View full-size slide

  52. palkan_tula
    palkan
    .rbs
    49
    double verified double
    No strings attached Method existence
    Method parameters

    View full-size slide

  53. palkan_tula
    palkan
    typed double
    49
    double verified double
    No strings attached Method existence
    Method parameters
    Method signature

    View full-size slide

  54. palkan_tula
    palkan
    typed_double
    — Intercept mocked calls
    — Type check them—that's it!
    50

    View full-size slide

  55. palkan_tula
    palkan
    51
    Interception
    RSpec::Mocks::VerifyingMethodDouble.prepend(
    Module.new do
    def proxy_method_invoked(obj, *args, &block)
    super.tap { TypedDouble.typecheck!(obj, *args) }
    end
    end
    )

    View full-size slide

  56. palkan_tula
    palkan
    52
    RBS::Test

    View full-size slide

  57. palkan_tula
    palkan
    53
    anyway_config.rbs
    module Anyway
    class Env
    class Parsed
    attr_reader data: Hash
    attr_reader trace: Tracing::Trace?
    end
    def fetch: (String prefix, ?include_trace: bool) -> Parsed
    end
    end

    View full-size slide

  58. 54
    $ bundle exec rspec --tags env
    1) Anyway::Loaders::Env loads data from Anyway::Env
    Failure/Error: raise
    RBS::Test::Tester::TypeError.new(errors) unless errors.empty?
    RBS::Test::Tester::TypeError:
    TypeError: [Anyway::Env#fetch] ReturnTypeError: expected
    `::Anyway::Env::Parsed` but returns `[{"a"=>"x",
    "data"=>{"key"=>"value"}}, nil]`

    View full-size slide

  59. palkan_tula
    palkan
    What if... we don't
    have types %
    55

    View full-size slide

  60. On-the-fly type
    signatures
    generation
    56

    View full-size slide

  61. palkan_tula
    palkan
    Type generators
    — rbs prototype / tapioca
    — TypeProfiler
    — Tracing → signatures &
    57

    View full-size slide

  62. palkan_tula
    palkan
    On-the-fly types
    — Collect method calls made on real objects
    58

    View full-size slide

  63. palkan_tula
    palkan
    59
    Tracking calls
    TracePoint.trace(:call, :return) do |tp|
    next unless trackable?(tp.defined_class, tp.method_id)
    target, mid = tp.defined_class, tp.method_id
    if tp.event == :call
    method = tp.self.method(mid)
    args = []
    kwargs = {}
    method.parameters.each do |(type, name)|
    val = tp.binding.local_variable_get(name)
    # ...
    end
    store[target][mid] << CallTrace.new(arguments: args, kwargs:)
    elsif tp.event == :return
    store[target][mid].last.return_value = tp.return_value
    end
    end

    View full-size slide

  64. palkan_tula
    palkan
    60
    Tracking calls
    if tp.event == :call
    method = tp.self.method(mid)
    args = []
    kwargs = {}
    method.parameters.each do |(type, name)|
    val = tp.binding.local_variable_get(name)
    # ...
    end

    View full-size slide

  65. palkan_tula
    palkan
    On-the-fly types
    — Collect method calls made on real objects
    — Generate types from real call traces
    61

    View full-size slide

  66. palkan_tula
    palkan
    62
    SignatureGenerator
    class SignatureGenerator
    def to_rbs = [header, method_sigs, footer].join("\n")
    def args_sig(args)
    args.transpose.map do |arg_values|
    arg_values.map(&:class).uniq.map do
    "::#{_1.name}"
    end
    end.join(", ")
    end
    end

    View full-size slide

  67. palkan_tula
    palkan
    63
    env.rbs
    module Anyway
    class Env
    def initialize: (?type_cast: (::Module)) -> (void)
    def fetch: (
    ::String,
    ?include_trace: (::FalseClass | ::TrueClass)
    ) -> ::Anyway::Env::Parsed
    end
    end

    View full-size slide

  68. palkan_tula
    palkan
    On-the-fly types
    — Collect method calls made on real objects
    — Generate types from real call traces
    — Identify tracing targets (mocked classes) &
    64

    View full-size slide

  69. Mock
    fixtures
    65

    View full-size slide

  70. palkan_tula
    palkan
    Fixturama
    evilmartians.com/chronicles/a-fixture-based-approach-to-interface-testing-in-rails
    66

    View full-size slide

  71. palkan_tula
    palkan
    67
    Fixturama
    github.com/nepalez/fixturama
    # fixtures/stubs/notifier.yml
    ---
    - class: Notifier
    chain:
    - create
    arguments:
    - :profileDeleted
    - <%= profile_id %>
    actions:
    - return: true
    - raise: ActiveRecord::RecordNotFound
    arguments:
    - "Profile with id: 1 not found" # for error message

    View full-size slide

  72. palkan_tula
    palkan
    YAML fixtures
    — YAML != Ruby, hard to mock with non-
    primitive types
    — Existing mocks are not re-usable—a lot of
    refactoring
    68

    View full-size slide

  73. palkan_tula
    palkan
    69
    Mock context
    mock_context "Anyway::Env" do
    before do
    env_double = instance_double("Anyway::Env")
    allow(Anyway::Env).to receive(:new).and_return(env_double)
    data = {"a" => "x", "data" => {"key" => "value"}}
    allow(env_double)
    .to receive(:fetch).with("TESTO", any_args)
    .and_return([data, nil])
    end
    end

    View full-size slide

  74. palkan_tula
    palkan
    Mock context
    — Just a shared context
    — Evaluated within a "scratch" example group
    on initialization to collect information
    about stubbed classes and methods
    70

    View full-size slide

  75. palkan_tula
    palkan
    71
    Mock context
    def evaluate_context!(context_id, tracking)
    Class.new(RSpec::Core::ExampleGroup) do
    include_context(context_id)
    specify("true") { expect(true).to be(true) }
    after do
    RSpec::Mocks.space.proxies.values.each do
    tracking.register_from_proxy(_1)
    end
    end
    end.run
    end

    View full-size slide

  76. palkan_tula
    palkan
    72
    Refactor
    describe Anyway::Loaders::Env do
    - let(:env) { instance_double(Anyway::Env) }
    + include_mock_context "Anyway::Env"
    @@ ...

    View full-size slide

  77. palkan_tula
    palkan
    73
    RSpec post-check
    config.after(:suite) do
    TypedDouble.infer_types_from_calls!(
    CallsTracer.calls
    )
    passed = MocksTracer.calls.all do
    TypedDouble.typecheck(_1)
    end
    unless passed
    exit(RSpec.configuration.failure_exit_code)
    end
    end

    View full-size slide

  78. palkan_tula
    palkan
    74
    Mocked objects
    finder
    Mocked calls
    collector
    Real calls
    collector
    Type checker
    Types generator
    after(:suite)
    before(:suite) run time

    View full-size slide

  79. Case #3.
    Non-matching
    behaviour
    75

    View full-size slide

  80. palkan_tula
    palkan
    76
    Refactor
    class Anyway::Env
    def fetch(prefix, **)
    + return if prefix.empty?
    @@ ...

    View full-size slide

  81. palkan_tula
    palkan
    76
    Refactor
    class Anyway::Env
    def fetch(prefix, **)
    + return if prefix.empty?
    @@ ...
    mock_context "Anyway::Env" do
    before do
    # ...
    allow(env_double)
    .to receive(:fetch)
    .with("", any_args)
    .and_return(
    Anyway::Env::Parsed.new({}, nil))
    end
    end

    View full-size slide

  82. palkan_tula
    palkan
    77
    env.rbs
    module Anyway
    class Env
    def fetch: (
    ::String,
    ?include_trace: (::FalseClass | ::TrueClass)
    ) -> ::Anyway::Env::Parsed?
    end
    end
    We cannot specify which
    string causes the return
    value to be nil

    View full-size slide

  83. palkan_tula
    palkan
    Double contract
    — Stubs represent contracts
    — Tests using real objects MUST verify
    contracts (unit, end-to-end)
    78

    View full-size slide

  84. palkan_tula
    palkan
    79
    github.com/psyho/bogus
    github.com/robwold/compact

    View full-size slide

  85. palkan_tula
    palkan
    Stubs → Contracts
    — Collect method stub expected arguments
    — Check that a real call with the matching
    arguments was made and its return type
    matches the mocked one
    80

    View full-size slide

  86. palkan_tula
    palkan
    81
    Stubs → Contracts
    allow(env_double).to receive(:fetch)
    .with("", any_args)
    .and_return(Anyway::Env::Parsed.new({}))
    Anyway::Env#fetch: ("", _) -> Anyway::Env::Parsed
    verification pattern

    View full-size slide

  87. $ bundle exec rspec --tags env
    Mocks contract verifications are missing:
    No matching call found for:
    Anyway::Env#fetch: ("", _) -> Anyway::Env::Parsed
    Captured calls:
    ("", _) -> NilClass
    82

    View full-size slide

  88. palkan_tula
    palkan
    83
    Mocked objects
    finder
    Mocked calls
    collector
    Real calls
    collector
    Type checker
    Types generator
    Call patterns
    verifier
    after(:suite)
    before(:suite) run time

    View full-size slide

  89. Are we there
    yet?
    84

    View full-size slide

  90. palkan_tula
    palkan
    Limitations
    — TracePoint could affect performance
    (~30 50% overhead)
    85

    View full-size slide

  91. palkan_tula
    palkan
    TP alternatives
    — Module#prepend
    — Source rewriting (Ruby Next ⏩)
    86

    View full-size slide

  92. palkan_tula
    palkan
    Limitations
    — TracePoint could affect performance
    — Parallel builds support is tricky
    — Verification patterns for non-value objects
    87

    View full-size slide

  93. To seam or not
    to seam?
    88

    View full-size slide

  94. palkan_tula
    palkan
    “Slow and reliable tests are in general
    much better than fast tests that break
    without reason (false positives) and don't
    catch actual breakage (false negatives).”
    –Jeremy Evans, Polished Ruby programming
    89

    View full-size slide

  95. palkan_tula
    palkan
    “If you track your test coverage, try for
    100% coverage before integrations
    tests. Then keep writing integration
    tests until you sleep well at night.”
    –Active Interactor's Readme
    90

    View full-size slide

  96. palkan_tula
    palkan
    Keep it real
    — Integration tests are the best seams
    91

    View full-size slide

  97. palkan_tula
    palkan
    Keep it close to real
    — Integration tests are the best seams
    — Know your doubles
    92

    View full-size slide

  98. 93
    $ stree search ~/dev/double_query.txt spec/**/*.rb
    spec/broadcast/redis_spec.rb:15 4: allow(Redis).to receive(:new) { redis_conn }
    spec/broadcast/redis_spec.rb:38 6: allow(redis_conn).to receive(:publish)
    spec/broadcast/nats_spec.rb:14 4: allow(NATS::Client).to receive(:new) { nats_conn }
    spec/broadcast/redis_spec.rb:52 6: allow(redis_conn).to receive(:publish)
    spec/broadcast/nats_spec.rb:15 4: allow(nats_conn).to receive(:connect)
    spec/broadcast/http_spec.rb:48 6: allow(AnyCable.logger).to receive(:error)
    spec/anycable_spec.rb:77 16: adapter = double("adapter", broadcast: nil)
    spec/broadcast/http_spec.rb:86 8: allow(adapter).to receive(:sleep)
    spec/broadcast/nats_spec.rb:38 6: allow(nats_conn).to receive(:publish)
    spec/broadcast/http_spec.rb:87 8: allow(AnyCable.logger).to receive(:error)

    View full-size slide

  99. palkan_tula
    palkan
    94
    Syntax Tree Search
    CallNode[
    receiver: NilClass,
    message: Ident[value: "double" | "instance_double"],
    arguments: ArgParen[
    arguments: Args[
    parts: [StringLiteral | VarRef[value: Const], BareAssocHash]
    ]
    ]
    ] |
    Command[
    message: Ident[value: "double" | "instance_double"],
    arguments: Args[
    parts: [StringLiteral | VarRef[value: Const], BareAssocHash]
    ]
    ] |
    # ...
    bit.ly/stree-doubles

    View full-size slide

  100. palkan_tula
    palkan
    Keep it close to real
    95
    — Integration tests are the best seams
    — Know your doubles
    — Fixturize your doubles
    — Embrace types

    View full-size slide

  101. palkan_tula
    palkan
    96
    Mock Suey
    github.com/test-prof/mock-suey
    gem "mock-suey"

    View full-size slide

  102. Thanks!
    @palkan
    @palkan_tula
    evilmartians.com
    @evilmartians

    View full-size slide