Testing 201, or: Great Expectations

Ruby developers are a testy bunch. Who else writes more tests that we do? Which makes sense! Our language allows us freedom, and the opportunity to abuse it.

Most of us learned to test informally, from existing projects or tutorials. Sadly, that often leads us to the false conclusion that is all about verifying that our code works. Would you believe that’s only the third most important reason to test?

Let’s take a non-dogmatic look at the less obvious roles for testing with some real examples. We can get so much more from testing.

Transcript

1. TESTING 201 OR: GREAT EXPECTATIONS JOE MASTEY, AUGUST 2020
2. LOTS OF RSPEC, FACTORY BOT, AND RAILS
3. EVERYONE IS DOING THEIR BEST
4. HEURISTICS, NOT RULES
5. PROLOGUE: THE TROUBLE WITH A SIMPLE TEST
6. class CreateShipmentHistoryReport attr_accessor :user, :shipments def initialize(user) @user = user @shipments = user.shipments end def process shipments.sort_by(&:created_at) data = shipments.map do |shipment| shipment.to_json.values + [ shipment.user.name, shipment.product.name ] end file = CSV.open(tmp_filename) do |csv| csv << shipments.first.to_json.keys + ['user', 'product'] data.map do |row| csv << row end end persist_csv_to_s3(file) notify_user(user.email) file end end
7. class CreateShipmentHistoryReport attr_accessor :user, :shipments def initialize(user) @user = user @shipments = user.shipments end
8. def process shipments.sort_by(&:created_at) data = shipments.map do |shipment| shipment.to_json.values + [ shipment.user.name, shipment.product.name ] end # continued below...
9. # ... process continued file = CSV.open(tmp_filename) do |csv| csv << shipments.first.to_json.keys + ['user', 'product'] data.map do |row| csv << row end end persist_csv_to_s3(file) notify_user(user.email) file end
10. class CreateShipmentHistoryReport attr_accessor :user, :shipments def initialize(user) @user = user @shipments = user.shipments end def process shipments.order('created_at desc') data = shipments.map do |shipment| shipment.to_json.values + [ shipment.user.name, shipment.product.name ] end file = CSV.open(tmp_filename) do |csv| csv << shipments.first.to_json.keys + ['user', 'product'] data.map do |row| csv << row end end persist_csv_to_s3(file) notify_user(user.email) file end end
11. describe CreateShipmentHistoryReport do describe "#process" do # ... it "generates an array of strings" do expect(csv_data.length).to eq(shipments.length) expect(csv_data).to all(be_an(Array)) end it "matches the expected output" do expect(csv_data).to eq(output_records) end end end
12. let(:subject) { described_class.new(user).process } let(:csv_data) { CSV.parse(subject) }
13. let(:user) { create(:user, :signup_complete) } let(:subscription) { create(:subscription) } before do user.subscriptions << subscription expect(user) .to receive(:shipments) .and_return(shipments) end
14. let(:shipments) do [ shipment_1, shipment_2, shipment_3, shipment_4 ] end
15. let!(:shipment_1) { create(:shipment, product: product_1, created_at: 2.days.ago) } let!(:shipment_2) { create(:shipment, product: product_2, created_at: 3.days.ago) } let!(:shipment_3) { create(:shipment, product: product_3, created_at: 4.days.ago) } let!(:shipment_4) { create(:shipment, product: product_4, created_at: 5.days.ago) }
16. let(:product_1) { create(:product) } let(:product_2) { create(:product) } let(:product_3) { create(:product) } let(:product_4) { create(:product) }
17. let(:output_records) do [ [shipment_4.created_on, shipment_4.completed_on, user.name, product_4.name], [shipment_3.created_on, shipment_3.completed_on, user.name, product_3.name], [shipment_2.created_on, shipment_2.completed_on, user.name, product_2.name], [shipment_1.created_on, shipment_1.completed_on, user.name, product_1.name], ] end
18. before do expect_any_instance_of(described_class) .to receive(:save_to_s3) .and_return(true) expect_any_instance_of(described_class) .to receive(:email_user) .and_return(true) end
19. it "matches the expected output" do expect(csv_data).to eq(output_records) end
20. describe CreateShipmentHistoryReport do describe "#process" do let(:user) { create(:user, :signup_complete) } let(:subscription) { create(:subscription) } let(:product_1) { create(:product) } let(:product_2) { create(:product) } let(:product_3) { create(:product) } let(:product_4) { create(:product) } let!(:shipment_1) { create(:shipment, product: product_1, created_at: 2.days.ago) } let!(:shipment_2) { create(:shipment, product: product_2, created_at: 3.days.ago) } let!(:shipment_3) { create(:shipment, product: product_3, created_at: 4.days.ago) } let!(:shipment_4) { create(:shipment, product: product_4, created_at: 5.days.ago) } let(:subject) { described_class.new(user).process } let(:csv_data) { CSV.parse(subject) } let(:shipments) do [ shipment_1, shipment_2, shipment_3, shipment_4 ] end let(:output_records) do [ [shipment_4.created_on, shipment_4.completed_on, user.name, product_4.name], [shipment_3.created_on, shipment_3.completed_on, user.name, product_3.name], [shipment_2.created_on, shipment_2.completed_on, user.name, product_2.name], [shipment_1.created_on, shipment_1.completed_on, user.name, product_1.name], ] end before do user.subscriptions << subscription expect(user) .to receive(:shipments) .and_return(shipments) expect_any_instance_of(described_class) .to receive(:save_to_s3) .and_return(true) expect_any_instance_of(described_class) .to receive(:email_user) .and_return(true) end it "generates an array of strings" do expect(csv_data.length).to eq(shipments.length) expect(csv_data).to all(be_an(Array)) end it "matches the expected output" do expect(csv_data).to eq(output_records) end end end
21. FOCUSING ON THE WRONG THINGS
22. THE 3 ROLES OF TESTING (IN ORDER)
23. 1. DESIGN FEEDBACK 2. DOCUMENTATION 3. VERIFICATION
24. TESTS AS DESIGN FEEDBACK
25. A NOTE ON TDD
26. CODE THAT’S DIFFICULT TO TEST IS TRYING TO TELL YOU SOMETHING IMPORTANT
27. IT’S EASIER TO WORK WITH OBJECTS IN SMALL CHUNKS.
28. let(:csv_data) { CSV.parse(subject) } before do expect_any_instance_of(described_class) .to receive(:save_to_s3) .and_return(true) expect_any_instance_of(described_class) .to receive(:email_user) .and_return(true) end
29. def data(shipments: @shipments) shipments.sort_by!(&:created_at) shipments.map do |shipment| shipment.to_json.values + [ shipment.user.name, shipment.product.name ] end end def headers shipments.first.to_json.keys + ['user', 'product'] end
30. file = CSV.open(tmp_filename) do |csv| csv << headers data(shipments).map { |row| csv << row } end
31. it "matches the expected output" do subject = described_class.new(user) expect(subject.data).to eq(output_records) end
32. before do # expect_any_instance_of(described_class) # .to receive(:save_to_s3) # .and_return(true) # expect_any_instance_of(described_class) # .to receive(:email_user) # .and_return(true) end
33. before do expect(user) .to receive(:shipments) .and_return(shipments) end
34. class CreateShipmentHistoryReport def initialize(user, shipments = user.shipments) @user = user @shipments = shipments end end
35. before do # expect(user) # .to receive(:shipments) # .and_return(shipments) end described_class.new(user, shipments)
36. it "matches the expected output" do subject = described_class.new(user, shipments) expect(subject.data).to eq(output_records) end
37. def data(shipments) shipments.sort_by!(&:created_at) shipments.map do |shipment| shipment.to_json.values + [ shipment.user.name, shipment.product.name ] end end def headers shipments.first.to_json.keys + ['user', 'product'] end
38. def serialize(shipment) { created_on: shipment.created_on, completed_on: shipment.completed_on, user: shipment.user.name, product: shipment.product.name, } end
39. def data(shipments) shipments.sort_by!(&:created_at) shipments.map { |shipment| serialize(shipment) } end def headers serialize(shipments.first).keys end
40. describe "#serialize" do it "serializes some shipment and user data" do shipment = create(:shipment) report = described_class.new(user, []) result = report.serialize(shipment) expect(result).to eq({ created_on: shipment.created_on, completed_on: shipment.completed_on, user: shipment.user.name, product: shipment.product.name, }) end end
41. # let(:output_records) do # [ # [shipment_4.created_on, shipment_4.completed_on, # user.name, product_4.name], # [shipment_3.created_on, shipment_3.completed_on, # user.name, product_3.name], # [shipment_2.created_on, shipment_2.completed_on, # user.name, product_2.name], # [shipment_1.created_on, shipment_1.completed_on, # user.name, product_1.name], # ] # end
42. describe "ordering" do it "reorders shipments by their creation date" do report = described_class.new(user,[shipment_1, shipment_2]) result = report.data expect(result.map(&:first)).to eq( [shipment_2.created_on, shipment_1.created_on] ) end end
43. CREATE AS FEW RECORDS AS YOU CAN (AND ONLY RELATED ONES)
44. let(:user) { create(:user, :signup_complete) } let(:subscription) { create(:subscription) } before do user.subscriptions << subscription end
45. Factory AR records AR queries create(:user) 1 10 create(:meal) 5 41 create(:full_menu) 94 584 create(:weekly_basket) 104 644 create(:user, :with_order_history) 379 2336
46. let(:user) do instance_double(User, email: "test@test.com", shipments: shipments) end
47. let(:user) { User.new(email: "test@test.com", name: "Joe") }
48. describe "ordering" do it "reorders shipments by their creation date" do report = described_class.new(user, [shipment_1, shipment_2]) result = report.data expect(result.map(&:first)).to eq( [shipment_2.created_on, shipment_1.created_on] ) end end
49. PAY ATTENTION WHEN CLASSES DO TOO MUCH
50. def process def serialize(shipment) def data(shipments) def headers def save_to_s3(file) def notify_user(email)
51. TESTS AS DOCUMENTATION
52. OPTIMIZE TESTS FOR HUMAN UNDERSTANDING
53. it "matches the expected output" do expect(csv_data).to eq(output_records) end
54. describe "ordering" do it "reorders shipments by their creation date" do old = create_shipment(date: 9.days.ago) new = create_shipment(date: 2.days.ago) report = described_class.new(user, [new, old]) result = report.data expect(result.map(&:first)).to eq( [old.created_on, new.created_on] ) end end
55. it "serializes users as passed into the service" do users = create_list(:user, 3) response = subject.new(User.all).process response = response[0][4] expect(response).to eq(users.first.name) end
56. it "serializes users as passed into the service" do users = create_list(:user, 3) serialized_users = subject.new.serialize(users) response = serialized_users.first.full_name expect(response).to eq(users.first.name) end
57. PAY ATTENTION TO SIMILARITY AND DIFFERENCE
58. let(:product_1) { create(:product) } let(:product_2) { create(:product) } let(:product_3) { create(:product) } let(:product_4) { create(:product) }
59. let!(:shipment_1) { create(:shipment, product: product_1, created_at: 2.days.ago) } let!(:shipment_2) { create(:shipment, product: product_2, created_at: 3.days.ago) } let!(:shipment_3) { create(:shipment, product: product_3, created_at: 4.days.ago) } let!(:shipment_4) { create(:shipment, product: product_4, created_at: 5.days.ago) }
60. let(:output_records) do [ [shipment_4.created_on, shipment_4.completed_on, user.name, product_4.name], [shipment_3.created_on, shipment_3.completed_on, user.name, product_3.name], [shipment_2.created_on, shipment_2.completed_on, user.name, product_2.name], [shipment_1.created_on, shipment_1.completed_on, user.name, product_1.name], ] end
61. def create_shipment(date:) product = create(:product) create(:shipment, product: product, created_at: date) end
62. describe "ordering" do it "reorders shipments by their creation date" do old = create_shipment(date: 9.days.ago) new = create_shipment(date: 2.days.ago) report = described_class.new(user, [new, old]) result = report.data expect(result.map(&:first)).to eq( [old.created_on, new.created_on] ) end end
63. # let(:product_1) { create(:product) } # let(:product_2) { create(:product) } # let(:product_3) { create(:product) } # let(:product_4) { create(:product) } # let!(:shipment_1) { create(:shipment, product: product_1, # created_at: 2.days.ago) } # let!(:shipment_2) { create(:shipment, product: product_2, # created_at: 3.days.ago) } # let!(:shipment_3) { create(:shipment, product: product_3, # created_at: 4.days.ago) } # let!(:shipment_4) { create(:shipment, product: product_4, # created_at: 5.days.ago) } # # let(:shipments) do # [ # shipment_1, # shipment_2, # shipment_3, # shipment_4 # ] # end # let(:output_records) do # [ # [shipment_4.created_on, shipment_4.completed_on, # user.name, product_4.name], # [shipment_3.created_on, shipment_3.completed_on, # user.name, product_3.name], # [shipment_2.created_on, shipment_2.completed_on, # user.name, product_2.name], # [shipment_1.created_on, shipment_1.completed_on, # user.name, product_1.name], # ] # end
64. TRY TO STICK TO A GIVEN/ WHEN/THEN STRUCTURE
65. describe "ordering" do it "reorders shipments by their creation date" do old = create_shipment(date: 9.days.ago) new = create_shipment(date: 2.days.ago) report = described_class.new(user, [new, old]) result = report.data expect(result.map(&:first)).to eq( [old.created_on, new.created_on] ) end end
66. describe "ordering" do it "reorders shipments by their creation date" do # Given old = create_shipment(date: 9.days.ago) new = create_shipment(date: 2.days.ago) # When report = described_class.new(user, [new, old]) result = report.data # Then expect(result.map(&:first)).to eq( [old.created_on, new.created_on] ) end end
67. it "reorders shipments by their creation date" do old = create_shipment(date: 9.days.ago) new = create_shipment(date: 2.days.ago) report = described_class.new(user, [new, old]) expect(report.data.map(&:first)).to eq([old, new]) end
68. before do expect_any_instance_of(described_class) .to receive(:save_to_s3) .and_return(true) expect_any_instance_of(described_class) .to receive(:email_user) .and_return(true) end
69. USING G/W/T EMPHASIZES SAMENESS AND DIFFERENCE.
70. it "returns order units summed, divided by units per shipper" do create_store_order(count_units: 4) create_store_order(count_units: 8) subject = described_class.new(store_orders: store_orders) total_shippers = subject.total_shipper_count expect(total_shippers).to eq(3) end
71. it "returns a ceiling rounded value" do create_store_order(count_units: 7) subject = described_class.new(store_orders: store_orders) total_shippers = subject.total_shipper_count expect(total_shippers).to eq(2) end
72. let!(:shipment_1) { create(:shipment, product: product_1, created_at: 2.days.ago) } let!(:shipment_2) { create(:shipment, product: product_2, created_at: 3.days.ago) } let!(:shipment_3) { create(:shipment, product: product_3, created_at: 4.days.ago) } let!(:shipment_4) { create(:shipment, product: product_4, created_at: 5.days.ago) }
73. TESTS DON’T NEED TO BE (TOO) DRY
74. it "returns order units summed, divided by units per shipper" do create_store_order(count_units: 4) create_store_order(count_units: 8) subject = described_class.new(store_orders: store_orders) total_shippers = subject.total_shipper_count expect(total_shippers).to eq(3) end it "returns a ceiling rounded value" do create_store_order(count_units: 7) subject = described_class.new(store_orders: store_orders) total_shippers = subject.total_shipper_count expect(total_shippers).to eq(2) end
75. describe "#total_shipper_count" do subject { described_class.new(store_orders: StoreOrder.all) } context "even division" do let!(:order1) { create_store_order(count_units: 4) } let!(:order2) { create_store_order(count_units: 8) } let!(:expected_total) { 3 } # ... lots of other tests ... it "returns order units summed, divided by units per shipper" do expect(subject.total_shipper_count).to eq(expected_total) end end end
76. context "rounding" do let!(:order) { create_store_order(count_units: 7) } let!(:expected_total) { 2 } # ... lots of other tests ... it "returns order units summed, divided by units per shipper" do expect(subject.total_shipper_count).to eq(expected_total) end end
77. IF YOUR TESTS ARE SUPER DUPLICATIVE, THAT’S ACTUALLY DESIGN FEEDBACK
78. SAY WHAT YOU MEAN, LITERALLY
79. def user_summary(user) "(#{user.id}) #{user.full_name}, #{user.role}" end
80. it "returns a user summary for printing" do user = User.new(id: 5, full_name: "Dave G", role: :admin) expected = "(#{user.id}) #{user.full_name}, #{user.role}" summary = user_summary(user) expect(summary).to eq(expected) end
81. it "returns a user summary for printing" do user = User.new(id: 5, full_name: "Dave G", role: :admin) summary = user_summary(user) expect(summary).to eq("(5) Dave G, admin") end
82. LEVERAGE YOUR TOOLS TO HELP GUIDE READERS
83. describe "#process" do it “has a correct return value" end describe "#process" do it "returns the number of records correctly persisted" end
84. describe CreateShipmentHistoryReport do describe "#sorted_shipments" do it "returns an empty array when there are no shipments" end end
85. rspec ./spec/services/create_shipment_history_report_spec.rb --format=‘documentation’ CreateShipmentHistoryReport#sorted_shipments returns an empty array when there are no shipments
86. it “picks randomly, but correctly” do double1 = double double2 = double result = [double1, double2].sample expect(result).to eq(double1) end # expected: #<Double (anonymous)> # got: #<Double (anonymous)>
87. it “picks randomly, but correctly” do double1 = double("first record") double2 = double("second record”) result = [double1, double2].sample expect(result).to eq(double1) end # expected: #<Double "first record"> # got: #<Double "second record">
88. it "checks result size" do arr = [1,2,3] response = arr.length expect(response > 3).to be_true end # expected true # got false
89. it "checks result size" do arr = [1,2,3] response = arr.length expect(response).to be > 3 end # expected: > 3 # got: 3
90. it "checks validity of a record" do thing = double(valid?: true) expect(thing.valid?).to be_false end # expected: false # got: true
91. it "checks validity of a record" do thing = double("order", valid?: true) expect(thing).not_to be_valid end # expected `#<Double "order">.valid?` to return false, got true
92. TESTS AS VERIFICATION
93. TEST WHAT’S COMPLICATED, RISKY, OR CHEAP
94. YOU CAN’T HAVE 100% COVERAGE
95. DON’T TEST WHAT YOU DON’T OWN
96. class JobLog < ActiveRecord::Base validate :status, presence: true end it { should validate_presence_of(:status) }
97. class JobLog < ActiveRecord::Base validate :failure_message, presence: true, if: :failed? end it "requires failure_message for failures” do job = JobLog.new(status: :failed) expect(job).to validate_presence_of(:failure_message) job = JobLog.new(status: :completed) expect(job).not_to validate_presence_of(:failure_message) end
98. DON’T BE AFRAID TO DELETE YOUR TESTS
99. it "generates an array of strings” do expect(csv_data.length).to eq(shipments.length) expect(csv_data).to all(be_an(Array)) end
100. BEWARE OF OVER-MOCKING
101. def user_summary(user) "(#{user.id}) #{user.full_name}, #{user.role}" end it "returns a user summary for printing, bad" do user = double(id: 5, full_name: "Dave G", role: :admin) response = user_summary(user) expect(response).to eq("(5) Dave G, admin") end
102. it "returns a user summary for printing, better" do user = instance_double(User, id: 5, full_name: "Dave G", role: :admin) response = user_summary(user) expect(response).to eq("(5) Dave G, admin") end
103. it "returns a user summary for printing, better" do user = User.new(id: 5, full_name: "Dave G", role: :admin) expect(user_summary(user)).to eq("(5) Dave G, admin") end
104. IN SUMMARY!
105. IT GETS EASIER WITH PRACTICE
106. 2. WRITE TESTS FOR HUMANS 3. VERIFICATION COMES LAST 1. USE TESTS TO DRIVE DESIGN
107. THANKS!