Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Functional Test Automation Reporting with Sneez...

Functional Test Automation Reporting with Sneeze & Pocket Change

Software development engineer Silas Ray gave this talk to the NYC Python meetup.

• Watch video from Silas's talk.
• Read more about the Sneeze and Pocket Change libraries on the Open blog.
Get the project code.

More Decks by The New York Times Developers

Other Decks in Technology

Transcript

  1. Hudson • Launch tests • Scheduling runs • Build process

    integration • Parsed xUnit reporting
  2. Poor Result Visibility • Job runs:test suites not 1:1 ◦

    parameterized jobs ◦ cancelled/incomplete runs
  3. Poor Result Visibility • Poor usability of dumped xUnit results

    ◦ confusing drill down ◦ flat text ◦ doesn’t capture all data ◦ not searchable ◦ poorly archived
  4. Poor Result Visibility • Results not responsive ◦ nothing until

    job completes ◦ cancelled job gives no output
  5. Sneeze + Pocket Change • Nose plugin - test state/logging

    > DB • Web frontend - browse/filter test cycles • Collect test metadata (environment, etc) • Workflow integration - JIRA • Future features ◦ Run from web ◦ Rich result analysis ◦ More integrations ◦ Multimedia logging (screenshots!)
  6. Examples! import logging, time log = logging.getLogger('repro_steps') def validate(operation, result,

    message): log.info('verifying {}'.format(operation)) assert result, message def test_abctastic(): '''Doing some math with a b and c.''' a, b, c = 1, 2, 3 log.info('a, b, c = 1, 2, 3') validate('a + b == c', a + b == c, 'expected {}, found {}'.format(a + b, c)) validate('c / b == 1.5', c / b == 1.5, 'expected 1.5, found {}'.format(c / b)) def test_sleepy(): '''A test that logs every .1 second for roughly a minute.''' for i in xrange(600): log.info('Message {}'.format(i)) time.sleep(.1) Some setup code