Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Software Quality Metrics

yunglin
June 27, 2012
88

Software Quality Metrics

yunglin

June 27, 2012
Tweet

Transcript

  1. Ң݊ም • ࡡᓃ߅Ҧ௴፬ࠋபɛ • Server Side Developer for 8 years.

    • QA Engineer @ Glu Mobile • SDET @ Real Networks • SDE @ Medio Systems.
  2. Availability • of your software • of your IT /

    cloud infrastructure • of your network environment • of your external dependencies • of your client side network environment
  3. Network Availability • 2001/02/09 ʕߕऎ᝙ቊ௡୵ʵॎ • 2003/10/02 ʕߕऎ᝙ቊ௡୵ʵॎ • 2006/12/26

    ܩ݆ήቤ • 2009/08/13 ୽זдჱࠬ • 2010/03/04 ৷ඪ̀͠ήቤ • 2011/11/14 ऎ᝙ආБၪࡌʬ඄
  4. Manageability • Expose Information for System Admin to check the

    condition of the online system. • Configurability • Monitorability
  5. Monitoring • Service credits are not automatics. • client has

    to file a ticket immediately. • claim downtime of certain time period. • proof / log of outrage and area.
  6. Scalability • Slashdot Effect. • Scale Up and Scale Out

    • ERP system are much more sticky. • A mid-size website may have ~200 active sessions • But a ERP system used by a company of 200 employees would have 200 active sessions.
  7. • Internal Audit •DEV Test •QA Process • External Audit

    • ISO/27001 - security management standard. • [email protected] :)
  8. How do we know Service A can handle X amount

    of traffic before we launch the Service?
  9. Performance Test • Key figures • request per minutes •

    number of concurrent requests • number of machines • overall throughput
  10. Performance Test • Ꮠ͜೻ό࿁ᏀɢٙˀᏐ݊νО • ஈଣඎ throughput ݊щึᎇഹ request ᅰᄣ̋Ͼᄣ̋d݊щί൴

    ཀݔࡈᔷұᓃࣛdึක֐ ˀᔷɨࠥ • ਂމ͊Ը൙П݊щცࠅᄣ̋ዚኜא٫݊؁ॴዚኜٙਿ๟࠽f • ཫПɪᇞ༈Դ͜εˇዚኜ ٙ࣬ኽf
  11. Important Figures • minimum • maximum • mean • standard

    deviation • 75 percentile • 95 percentile • 98 percentile • 99 percentile • 99.9 percentile
  12. Longevity Test • evaluates a system's ability to handle a

    constant, moderate work load for a long time. • find memory leak • find hidden routine that exhausts system resource.
  13. Monitoring System Under Test • OS Level: CPU, Memory, Disc

    I/O, and Network Usage • JVM: Memory Usage, Object counts, GC Cycles. • Application Level: request count, method call time, size of queue, cache hit/miss ratio
  14. Shopping Cart • SQL Query • Page Rendering • Session

    Replication • Load test result: 300ms.
  15. Shopping Cart • SQL Query • Page Rendering • Session

    Replication we shall cache sql results!
  16. Shopping Cart • SQL Query • Page Rendering • Session

    Replication • 50 ms • 20 ms • 230 ms