Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Software Quality Metrics
Search
yunglin
June 27, 2012
1
89
Software Quality Metrics
yunglin
June 27, 2012
Tweet
Share
More Decks by yunglin
See All by yunglin
NoSQL 大腸花
yunglin
27
3.7k
Manage cloud server with open source tools
yunglin
1
110
Java Developer Day 2013 Scala Future API
yunglin
3
720
Java Developer Day 2012 Introduction to Actor Model
yunglin
1
120
Featured
See All Featured
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
27
4.3k
Gamification - CAS2011
davidbonilla
80
5k
Mobile First: as difficult as doing things right
swwweet
222
8.9k
Bash Introduction
62gerente
608
210k
[RailsConf 2023] Rails as a piece of cake
palkan
52
4.9k
It's Worth the Effort
3n
183
27k
A designer walks into a library…
pauljervisheath
204
24k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
131
33k
The MySQL Ecosystem @ GitHub 2015
samlambert
250
12k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
28
9.1k
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
0
110
Site-Speed That Sticks
csswizardry
0
28
Transcript
ழᐶࣖ ࡡᓃ߅Ҧ О͑
[email protected]
Ң݊ም • ࡡᓃ߅Ҧ௴፬ɛ
Ң݊ም • ࡡᓃ߅Ҧ௴፬ࠋபɛ
Ң݊ም • ࡡᓃ߅Ҧ௴፬ࠋபɛ • Server Side Developer for 8 years.
Ң݊ም • ࡡᓃ߅Ҧ௴፬ࠋபɛ • Server Side Developer for 8 years.
• QA Engineer @ Glu Mobile
Ң݊ም • ࡡᓃ߅Ҧ௴፬ࠋபɛ • Server Side Developer. • QA Engineer
@ Glu Mobile • SDET @ Real Networks
Ң݊ም • ࡡᓃ߅Ҧ௴፬ࠋபɛ • Server Side Developer for 8 years.
• QA Engineer @ Glu Mobile • SDET @ Real Networks • SDE @ Medio Systems.
TODAY’S TOPIC
Software Quality
Why Software Quality is Important?
When I am busy working on features
and finding customers
Because it is a feature
Because it is a strategic weapon
Trust me, I can build a data center for you
VMWare: We know our service.
software is all about trust.
Users need to trust the cloud won’t explode and bring
down their business
slashdot effect - overnight success turns into a disaster.
How to build trust?
Trust is faith, Faith is a religion. But engineering is
a science not religion.
None
Trust is Quality
What is Quality • Availability • Manageability • Performance •
Reliability • Scalability • Security
Availability • of your software • of your IT /
cloud infrastructure • of your network environment • of your external dependencies • of your client side network environment
Calculate Availability source: Economics of High Availability for Telecommunications Systems.
An Intel® Primer
Network Availability • 2001/02/09 ʕߕऎቊ୵ʵॎ • 2003/10/02 ʕߕऎቊ୵ʵॎ • 2006/12/26
ܩ݆ήቤ • 2009/08/13 זдჱࠬ • 2010/03/04 ৷ඪ̀͠ήቤ • 2011/11/14 ऎආБၪࡌʬ
Availability of External Dependencies
None
SOA in reality
Availability • Service Level Agreement • Scheduled down time •
Unscheduled down time
Manageability • Expose Information for System Admin to check the
condition of the online system. • Configurability • Monitorability
Manageability • Monitoring is must, every cloud Infrastructure fails •
Amazon • Azure • Google
None
None
None
Monitoring • Service credits are not automatics. • client has
to file a ticket immediately. • claim downtime of certain time period. • proof / log of outrage and area.
Performance • Response Time. • Throughput • Network Bandwidth
Reliability • Availability v.s. Reliability • Availability: system uptime •
Reliability: number of errors in a given time.
Reliability • Crash or become unresponsive occasionally. • Output is
inconsistent.
Scalability • Slashdot Effect. • Scale Up and Scale Out
• ERP system are much more sticky. • A mid-size website may have ~200 active sessions • But a ERP system used by a company of 200 employees would have 200 active sessions.
Security • Secure communication. • User Authorization and Authentication. •
Data Management. • Usage/Operation logging.
Trust is tangible evidences.
• Internal Audit •DEV Test •QA Process • External Audit
• ISO/27001 - security management standard. •
[email protected]
:)
Software Metrics
How do we know Service A can handle X amount
of traffic before we launch the Service?
Load Test
Load Tests • Performance Test • Stress Test • Longevity
Test
Performance Test • find • best case performance. • baseline
performance.
Performance Test • Key figures • request per minutes •
number of concurrent requests • number of machines • overall throughput
Performance Test • Ꮠ͜ό࿁ᏀɢٙˀᏐ݊νО • ஈଣඎ throughput ݊щึᎇഹ request ᅰᄣ̋Ͼᄣ̋d݊щί൴
ཀݔࡈᔷұᓃࣛdึක ˀᔷɨࠥ • ਂމ͊Ը൙П݊щცࠅᄣ̋ዚኜא٫݊ॴዚኜٙਿ࠽f • ཫПɪᇞ༈Դ͜εˇዚኜ ٙ࣬ኽf
Important Figures • minimum • maximum • mean • standard
deviation • 75 percentile • 95 percentile • 98 percentile • 99 percentile • 99.9 percentile
A Picture Worth A Thousand Words
None
Stress Test • Test how system perform under extreme situation.
• ie: iphone5 is on sales.
Longevity Test • evaluates a system's ability to handle a
constant, moderate work load for a long time. • find memory leak • find hidden routine that exhausts system resource.
Monitoring System Under Test • OS Level: CPU, Memory, Disc
I/O, and Network Usage • JVM: Memory Usage, Object counts, GC Cycles. • Application Level: request count, method call time, size of queue, cache hit/miss ratio
Load Test is the external view of the system.
we shall measure the code from developer’s point of view
too.
what we thought != reality
Shopping Cart • SQL Query • Page Rendering • Session
Replication
Shopping Cart • SQL Query • Page Rendering • Session
Replication It is slow!
Shopping Cart • SQL Query • Page Rendering • Session
Replication • Load test result: 300ms.
Shopping Cart • SQL Query • Page Rendering • Session
Replication we shall cache sql results!
Shopping Cart • SQL Query • Page Rendering • Session
Replication • 50 ms • 20 ms • 230 ms
We need to measure the code.
To make better decision.
•Obverse ᝈ࿀ •Orient ሜ •Decide Ӕഄ •Act Бਗ
• Perf4j • Java Simon • Google Guava Stopwatch •
Metrics by Coda Hale. TOOLS
Thank you • Any Questions?