The Metric Talks or Not

at Ruby Kaigi 2013, day 3.

Transcript

1. The Metric Talks or Not RubyKaigi 2013 @irohiroki
3. Topics Background Various Metrics Real World Survey
4. Background
5. Do you code?
6. What kind of software do you code?
7. Does it evolve?
8. 3 types of system S-Type P-Type E-Type Calculator Ches player Most of systems
9. As an E-type system evolves its complexity increases unless work is done to maintain or reduce it. Lehman’s law
10. Too complex to evolve → Software decay
11. What makes system evolve New features Environment changes Technical findings → Press to evolve → Press to decay
12. Visualized Pressure sample
13. Technical Debt... ? sample
14. What makes complexity decrease sample ?
15. Paranoids? djb? What makes complexity decrease
16. Is he objective? or structured? Can he persuade your boss?
17. No djb in your team... ?
18. Metrics
19. Metrics are objective and structured
20. Metrics can be visualized
21. Metrics for Ruby
22. Category Static analysis Coverage analysis Statistical analysis
23. Curious Ninja Master Maid
24. What is static analysis? Analysis which evaluates a snapshot of source code without running it. Static analysis includes: - Cyclomatic complexity, - ABC metric, - Duplicate code detection, - many kind of checkers, - and very original scoring tool, Flog.
25. Cyclomatic complexity? What is that? Can you draw control flow graphs of any of your methods? A graph consists of nodes and edges, right? Cyclomatic complexty is calculated as e - n + 2 where e and n are the number of edges and nodes. so, the cyclomatic complexity of this is “2”.
26. OK. So, what is it good for? It suggests the number of tests the method ought to have. And of course, it represents how hard the method is to understand.
27. I see. Which tool do I use to get that cyclomatic complexity? There is Saikuro. The fork of metricfu is maintained on Github. Roodi, a source code checker, has a couple of check items based on cyclomatic complexity.
28. Huh. Is it really of any help? Cyclomatic complexity provides one point of view on how to examine code. The more views you consider, the more findings you get.
29. OK. What view does ABC metric provide? ABC stands for “Assign”, “Branch”, and “Condition”. These are the basic elements of imperative programming languages like Ruby. ABC metric indicates magnitude of code with the following formula: where a, b, and c are the number of assigns, branches, and conditions respectively.
30. Magnitude of code? Aren’t you happy with LoC? LoC, Lines of Code, depends on coding style too much. ABC metric is better than LoC.
31. OK. How do you use these magnitude values? Imagine a bloated method which fills your entire editor window. Bloated methods like that signal the beginning of software decay.
32. What is bloated? At what value should we fix the code? It’s up to development teams. Metrics provide a sign of issues, and only that. If you want a precise criteria, base it on past records. If you set a target too high, your team will get exhausted.
33. I see. Do we have any gem for ABC metric? Yes. The Metric_ABC gem calculates the magnitude of the ABC metric for each method in your code.
34. I’ve heard of Flog. Isn’t it better than ABC? Good point. Metric_ABC and Flog are similar. And Flog provides another point of view. It emphasizes how hard code is to test. e.g., it gives relatively high points to `eval’. It’s not a generic algorithm, but specific to Ruby.
35. Cyclomatic complexity, ABC metric, and Flog. Which one should I use? Let’s say you are hiring, how many perspectives do you need at job interviews? Like perspectives, a single tool can be easily cheated. Would you like to know more?
36. Sure. I guess there are many source code checkers out there. True. One of the most important checkers is duplicate code detector. Ruby has Flay for that.
37. So it detects copy-pastes to DRY up the code? Yes, it compares portions of code grammatically. That is, differences of variable names, literals, and other non-significant words are ignored when matching.
38. Sounds nice. What about other checkers? Reek, Cane, and Roodi are all similar tools. They checks things like long methods, assignments in conditionals, documentation, and many more.
39. So many tools! How can I make all this work!?
40. What about Metric_fu?
41. Visualization by Metric_fu
42. OK, enough for static analysis. But I think coverage tools are more popular than that. Maybe. SimpleCov is the de facto standard for Ruby 1.9+.
43. Is 100% coverage possible? It depends on the layer. For business logic, you can go for 100%. But what’s important is to be realistic. That is the same for both coverage and static metrics.
44. Last question: What’s statistical analysis? Statistical analysis predicts software defects based on the modification history. It is known that frequently changed code blocks are more likely to have bugs. Do you use any version control system like Git? Then you are ready!
45. Sweet. The name of the statistical analysis tool, please? There are a few of them. Churn provides method level analysis. Bugspots is an implementation of the bug prediction heuristic at Google. It uses only “fixing” commits, and works at the file level.
46. How should we use these predictions? Does your team perform code review? How long does it take? Is it efficient enough? What if you know hotspots prior to the review?
47. Thank you, Master... Maid. You are welcome.
48. Real World Survey
49. Samples Github 10 stars minimum 1,311 Repositories Cyclomatic complexity, ABC, Flog
50. Histogram of Cyclomatic Complexity
51. Histogram of ABC Metric
52. Histogram of Flog
53. Cyclomatic Complexity Worst 3 1. 118: FooBarWidget/rubyenterpriseedition RDoc::Fortran95parser#parse_program_or_module 2. 92: rubinius/rubinius REXML::XPathParser#expr 2. 92: jsn/rbot Irc::Client#process
54. What does that mean? Parser / Protocol handler Pile of conditions Big case block
55. ABC Metric Worst 3 1. 301: FooBarWidget/rubyenterpriseedition RDoc::Fortran95parser#parse_program_or_module 2. 291: jsn/rbot Irc::Bot#initialize 3. 277: edavis10/redmine Redmine::Export::PDF#issue_to_pdf
56. What does that mean? Unique Sequential Static
57. Flog Worst 3 1. 4103.6: evanfarrar/opensprints main#none 2. 3858.4: jamtur01/puppet_old main#none 3. 3813.7: lak/puppet-old main#none
58. What does that mean? Flog can measure top level scripts setup.rb Same as ABC metric
59. Conclusion Objective and structured Visualizable Worst score can be justified
60. Thank You. Presented by @irohiroki