Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Regression test for RuboCop

Avatar for pocke pocke
December 14, 2019

Regression test for RuboCop

Avatar for pocke

pocke

December 14, 2019
Tweet

More Decks by pocke

Other Decks in Programming

Transcript

  1. pocke.display • Masataka “Pocke” Kuwabara • Work for Bit Journey,

    Inc. We develop Kibela. • rurema team member • RuboCop core team member
  2. Kibela • Documentation SaaS ◦ Access https://kibe.la! • We use

    ◦ Ruby on Rails ◦ RuboCop ◦ GraphQL ◦ TypeScript ◦ React ◦ etc...
  3. RuboCop core team member • Many patches (recently a few

    patches) • The author of ◦ rubocop-rake ◦ rubocop-extention-generator ◦ mry ◦ gry ◦ rubocop-rubycw ◦ rubocop-typed ◦ rubocop-regression-test ← Today’s topic
  4. rurema team member • The Ruby Reference Manual project. ◦

    Work under https://github.com/rurema • I write documents and review pull requests.
  5. Rurema reviewing online meetup • We review rurema pull requests

    every Tuesday! ◦ Next meetup is 17th Dec. • Everyone can join the meetup! ◦ connpass: https://rurema-review.connpass.com/event/159182 ◦ slack: https://ruby-jp.github.io/ #rurema channel
  6. Agenda • Why does RuboCop need the regression test? •

    What’s the regression test? • How to work. • How to contribute to RuboCop. • Future work.
  7. RuboCop breaks easily, with Ruby syntax • Ruby has too

    complex syntax. • Actually we do not understand Ruby syntax perfectly. • A cop, implemented by us, will overlooks edge cases of Ruby syntax. • Example: omitting parens
  8. RuboCop breaks easily, with the configuration • RuboCop has many

    options. You can configure RuboCop for your style. • But it introduces complexity. • Not default option is not maintained enough.
  9. Conclusion of this section • RuboCop breaks easily ◦ with

    Ruby syntax ◦ with configurations • The regression test is necessary to find bugs before releasing RuboCop.
  10. What’s the regression test? • In short, it executes RuboCop

    ◦ with all option combinations, and ◦ in real Ruby projects. • Check exceptions.
  11. Example • In project-a ◦ rubocop --config .rubocop.1.yml ◦ rubocop

    --config .rubocop.2.yml ◦ rubocop --config .rubocop.3.yml ◦ … • In project-b ◦ rubocop --config .rubocop.1.yml ◦ rubocop --config .rubocop.2.yml ◦ rubocop --config .rubocop.3.yml ◦ … • In project-c ◦ ...
  12. What does check the test? • Check raised exceptions. ◦

    Grep “An error occurred while ...” message. ◦ Do not check false positives.
  13. What’s the regression test? (page 14) • In short, it

    executes RuboCop ◦ with all option combinations, and ◦ in real Ruby projects. • Check exceptions.
  14. with all option combinations • Many cops has EnforcedStyle option

    to specify style. • The style candidates is available as SupportedStyles.
  15. Example of default configuration Style/StringLiterals: EnforcedStyle: single_quotes SupportedStyles: - single_quotes

    - double_quotes Layout/DotPosition: EnforcedStyle: leading SupportedStyles: - leading - trailing
  16. Example of all options combinations # .rubocop.1.yml Style/StringLiterals: EnforcedStyle: single_quotes

    Layout/DotPosition: EnforcedStyle: leading # .rubocop.2.yml Style/StringLiterals: EnforcedStyle: double_quotes Layout/DotPosition: EnforcedStyle: trailing
  17. It tests RuboCop in real Ruby projects • Unit test

    contains only a few code. ◦ It may overlook. • Real Ruby projects contain many codes. ◦ We can test cop with many patterns.
  18. It tests RuboCop in real Ruby projects • Now it

    tests with: ◦ tric/trick2013, 2015 and 2018 ◦ rubocop-hq/rubocop • In the future, I plan adding test with: ◦ ruby/spec (pocke/rubocop-regressiontest#16) ◦ ruby/ruby ◦ rails/rails ◦ gitlabhq/gitlabhq ◦ discourse/discourse
  19. How to contribute to RuboCop • We have two contributing

    ways. ◦ Contribute to RuboCop ◦ Contribute to the regression test
  20. Contribute to RuboCop • Find bugs with the test on

    your codebase ↓ If you find bags, you can report them to RuboCop!
  21. Execute the following command $ git clone https://github.com/pocke/rubocop-regression-t est $

    cd rubocop-regression-test # Clone repo from GitHub and run the test $ ruby main.rb check OWNER/REPO # You can specify local directory also $ ruby main.rb check path/to/your/project/dir
  22. Contribute to RuboCop Regression Test • You can add more

    repositories to the test. ◦ Add them to CircleCI configuration. ◦ If it is green on CI, I can merge it. • Work for TODOs ◦ See the next section, “Future work”.
  23. Test infinite loop in auto correction • Currently it ignores

    infinite loop in auto correction. ◦ Because it is too noisy. • But it also should detect and fix infinite loops.
  24. More test for --auto-correct • Auto correction should generate code

    that have same meaning if the cop is marked as “safe”. • In short, test `rubocop -a && rspec`.
  25. Test plugins • RuboCop has many plugins under the rubocop-hq

    organization. It should test them also. ◦ rubocop-rails ◦ rubocop-performance ◦ rubocop-rspec ◦ rubocop-rake ◦ and so on.