Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Work Practices and Challenges in Pull-Based Dev...

Work Practices and Challenges in Pull-Based Development: the Integrator’s Perspective

Slides from my talk at ICSE 2015

Georgios Gousios

May 21, 2015
Tweet

More Decks by Georgios Gousios

Other Decks in Technology

Transcript

  1. Work Practices and Challenges in Pull-Based Development: the Integrator’s Perspective

    Georgios Gousios, Andy Zaidman, Margaret-Anne Storey, Arie van Deursen
  2. The pull-based development model Here are my changes Please fix

    those issues Here are my updates Looks great, thanks! contributor integrator changes integrated changes examined changes re- examined
  3. 45% of collaborative projects Projects with > 1 committers 55%

    use shared repository 45% use pull requests Gousios et al. ICSE14
  4. Large scale collaboration 2710 committers 309 code reviewers 9120 commenters

    657 collaborators 125 code reviewers 3904 commenters 875 committers 92 code reviewers 5200 commenters
  5. Too successful? “Lack of knowledge of git from contributors; most

    don’t know how to resolve a merge conflict.” “Sifting through the GitHub information flood to find what, if any, I should address.” “Dealing with loud and trigger-happy developers.”
  6. How we reached out to integrators Lifelines for 10% slowest

    pull reqs community participation in commits % comments (red) and commenters (blue) from the community http://ghtorrent.org/pullreq-perf/
  7. GHTorrent Final Survey Analysis / filtering (15 closed questions) Performance

    Reports 4 researchers Card sorting (7 open questions) 3,200 integrators (749 responses)
  8. GHTorrent Final Survey Analysis / filtering (15 closed questions) Performance

    Reports 4 researchers Card sorting (7 open questions) 3,200 integrators (749 responses) Results
  9. GHTorrent Final Survey Analysis / filtering (15 closed questions) Performance

    Reports 4 researchers Card sorting (7 open questions) 3,200 integrators (749 responses) Results 645 integrators
  10. How integrators use pull requests? Code reviews Issue fixes Soliciting

    contributions Discussing new features Work distribution Other 0 20 40 60 80 % of responses
  11. How integrators use pull requests? Code reviews Issue fixes Soliciting

    contributions Discussing new features Work distribution Other 0 20 40 60 80 % of responses
  12. How integrators use pull requests? Code reviews Issue fixes Soliciting

    contributions Discussing new features Work distribution Other 0 20 40 60 80 % of responses
  13. How projects do code reviews? 65%: Inline code comments 60%:

    Integration pipeline 55%: The community participates
  14. Factors for determining acceptance Code quality Code style Project fit

    Technical fit Testing Documentation Feature importance Code review result 0 5,5 11 16,5 22 % of responses
  15. Factors for determining acceptance Code quality Code style Project fit

    Technical fit Testing Documentation Feature importance Code review result 0 5,5 11 16,5 22 Quality Project Fit % of responses
  16. PR characteristics leading to merge PR hotness PR has tests

    PR track record PR churn PR # comments 0 50 100 150 200 Not or Mildly Important Quite or Very Imporant 100 0 100 % of responses
  17. - Following project's code style - Comments in code -

    Quality of the commit message - Readability of the code - Test coverage and their readability - Included documentation updates “Proper use of C++ language constructs. Knowledge and use of our dependent libraries […]” Views on quality
  18. Contribution quality perception Style conformance Test coverage of PR Code

    quality Code review Test/CI result Documentation Experience Author reputation Project conventions 0 3,25 6,5 9,75 13 % of responses
  19. Contribution quality perception Style conformance Test coverage of PR Code

    quality Code review Test/CI result Documentation Experience Author reputation Project conventions 0 3,25 6,5 9,75 13 % of responses Conformance
  20. Contribution quality perception Style conformance Test coverage of PR Code

    quality Code review Test/CI result Documentation Experience Author reputation Project conventions 0 3,25 6,5 9,75 13 % of responses Conformance Technical Excellence
  21. Contribution quality perception Style conformance Test coverage of PR Code

    quality Code review Test/CI result Documentation Experience Author reputation Project conventions 0 3,25 6,5 9,75 13 % of responses Conformance Technical Excellence
  22. Prioritisation factors “How important the issue is to the project.

    How serious the bug is. […]” Criticality
  23. Prioritisation factors Urgency Criticality “Whether the functionality is critical (bug

    fixes or very important feature blocking many users)[…]”
  24. Prioritisation factors Urgency Criticality Size/Complexity “If the merge is small,

    or affects a large issue, it'll be merged fast. If it's a feature request or large change it will take longer.”
  25. Prioritisation factors Urgency Mostly FIFO, but critical bug fixes or

    hot fixes get pushed to the top Age Criticality Size/Complexity
  26. Technical challenges impact assessment “Huge, unwieldy, complected bundles of ‘hey

    I added a LOT of features and fixes ALL AT ONCE!’ that are hell to review and that I’d like to *partially* reject if only the parts were in any way separable…” code base quality
  27. Technical challenges impact assessment contributor experience code base quality “Lack

    of knowledge of git from contributors; most don’t know how to resolve a merge conflict.”
  28. Technical challenges impact assessment “Sifting through the GitHub information flood

    to find what, if any, I should address” contributor experience code base quality volume
  29. Social challenges “Time is the biggest challenge -- I would

    love to have a more hierarchical structure to spread out the work; however, the community's development capacity is somewhat limited.” workload
  30. Social challenges workload responsiveness “People become non-responsive, never write tests,

    and never update style. They make a change once when they need their problem solved and disappear basically.”
  31. Social challenges explaining rejection workload responsiveness motivating contributors “On-boarding new

    contributors: helping them set up the development environment and learn enough Git/ Github to submit a pull request”