Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Contributor Leaderboards to Incentivize Good Co...

Bitergia
September 02, 2022

Contributor Leaderboards to Incentivize Good Community Citizenship

It takes a village to build open source software, or really a community of people with many different skills. Leaderboards can help us show the top-performing contributors and celebrate them, but they are double-edge swords in a community's toolbox. Tracking contributions can introduce an element of gamification, especially when contributors gain privileges in a community based on their level of contributions. As a positive outcome, leaderboards can increase participation and contributions which hopefully advances the community.

Bitergia

September 02, 2022
Tweet

More Decks by Bitergia

Other Decks in Technology

Transcript

  1. OpenJS World June 6-10, 2022 Incentivizing Good Community Citizenship with

    Contributor Leaderboards Georg Link, Director of Sales at Bitergia
  2. What are Leaderboards? : a large board for displaying the

    ranking of the leaders in a competitive event (such as a golf tournament) Definition of leaderboard @GeorgLink
  3. image goes here Why do we use Leaderboards? • Give

    instant feedback for contributors • Create sense of community • Recognize contributors • Reward activity • Create competition • Incentivize more activity @GeorgLink
  4. How do Leaderboards work? • The Self-Determination Theory ◦ The

    need for competence ◦ The need for autonomy ◦ The need for social relatedness • Goal-Setting Theory of Motivation ◦ Goal specific leaderboards work better • Social Comparison Theory ◦ Understanding of your own abilities and reactions ◦ Facilitating social interaction @GeorgLink
  5. Motivational issues with Leaderboards If competitors were not of equal

    ability: Less skilled competitors became less motivated as the task seemed unreachable Highly skilled competitors found the activity too easy, eventually making them less motivated too Landers (2015) @GeorgLink
  6. How a Leaderboard Ranks People? • Score ◦ Progress towards

    a success measure ◦ Embedded criteria for what counts ◦ Bias towards measurable things @GeorgLink
  7. Metrics A method of measuring something, or the results obtained

    from this: project data metric method @GeorgLink
  8. Metrics • Created accidentally • Contributions and their metadata •

    Contains personal identifiable information (PII) like names and emails Project Data as Trace Data @GeorgLink
  9. Where is the data? Where is the community? Metrics Git

    Wiki GitHub Discourse GitLab Mailing List BitBucket IRC Jira Slack Gerrit Meetup.com Confluence StackOverflow @GeorgLink
  10. Raw data: • Get data from data sources Enriched data:

    • Unify data • Manage identities • Calculate metrics Useful data: • Visualize, report How to get the data @GeorgLink
  11. Unify data • Date formats • Level of detail •

    Metadata about different contributions • Convert everything into the desired database structure Manage identities Calculate metrics Enriched data @GeorgLink
  12. Unify data Manage identities • Who is who in the

    community • Who do contributors work for (now and before) • Different usernames and email • Assigning contributions to the correct person Calculate metrics Enriched data @GeorgLink
  13. Unify data Manage identities Calculate metrics • Primary metrics -

    summarizing original data • Secondary metrics ◦ Calculation from different data fields ◦ Combining data from different data sources ◦ Value judgements on data - e.g., quality models Enriched data @GeorgLink
  14. Useful data Who is the data user? How should the

    data be presented? @GeorgLink
  15. Useful data Who is the data user? How should the

    data be presented? What visualizations are most meaningful? @GeorgLink
  16. Useful data Who is the data user? How should the

    data be presented? What visualizations are most meaningful? What story does the data tell? @GeorgLink
  17. • Cauldron.io • CHAOSS GrimoireLab • CHAOSS Augur • Apache

    Kibble • CNCF Dev Analytics OSS Solutions @GeorgLink
  18. image goes here About CHAOSS Short for: Community Health Analytics

    Open Source Software Linux Foundation project, cofounded by Bitergia Started in 2017 Focused on “creating analytics and metrics to help define community health” -- https://chaoss.community @GeorgLink
  19. image goes here Analytics Best Practices Follow Goal-Question-Metric approach Use

    metrics to tell a story Evaluate the usefulness of metric strategy Minimize gaming of metrics - careful how leaderboards score Start small and “get off zero” @GeorgLink
  20. image goes here Leaderboards Best Practices • Be transparent about

    scoring algorithm • Revise scoring algorithm with community feedback • Continuously update leaderboard (automate) • Many leaderboards for different categories • Recognize diverse types of contributions • Contextualize by showing immediate neighbors • Reset leaderboard or ignore older contributions @GeorgLink