Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Reconstructing Taiwan's Internet-Based Party Politics by Ruby

Reconstructing Taiwan's Internet-Based Party Politics by Ruby

In an era where most activities in life can be executed online, the majority of countries still lack the ability to interact politically via the internet, such as online voting, party membership registration, and petition signing.

In Taiwan, a political party named “New Power Party” has, since its establishment in 2015, relied on volunteers to develop a complete system enabling online party membership, donations, and voting. After several years of developing and operating in a dynamic and evolving manner, the system accumulated substantial technical debt. This led to issues with proper and complete system functionality and maintenance, impacting the party’s operations. Eventually, we decided to re-evaluate the specifications and rebuild the entire system using the latest versions of Ruby and Rails.

In this session, I will explore the process of revamping an outdated Rails application, lacking in testing and specification documentation, with the new Ruby 3 and Rails 7. This will include code refactoring, selecting the Runtime Stack, and integrating automated deployment processes using Ruby-based tools. I will also share practical experiences and reflections on integrating data from old and new versions and relaunching the system.

Ryudo Awaru

April 16, 2024

More Decks by Ryudo Awaru

Other Decks in Programming


  1. Self Introduction • Mu-Fan Teng • Ruby Evangelist in Taiwan.

    • Organize most Ruby Community Events in Taiwan. • CEO of 5xruby.com • Twitter / GitHub: @ryudoawaru
  2. RubyConf Taiwan • 2010~ • One of Matz's "Major" Ruby

    Conference choices outside Japan • Featured Matz all the time after 2012
  3. Regular Ruby Meetups in Taiwan • Monthly Event @ PicCollage

    O ffi ce, Taipei. • Sharing and networking. • If you happen to be in Taiwan, please remember to join. • @rubytaiwan
  4. COSCUP 2024 Ruby Track 2024/8/3~4, Taipei • Community-led agenda track

    at COSCUP. • Scheduled for August 3-4 at National Taiwan University of Science and Technology, Taipei. • Call for proposals now open. • CFP open now at: https:// pretalx.coscup.org/ coscup-2024/
  5. AGENDA • Introduction of New Power Party and major political

    parties' internet participation in Taiwan. • Development history and issues of the legacy version. • Challenges and problems encountered when creating the new version.
  6. New Power Party(NPP, 時代 力 量) • Was one of

    the major political parties in Taiwan. • Known for its internet participation. • Maximum number of active party members was around 2,000.
  7. Party Joining Process • Fill in basic information. • Pay

    online or O ff l ine in local branches. • Become a member after Review & Approve by Headquarter Sta ff s.
  8. Membership Expiration Rules • Annual membership is similar to a

    subscription service. • Divided by fi rst and second half of the year. • Unpaid memberships will enter an expired state. • Membership can be restored by paying within two years. • After two years, it will change to a withdrawa state.
  9. NPP’s Distinctive Approach Fully Online Elections for Core Decision-Making Bodies

    • Decision Making Committee (DMC) • Serves as the primary operational decision-making body. • Elects the party chairperson from among its members. • Party Representative Committee (PRC) • A statutory body required by Taiwanese party law. • Convenes semi-annually for tasks like amending the party constitution and ratifying fi nancial reports.
  10. Legacy Version Development Status • Built with Ruby on Rails

    from day 1. • First version completed by volunteers. • Later taken over by a full-time Junior Engineer for development. • Took over by 2021 after the last engineer quits.
  11. Issues of Legacy Version—Project Management • No professional project management

    from the start. • Many features developed but rarely or never used. • Senior members unclear on the purpose of numerous functions. • Over-engineered features with unnecessary complexity and poor system design.
  12. Issues of Legacy Version—Data Integrity • Frequent errors due to

    data errors. • DB Migration cannot be executed normally.
  13. The History of Remaking Decision Making • Took over project

    maintenance in 2021, focusing on front-end and infrastructure. • End of 2021, faced challenges due to party charter amendments and dues calculation changes. • Spent considerable time on remediation and data adjustments. • Initiated project rebuild in Q3 of 2022, o ffi cially starting in October 2022.
  14. Application Stack Changes Legacy New Ruby 2.6 3.1 Rails 5.2

    7.0 Database PostgreSQL 9 PostgreSQL 14 Worker Sidekiq 5 + Redis GoodJob CSS Framework Bootstrap 3 TailwindCSS 3 + CSS Bundling JS Framework jQuery Stimulus JS + Importmap Deploy Tool Capistrano Docker Swarm
  15. Main Improvement Goals for the New Version • Normalize the

    database schema. • Retain only essential functions. • Isolate election functionality, enhancing security and reliability. • Achieve >80% unit test coverage, including feature tests for critical functions. • Improve DevOps practices with CI/CD automation. • Integrate data from the legacy system into the new system.
  16. Before class User def set_status if ["friend", "member_unchecked", "member_unchecked_paperworked", "member"].include?

    (self.classi fi cation.to_s) if con fi rmed_membership_dues.any?(&:lifelong?) && !self.vip? && self.member? self.vip! self.issue_membership_number if self.membership_number.blank? update_columns membership_expires_on: LIFELONG_MEMBERSHIP_EXPIRES_ON return self.membership_number end last_due = con fi rmed_membership_dues.last # …… end end
  17. After class Membership aasm column: :state do event(:pay) do before

    { |due| extend_by_due(due) } after_commit { |due| send_applied_mail(due) } transitions from: :applying_unpaid, to: :applying_paid transitions from: :legal, to: :legal transitions from: :expired, to: :legal end event(:cancel_applying, after_commit: :send_cancel_mailer) do transitions from: %i[applying_unpaid applying_paid], to: :applying_cancelled end end end class MembershipDue aasm column: :state do state :pending, initial: true state :paid event :pay, before: :before_pay, after: :after_pay, guard: :payable? do transitions(from: :pending, to: :paid) end end end
  18. Improvement Results: Lines of Code Model Before After Di f

    User 1,212 190 -1,022 Membership 0 225 225 MembershipDue 369 157 -212 Total 1,581 572 -1,009 *W/O Comment, Concerns Included.
  19. Issues with the Legacy Election Functions • Need to manually

    generate voter lists, noti fi cations, and ballots using rake tasks • Voting can be done just by accessing the ballot URL, although part of the ballot URL uses random numbers and alphabets, the identity of the voter cannot be ensured. • Ballot data is not encrypted, so risks cannot be ensured in case of data leakage.
  20. Election System Refinement Plan • Separate the election functionality as

    an independent website. • Ensure compatibility with both new and old website logins and data. • Complete the development within one month due to unexpected political schedule changes.
  21. OAuth Sign In • Use Doorkeeper Gem to establish OAuth

    Authorization Server on both legacy & new sites. • Respond with the same OAuth Raw Info through API.
  22. Member Data Integration • Synchronize member data from the o

    ffi cial website for voter roll creation. • Initially used database syncing from the legacy version to the election site’s Voter Model due to tight development schedules. • Adopt RESTful integration with the new o ffi cial website for subsequent updates.
  23. Encrypting Ballot Data • The ballot data stores the voter's

    member number. • Use ActiveRecord Encryption to encrypt the ballot data to prevent leakage of voting details.
  24. Automate All Batch Processing • Write ActiveJob to do the

    batch tasks. • Use ActiveJob's set_wait to schedule corresponding Jobs based on the creation time of the election data.
  25. The Test Coverage • Developed by 2 members in 1

    month. • Achieved 81% test coverage.
  26. Concrete Goals • When submitting a Pull Request: • Automatically

    scan and check the code for syntax and security issues. • Automated testing. • Automatically generate an executable environment for User Testing (Review Apps). • When a Pull Request is merged into the main branch: • Can automatically deploy to staging environment. • Can semi-automatically deploy the program to the production through CI/CD Job.
  27. Actions Needed to Achieve Goals • Dockerize the project. •

    Write CI/CD pipeline jobs to automate the work fl ow. • Choose an appropriate runtime stack.
  28. Gems For Dockerize The Project Quickly • boxing:Generate a Docker

    fi le for the project. • openbox:Generates the Entrypoint for Docker. • liveness:Generates a health check endpoint.
  29. boxing Gem • https://rubygems.org/gems/boxing • Multi-stage builds. • Use Alpine

    Linux to compact the image size. • DSL & Ruby way to set up Docker fi le. # con fi g/boxing.rb Boxing.con fi g do |c| c.revision = true c.sentry_release = true c.health_check = true c.build_packages = %w[gcompat] end
  30. openbox Gem • https://github.com/elct9620/openbox • Checks DB connection automatically before

    starting any command • Easy to extend new commands • Use the same entrypoint with di ff erent entrypoint commands # Docker fi le ENTRYPOINT ["bin/openbox"] # docker-compose.yml version: '3.8' x-application: &application image: "${IMAGE_NAME}:$ {IMAGE_TAG}" #... services: # Default entrypoint: 'bin/openbox server' application: <<: *application good_job: <<: *application command: good_job
  31. liveness Gem • The Rack middleware to provide health check

    endpoints • Con fi gurable by DSL • Easy to extend check provider and add endpoint security like token and IP whitelist # con fi g/routes.rb Rails.application.routes.draw do mount Liveness::Status => '/status' end # con fi g/initializers/liveness.rb Liveness.con fi g do |c| c.add :postgres, timeout: 10 c.add :redis, timeout: 10 end
  32. Docker Compose File version: '3.8' x-application: &application image: "${IMAGE_NAME}:${IMAGE_TAG}" environment:

    - AUTO_MIGRATION=yes services: good_job: <<: *application deploy: replicas: ${WORKER_REPLICAS:-1} command: good_job healthcheck: test: ["NONE"] application: <<: *application deploy: replicas: ${APPLICATION_REPLICAS:-1} labels: - trae fi k.enable=true - trae fi k.http.routers.${DEPLOY_NAME}-https.rule=Host(`${DEPLOY_DOMAIN}`) - trae fi k.http.routers.${DEPLOY_NAME}-https.entrypoints=https - trae fi k.http.services.${DEPLOY_NAME}.loadbalancer.server.port=3000 networks: - trae fi k-public
  33. Predefined CI/CD variables review:deploy: when: manual extends: .deploy stage: deploy

    environment: name: review/$CI_COMMIT_REF_NAME #prede fi ned variable on_stop: review:stop # Job name for stopping the app variables: # $CI_COMMIT_REF_NAME is one of prede fi ned vars DEPLOY_NAME: npp2023-$CI_ENVIRONMENT_SLUG tags: # Ensure Run on Docker Swarm - swarm-deploy only: - merge_requests needs: - docker - job: assets:precompile artifacts: true
  34. Use Templates to Templatize Repeated Configurations • https://github.com/elct9620/ruby-gitlab-ci • GitLab

    CI templates for Ruby and Rails projects. • Contains many examples for reference. • Supports deployment via Trae fi k and Docker Swarm Mode.
  35. Encapsulate Migration Code as a Rails Engine • The Migrator

    works for: 1. Migrate Database. 2. Migrate Carrierwave Attachment to Active Storage. • Developed as an independent Rails Engine Gem. gem 'legacy', git: 'https://gitlab.com/newpowerparty/legacy_migrator.git', branch: 'main'
  36. Migrator Implementation: Base Model class Legacy::ApplicationRecord < ActiveRecord::Base self.abstract_class =

    true establish_connection ENV['LEGACY_DATABASE_URL'] scope :scope2export, -> { all } def skip?; false; end # De fi ne conditions if do want this row to be converted def migrate; raise NoMethodError; end def migrate! migrate.save!(validate: self.class.const_get(:VALIDATE_ON_MIGRATING), callbacks: self.class.const_get(:SAVE_WITH_CALLBACKS)) end class << self def after_migration; return; end def migrate! ::ApplicationRecord.transaction do scope2export. fi nd_each do |lrec| unless lrec.skip? new_record = lrec.migrate! lrec.record_migration!(new_record) end end rescue StandardError => e logger.info e.inspect end after_migration
  37. Migrator Implementation: Attachments class Legacy::FileVault < ApplicationRecord mount_uploader : fi

    le, GeneralUploader scope :scope2export, -> { preload(:creator, :updator).all } def migrate @new_ fi le_vault = ::FileVault.new(id:, legacy_creator_name: creator&.name, legacy_updater_name: updator&.name) @new_ fi le_vault. fi le.attach(io: fi le.sanitized_ fi le. fi le, content_type: fi le.content_type, fi lename: fi le. fi le. fi lename ) @new_ fi le_vault end def skip? begin fi le.cache_stored_ fi le! # Skip the record if the fi le does not exist rescue Errno::ENOENT return true end false end
  38. Use Git Token When Building Image # Docker fi le

  39. On Launch Day • Migration took about 30 minutes •

    Including subsequent data veri fi cation, it took about 2 hours • Re-entering news and other CMS data took about 2 days • No data loss issues occurred