Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Alkemics CI & CD with Jenkins and Docker

Alkemics CI & CD with Jenkins and Docker

Presented by Florent Paulais & Guillaume Morin
SysadminDays #7 : https://sysadmindays.fr/

Renaud Chaput

October 19, 2017
Tweet

More Decks by Renaud Chaput

Other Decks in Technology

Transcript

  1. Summary • Our platform • QA at Alkemics • CI/CD

    workflow • Tests • Build chain • SQL schema changes • What we have learnt & next steps
  2. Alkemics platform • Automation ◦ No more excel files ◦

    Import/export system ◦ Data quality control ◦ Accelerate content updates • Network ◦ Connect to retailers, manufacturers and third parties ◦ France, aiming for the world
  3. Infrastructure • 29 micro services in python and go •

    Data storage ◦ Mysql ◦ Cassandra ◦ ElasticSearch ◦ Couchbase
  4. SRE • Sysadmin / Devops / QA • Scope ◦

    Infrastructure ◦ Production ◦ Tooling • Automation
  5. QA at Alkemics, a brief history 2015 • Single server

    2016 • Multiple docker instances • Browserstack 2017 • Multiple platforms with “Pillar”
  6. Jenkins definition • Pipeline job is a job described with

    a Jenkinsfile for each integration branch • Jenkinsfile ◦ Define a workflow to run tests and deployment in groovy ◦ Stored in service repository • Dedicated groovy lib to share functions between all Jenkinsfile
  7. Introducing Pillar • Python tool designed to ◦ Create complete

    environment from scratch ◦ Orchestrate docker instances ◦ Seed data according to pre made scenarios
  8. Pillar demo mysql: container_name: teste2e-mysql image: alkreg:5000/alkemics/mysql:staging service-application: command: bash

    -c "sed -i ..." container_name: teste2e-service-application image: alkreg:5000/alkemics/service-application:staging volumes: - service-application-logs:/var/log working_dir: /var/www/service-application
  9. Pillar demo core: accounts: retailer-admin: products: '03663215733878': assigned: - 'Alkemics

    Test - Manufacturer' user: permissions: 'ReadWrite': ['product.show', 'product.update']
  10. Build chain • Docker images stored and built on the

    registry • Same base image for all tests • Inheritance
  11. SQL schema changes • No manual ALTER • shmig ◦

    Included in the repository ◦ Run when deploying in any environment ◦ Periodic merge to speed up tests execution
  12. Results Average time to build and test Bugs identified by

    customers between 2016 and 2017 3 min 35 sec - 28 % “Pillar made my skin 51 % softer.” Youcef Mammar, Software Engineer, Alkemics
  13. What we have learnt • Building an existing platform from

    scratch is long and sometimes complex • External services are complicated to manage ◦ Use Mock when possible • Backward compatibility is mandatory
  14. Next steps • Scalability • Docker in production ◦ On

    demand instances • Environment for developers