Journée Respire - CI/CD at LIPN

Transcript

1. CI/CD at LIPN Jaime Arias arias@lipn.univ-paris13.fr Journée Respire April 1, 2025
2. Hello! I’m Jaime Arias • CNRS Research Engineer @ LIPN • Responsible of the development team @ LIPN • Member @ Software/Source Codes College • Ambassador @ Software Heritage You can find me at: ͐ arias@lipn.univ-paris13.fr Ì www.jaime-arias.fr Jaime Arias CI/CD at LIPN 2/42
3. La dev-team • Development team @ LIPN • Currently, 4 permanent members: ◦ 1 apprentice, 1 IE, 2 IR • We host interns, engineers, and apprentices throughout the year ◦ 2nd year BUT (6 students) ◦ 3rd year BUT (1 student) ◦ 3rd year bachelor (1 student) • Hackaton one day per week Jaime Arias CI/CD at LIPN 3/42
4. La dev-team • Development team @ LIPN • Currently, 4 permanent members: ◦ 1 apprentice, 1 IE, 2 IR • We host interns, engineers, and apprentices throughout the year ◦ 2nd year BUT (6 students) ◦ 3rd year BUT (1 student) ◦ 3rd year bachelor (1 student) • Hackaton one day per week • We code and learn a lot of things! Jaime Arias CI/CD at LIPN 3/42
5. Agile Methodology • Early and continuous delivery of valuable software. • Give to the team the environment and support needed. • Working software is the primary measure of progress. • … Source: Roger S. Pressman, Bruce Maxim. Software Engineering: A Practitioner’s Approach. McGraw Hill Jaime Arias CI/CD at LIPN 4/42
6. Table of contents 1. Fundamentals of CI/CD 2. GitLab 3. CI/CD Architecture @ LIPN 4. Development of the LIPN intranet 5. Gitlab Pages Jaime Arias CI/CD at LIPN 5/42
7. 1. Fundamentals of CI/CD Jaime Arias CI/CD at LIPN 6/42
8. Continuous Integration • It’s a software development practice • Each member of a team merges its changes into a codebase (e.g. SCM) regularly • Each of these integrations is verified by an automated build to detect integration errors (e.g. tests) as quickly as possible Source: Atlassian. Gitflow workflow Jaime Arias CI/CD at LIPN 7/42
9. Continuous Integration Benefits • Less time wasted in integration • Less bugs (they are also cumulative) • Enables refactoring for sustained productivity • Allows to maintain a release-ready mainline Jaime Arias CI/CD at LIPN 8/42
10. Continuous Delivery/Deployment • It’s a software development strategy where code changes are released (automatically) into the production environment. • Eliminates the gap between coding and customer value. Jaime Arias CI/CD at LIPN 9/42
11. Continuous Delivery/Deployment • Improved quality • Accelerated feedback loop • Better team collaboration • Faster time to market • Enhanced customer experience • Reduced costs Jaime Arias CI/CD at LIPN 10/42
12. 2. GitLab Jaime Arias CI/CD at LIPN 11/42
13. GitLab Jaime Arias CI/CD at LIPN 12/42
14. GitLab • Open Source code repository and collaborative software development platform. • It’s available as a community edition and a commercial edition. • It supports CI/CD pipelines (.gitlab-ci.yml). Jaime Arias CI/CD at LIPN 13/42
15. GitLab CI/CD Jaime Arias CI/CD at LIPN 14/42
16. GitLab CI/CD Jaime Arias CI/CD at LIPN 14/42
17. GitLab CI/CD Jaime Arias CI/CD at LIPN 14/42
18. GitLab CI/CD Jaime Arias CI/CD at LIPN 14/42
19. GitLab CI/CD Jaime Arias CI/CD at LIPN 14/42
20. GitLab CI/CD gitlab-ci.yml • To use GitLab CI/CD, the file .gitlab-ci.yml is placed at the root of your project. • It specifies the stages, jobs, and scripts to be executed during the CI/CD pipeline. Jaime Arias CI/CD at LIPN 15/42
21. GitLab CI/CD gitlab-ci.yml • To use GitLab CI/CD, the file .gitlab-ci.yml is placed at the root of your project. • It specifies the stages, jobs, and scripts to be executed during the CI/CD pipeline. • The runners are agents that run the jobs. They can run on physical machines or virtual instances. Jaime Arias CI/CD at LIPN 15/42
22. GitLab CI/CD gitlab-ci.yml • Pipelines are made up of stages and jobs: ◦ Stages define the order of execution: build, test, deploy, release, etc. ◦ Jobs specify the tasks to be performed in each stage, e.g. compile or test the code. Jaime Arias CI/CD at LIPN 16/42
23. GitLab CI/CD gitlab-ci.yml • Pipelines are made up of stages and jobs: ◦ Stages define the order of execution: build, test, deploy, release, etc. ◦ Jobs specify the tasks to be performed in each stage, e.g. compile or test the code. • Pipelines can be triggered by various events, like commits or merges, or can be on schedule. Jaime Arias CI/CD at LIPN 16/42
24. GitLab CI/CD gitlab-ci.yml • Use CI/CD variables to customize jobs by making values defined elsewhere accessible to jobs. • Use CI/CD component templates for common tasks and integrations. Jaime Arias CI/CD at LIPN 17/42
25. GitLab CI/CD gitlab-ci.yml 1 image: node:16 2 3 stages: 4 - test 5 - e2e 6 - build 7 - deploy 8 9 variables: 10 GIT_SUBMODULE_STRATEGY: recursive 11 12 include: 13 - template: Jobs/SAST.gitlab-ci.yml Jaime Arias CI/CD at LIPN 18/42
26. GitLab CI/CD gitlab-ci.yml 1 .only-tags: 2 variables: 3 NODE_ENV: "production" 4 IMAGE_VERSION: $CI_COMMIT_TAG 5 IMAGE_NAME: $ARTIFACT_NAME:$IMAGE_VERSION 6 rules: 7 - if: $CI_COMMIT_TAG 8 when: manual Jaime Arias CI/CD at LIPN 19/42
27. GitLab CI/CD gitlab-ci.yml 1 unit_tests: 2 stage: test 3 extends: .tests_setup 4 script: 5 - yarn tests 6 cache: 7 key: 8 files: 9 - yarn.lock 10 paths: 11 - .yarn-cache/ Jaime Arias CI/CD at LIPN 20/42
28. GitLab CI/CD gitlab-ci.yml 1 e2e_tests: 2 stage: e2e 3 image: cypress/base:16.0.0 4 services: 5 - name: mongo:latest 6 extends: .tests_setup 7 script: 8 - yarn build 9 - yarn start & yarn wait-on http://localhost:3000 10 - yarn tests:e2e 11 artifacts: 12 when: always 13 paths: 14 - tests/e2e/screenshots/ 15 - tests/e2e/videos/ 16 expire_in: 1 week Jaime Arias CI/CD at LIPN 21/42
29. GitLab CI/CD gitlab-ci.yml 1 build_image: 2 extends: .only-tags 3 stage: build 4 image: 5 name: himito/docker-publisher 6 entrypoint: [""] 7 script: 8 - sh /kaniko/publish_to_dockerhub.sh Dockerfile $CI_PROJECT_DIR $ARTIFACT_NAME $IMAGE_VERSION $CI_REGISTRY_AUTH 9 retry: 2 Jaime Arias CI/CD at LIPN 22/42
30. GitLab CI/CD gitlab-ci.yml 1 deploy: 2 image: alpine 3 stage: deploy 4 extends: .only-tags 5 variables: 6 USERNAME: "deployer" 7 CONFIG_FILE: "docker-compose.cosyverif.yml" 8 before_script: 9 - apk add --no-cache openssh rsync bash 10 - eval $(ssh-agent -s) 11 - ssh-keyscan $PRODUCTION_SERVER >> ~/.ssh/known_hosts 12 script: 13 - scp docker-compose.prod.yml "$USERNAME@$PRODUCTION_SERVER:${ COSYVERIF_PATH}/${CONFIG_FILE}" 14 - docker-compose -f ${CONFIG_FILE} up --detach --no-deps ${ SERVICE_NAME} 15 after_script: 16 - ssh-agent -k Jaime Arias CI/CD at LIPN 23/42
31. 3. CI/CD Architecture @ LIPN Jaime Arias CI/CD at LIPN 24/42
32. CI/CD Architecture @ LIPN Jaime Arias CI/CD at LIPN 25/42
33. CI/CD Architecture @ LIPN Jaime Arias CI/CD at LIPN 25/42
34. CI/CD Architecture @ LIPN Jaime Arias CI/CD at LIPN 25/42
35. 4. Development of the LIPN intranet Jaime Arias CI/CD at LIPN 26/42
36. Development of the LIPN intranet Microservice Architecture USER LDAP PUBLICATION DATABASE PUBLICATION SERVICE ARXIV ADAPTER HAL ADAPTER REST API USER SERVICE REST API AUTHENTICATION SERVICE REST API API GATEWAY ROOM BOOKING SERVICE REST API BOOKING DATABASE LOAN OBJECTS SERVICE REST API LOAN DATABASE SVN SERVICE REST API Notes: - All the micro-services communicate with the “User Service”. OFFICE SERVICE REST API OFFICE DATABASE PURCHASE SERVICE REST API PURCHASE DATABASE Jaime Arias CI/CD at LIPN 27/42
37. Development of the LIPN intranet Source Code Management (SCM) Jaime Arias CI/CD at LIPN 28/42
38. Development of the LIPN intranet CI/CD Pipeline Jaime Arias CI/CD at LIPN 29/42
39. Development of the LIPN intranet CI/CD Pipeline Jaime Arias CI/CD at LIPN 29/42
40. Development of the LIPN intranet CI/CD Pipeline Jaime Arias CI/CD at LIPN 29/42
41. Development of the LIPN intranet CI/CD Pipeline Jaime Arias CI/CD at LIPN 30/42
42. Development of the LIPN intranet Code Review Jaime Arias CI/CD at LIPN 31/42
43. Development of the LIPN intranet Code Review Jaime Arias CI/CD at LIPN 31/42
44. Development of the LIPN intranet Test Automation Jaime Arias CI/CD at LIPN 32/42
45. Development of the LIPN intranet Test Automation Jaime Arias CI/CD at LIPN 32/42
46. Development of the LIPN intranet Build Automation Jaime Arias CI/CD at LIPN 33/42
47. Development of the LIPN intranet Dependency Update Automation Jaime Arias CI/CD at LIPN 34/42
48. Development of the LIPN intranet Build Automation Jaime Arias CI/CD at LIPN 35/42
49. Development of the LIPN intranet Notifications Jaime Arias CI/CD at LIPN 36/42
50. Development of the LIPN intranet Notifications Jaime Arias CI/CD at LIPN 36/42
51. Development of the LIPN intranet Staging Server Jaime Arias CI/CD at LIPN 37/42
52. Development of the LIPN intranet Production Server Jaime Arias CI/CD at LIPN 38/42
53. 5. Gitlab Pages Jaime Arias CI/CD at LIPN 39/42
54. Gitlab Pages Jaime Arias CI/CD at LIPN 40/42
55. Gitlab Pages 1 image: ruby:latest 2 3 variables: 4 JEKYLL_ENV: production 5 LC_ALL: C.UTF-8 6 7 before_script: 8 - gem install jekyll bundler 9 - bundle install 10 11 pages: 12 stage: deploy 13 script: 14 - bundle exec jekyll build -d public 15 artifacts: 16 paths: 17 - public 18 only: 19 - main Jaime Arias CI/CD at LIPN 41/42
56. Gitlab Pages URL Convention https://sites.lipn.univ-paris13.fr/cosyverif/documentation Project Group Jaime Arias CI/CD at LIPN 42/42
57. Thank you!
58. CI/CD at LIPN Jaime Arias arias@lipn.univ-paris13.fr Journée Respire April 1, 2025