CircleCI Meetup #3 Slides

Transcript

2. 3 Why 2.0 is fast? Core tools/software in 2.0 1 2 Dev Culture at CircleCI 3 New Features in 2.0 4 5 Agenda QA
3. 4 Why 2.0 is fast? 2.0 is blazing fast
4. 5 Basic Architecture API Billing Webhook Usage Job Dispatcher gRPC Server Workflows Nomad Docker Container Build Agent Test Results 1.0/2.0 Services 2.0 Services Git Push
5. 6 Build Agent • Core component of 2.0 • Written in Go • gRPC and Asynchronous Build Execution • Running on Docker
6. 7 • Support Multiple Languages • Message Streaming • Asynchronous RPC call • SSL/TSL communication • Many many many features…. No More SSH Asynchronous Build Execution With gRPC High Performance RPC framework
7. 8 Basic Architecture API Billing Webhook Usage Job Dispatcher gRPC Server Workflows Nomad Docker Container Build Agent Test Results 1.0/2.0 Services 2.0 Services Git Push
8. 9 Build Execution: 1.0 vs 2.0 1.0 2.0
9. 10 SSH is everywhere in 1.0 Host Machine Container Step 1: ssh ubuntu@container1 bundle install Step 1: Exit Code & Output Step 2: Exit Code & Output Step 2: ssh ubuntu@container1 rspec Synchronous Build Step Execution
10. 11 gRPC in 2.0 Asynchronous Build Step Execution gRPC Server Build Agent Compiled Build Steps: step 1, step 2 Step 1: Exit Code & Output Step 2: Exit Code & Output
11. 12 • Pre-install packages • No more apt-get install • Creative Caching Custom Image & Flexible Caching
12. 13 Custom Image and Caching 1.0 2.0
13. 14 Other improvements • Inference daemon is gone • Remove extra tasks when starting container • The performance of each step is mostly the same • Proper resource allocation
14. 15 Result: google/guava 1.0 17:43s https://circleci.com/gh/marcomorain/guava/38 2.0 7:41s https://circleci.com/gh/marcomorain/guava/48
15. 16 Core Tools/Software in 2.0 Tools/software we love in 2.0
16. 17 • Leading Linux Container Platform • Replacing LXC in CircleCI 2.0 • Production ready • Very active development • Docker is not slow! Leading Linux Container Platform Docker The leading container platform
17. 18 Basic Architecture API Billing Webhook Usage Job Dispatcher gRPC Server Workflows Nomad Docker Container Build Agent Test Results 1.0/2.0 Services 2.0 Services Git Push
18. 19 LXC vs Docker 1.0 (LXC) 2.0 (Docker)
19. 20 Things We Learned About Docker Performance Docker is not slow Active Development Always use the latest Kernel Container Isolation Cgroup is not perfect
20. 21 Things We Learned About Docker Active Development • Always use the latest kernel • Performance and stability issues with old kernels Container Isolation • Using cgroup doesn’t guarantee resource isolation • We are working on better I/O performance • https://sysdig.com/blog/container-isolation-gone-wrong/ Performance • Docker is not slow • I/O penalty in CoW is small • CircleCI DB images with optimizations
21. 22 Revenge of VMs • VMs are used for machine executor in 2.0 • VMs are getting cheaper • Also faster: 45 ~ 60 secs to spin up a VM in GCP • Full control over hardware resource
22. 24 • Simple Architecture • Still beta but reliable Job Scheduler in 2.0 Nomad Easy Deploy Applications At Any Scale
23. 25 Basic Architecture API Billing Webhook Usage Job Dispatcher gRPC Server Workflows Nomad Docker Container Build Agent Test Results 1.0/2.0 Services 2.0 Services Git Push
24. 26 Nomad: Job Scheduler in 2.0 • All builds are Nomad jobs • Simple architecture • Still pre-1.0 but stable • Sometimes backward compatibility is broken, though...
25. 27 • Backbone for microservices in 2.0 • Rolling updates of services Breaking monolith into microservices Kubernetes Production-Grade Container Orchestration
26. 28 Breaking Monolith into Microservices CircleCI Monolith Microservices Kubernetes
27. 29 Advantages and Disadvantages in Microservices Advantages Disadvantages
28. 33 New Features in 2.0 Don’t miss them!
29. 34 jobs: build: docker: - image:quay.io/project/image:tag auth: username: $QUAY_USERNAME password: $QUAY_PASSWORD Sample .circleci/config.yml Private Images • Support Private Repositories • DockerHub, GCR, ECR, Quay.io and more
30. 35 • Missing feature in 2.0 • Filter job execution by tag workflows: version: 2 build-workflow: jobs: - build: filters: tags: ignore: /^testing-.*/ Sample .circleci/config.yml Git Tag Support
31. 36 • Creating sets of env vars in the organization page • Share the env vars among projects workflows: version: 2 my-workflow: jobs: - run-tests: context: org-global Sample .circleci/config.yml Contexts
32. 37 • Running macOS builds in 2.0 infrastructure • Workflow support • We want beta customers!! jobs: build: macos: xcode: version: 8.3.3 Sample .circleci/config.yml macOS in 2.0 (beta)
33. 38 • Enterprise in 2.0 was just out! • Only supporting AWS at the moment 2.0 in Enterprise (AWS)
34. Thank you. 39