Quadratisch. Praktisch. Gut. K8s-native Qualitätssicherung mit Testkube @GermanTesting

Continuous Delivery ist allgegenwärtig. Wirklich? Viele Teams straucheln immer noch dabei regelmäßig gut getestete Produktinkremente zu liefern. Normalerweise mit der gleichen alten Ausrede: die (nicht)-funktionalen Tests seien zu aufwändig und zu teuer umzusetzen. Doch genau das Gegenteil ist der Fall! In diesem Tutorial gehen wir kurz auf die Bedeutung früher und regelmäßiger (nicht)-funktionale Tests ein und erläutern warum monolithische CI Pipelines eine Sackgasse sind. Anschließend zeigen wir, wie einfach es ist, kontinuierliche Integrations, Performance, Security und Akzeptanz-Tests mit Hilfe von Testkube direkt im Cluster auszuführen. @GermanTesting #cloudnativenerd

Transcript

1. qaware.de Quadratisch. Praktisch. Gut. K8s-native Quality Assurance mit Testkube Mario-Leander Reimer mario-leander.reimer@qaware.de @LeanderReimer #CloudNativeNerd @GermanTesting
2. 2 Mario-Leander Reimer Managing Director | CTO @LeanderReimer #cloudnativenerd #qaware #gernperDude
3. kubeshop/testkube qaware/hands-on-testkube
4. QAware | 4 Monolithic systems were relatively easy to test. ▪ No distribution, no IPC ▪ Homogene technology stack ▪ Low infrastructure complexity ▪ Managed infrastructure ▪ Long release and test cycles ▪ Developed by one team
5. QAware | 5 Microservice-based systems are complex. Testing them is even more complex. ▪ High distribution with various communication channels ▪ Heterogeneous Technology Stacks ▪ High infrastructure complexity with many components ▪ New operating model with more responsibility for the developers ▪ Short release cycles. Many teams.
6. Continuous Delivery is not quite so easy and a lot of work! QAware | 6 Continuous Delivery Low Risk Releases Less Rework Fast Time to Market Better Products Lower Costs Happier Teams Happier Users Loosely Coupled Architectures Maintainable Code Empowered Teams Continuous Security from Day 1 Test Automation Continuous Integration GitOps Deployment Automation Monitoring and Alerting
7. Holistic Software Product Quality is a lot of work! QAware | 7 Software Product Quality (ISO 25010) • Modularity • Reusability • Analysability • Modifiability • Testability Maintainability • Confidentiality • Integrity • Non-repudiation • Authenticity • Accountability Security • Adaptability • Installability • Replaceability Portability • Co-existence • Interoperability Compatibility • Maturity • Availability • Fault Tolerance • Recoverability Reliability • Time Behaviour • Resource Utilization • Capacity Efficiency • Completeness • Correctness • Appropriateness Functional Suitability • Operability • Learnability • UI Aesthetics • Accessibility Usability Safety Deployability
8. Monolithic, linear CI/CD pipelines are suboptimal and will result in delayed feedback and long release cycles. QAware | 8 Usually delayed until the end of sprint or the release. Which one first? Functionality vs. Performance vs. Security?
9. A microservice architecture with many downstream dependencies is complex and really hard to test. QAware | 9 Cluster Microservice A Microservice B Microservice C External System X External System Y Team A Team C Team B Unknown
10. Why not run (non)-functional tests against a cloud-native microservice architecture continuously, or triggered on the cluster itself?
11. Core Principles and Concepts of a Product Assembly Line QAware | 11 ▪ Separation of Concerns – Do one thing, and do one thing well. – Separate and explicitly model distinct development lifecycle phases: Build, CI, Deploy, E2E Test, … ▪ Choreography over Orchestration – Codified or configured downstream dependencies in the pipelines and triggers must be avoided. – Loose coupling. Tests and Suites do not need knowledge about components or environments – The product assembly line (PAL) is event-driven: time-based, commit based, deployment-based, release-based, on-demand ▪ GitOps Principles – All individual and E2E product tests as well as required components are GitOps managed. Auditable. Versioned. Easy Time Travel (e.g. revert to previous version) – Current Test BOM (Bill of Material) can be determined via individual state repositories ▪ Holistic Quality – We need a Single source of (quality) truth! – All relevant quality attributes are continuously aggregated and validated.
12. Conceptual PAL Architecture QAware | 12 Packages Package publish update Run deploy watch Deploy watch Dev GitOps Build push Checkout Build Test Quality Package Dev Test (E2E, NFA) trigger test Tests
13. Hello Testkube. Your friendly cloud-native testing framework for Kubernetes QAware | 13 ▪ Testkube natively integrates test orchestration and execution into Kubernetes and your CI/CD or GitOps pipeline ▪ Avoids vendor lock-in for test orchestration and execution in CI/CD pipelines ▪ Makes it possible to decouple test execution from build processes; test engineers should be able to run specific tests whenever needed ▪ Makes it easy to run any kind of tests - functional, load/performance, security, compliance, etc. in your clusters, without having to wrap them in docker-images or providing network access ▪ Provides a modular architecture for adding new types of tests and executors ▪ https://github.com/kubeshop/testkube
14. Demo Architecture and Testkube Concepts QAware | 14 default testkube Deployment Testkube Dashboard Webhook Receiver Testkube API Server CRDs CI/CD System Dev Executors Test Test Suite Deployment trigger flux-system run Mongo DB NATS Minio S3 CLI start store watch Test Trigger SUT Monitoring System Test Source
15. kubeshop/testkube qaware/hands-on-testkube
16. Conceptual PAL Architecture QAware | 16 Packages Package publish update Run deploy watch Deploy watch Dev GitOps Build push Checkout Build Test Quality Package Dev Test (E2E, NFA) trigger test Tests Quality Cockpit report PO report Build Status Code Quality Test Results fitness test promote ?
17. Architectural fitness functions continuously validate the (non)-functional requirements aka -illities of a system. QAware | 17 https://www.thoughtworks.com/de/radar/techniques/architectural-fitness-function
18. Some Architectural Fitness Function Examples QAware | 18 https://www.thoughtworks.com/de/insights/articles/fitness-function-driven-development describe "Performance" do it "completes a transaction in under 2 seconds" do expect(k6.check_transaction_round_trip_time()).to < 2 end it "has less than 10% error rate for 10000 transactions" do expect(k6.check_error_rate_for_transactions(10000)).to < .1 end end describe "Security" do describe “Static Analysis” do it "should not have plaintext secrets in codebase" do expect(code.has_secrets_in_codebase()).to_not be(true) end end describe “Dynamic Analysis” do it "should not have any of the OWASP Top 10" do expect(zap.has_owasp_top_10_vulnerabilities()).to be(false) end end end
19. qaware.de QAware GmbH Aschauer Straße 32 81549 München Tel. +49 89 232315-0 info@qaware.de twitter.com/qaware linkedin.com/company/qaware-gmbh xing.com/companies/qawaregmbh slideshare.net/qaware github.com/qaware Contact details ...