CDK and Kubernetes: A Match Made in the Cloud

Recording: https://youtu.be/I5SGpnv8BaE?t=2124

In this talk we delve into CDK8s, a versatile framework that defines Kubernetes applications and abstractions using familiar programming languages and object-oriented APIs. Our presentation will include practical examples and use cases.

We'll explore the numerous advantages of CDK8s, such as utilising strongly-typed data types to articulate the abstraction's API, enabling rich interactions through methods and properties, and fostering polymorphic programming with interfaces and base classes. CDK8s also facilitates the versioning, sharing, and testing of these models, known as "Constructs", using standard package managers and conventional testing tools.

Our focus is to address the limitations of prevalent Kubernetes configuration methods like Helm and Kustomize. We'll scrutinise common issues with Helm, particularly the challenges with gotemplates and YAML inputs, such as indentation errors, and the absence of clear schemas, which often lead to user frustration, upgrade issues, and deployment inaccuracies.

Furthermore, we'll share practical tips and strategies for effectively utilising CDK8s. This includes demonstrating 2 design patterns and highlights with examples as well as highlight some tips and tricks, learned while adopting CDK8s, ensuring a comprehensive understanding of this framework.

Transcript

1. CDK and Kubernetes: A Match Made in the Cloud K8SUG - Singapore
2. TABLE OF CONTENTS • Problem Statement • Introduction to CDK8s • Introduction to CDK8s+ • Demo (if time permits) • CDK8s Advanced • CDK8s vs …
3. Kubernetes YAML is complex. Helm? Kustomize? How about … - Testing? - Redistribution? - Extensibility and Customization? PROBLEM STATEMENT
4. Kubernetes YAML is complex. Helm? Kustomize? How about … - grafana.com/blog/2020/01/09/introducing-tanka-our-way-of-deploying-to-kubernetes - kcl-lang.io/blog/2023-05-30-vs-helm - learnk8s.io/templating-yaml-with-code (CUE lang) - blog.palark.com/dhall-language-for-kubernetes-manifests - kubevela.io/blog/2020/12/7/kubevela-the-extensible-app-platform-based-on-open-applicatio n-model-and-kubernetes - … PROBLEM STATEMENT
5. Introduction to CDK8s What is CDK8s and why does it exist?
6. Define Kubernetes applications and reusable abstractions using familiar programming languages and rich object-oriented APIs. cdk8s.io WHAT IS CDK8s How? CDK8s apps synth to dist/ folder apply to cluster(s) directly or through gitOps.
7. WHY CDK8s • Express the abstraction’s API using strong-typed data types • Express rich interactions with methods and properties • Create polymorphic programming models through interfaces and base classes
8. WHY CDK8s • Share them through regular package managers (no special Helm packaging, OCI, … , just use NPM) • Test them using our familiar testing tools and techniques (just use JestJS / SnapShot Testing, … no need special Helm testing) • Version them (use conventional commits and versioning with package.json / Auto manage dependency bumping using NPM ecosystem)
9. CDK8s CORE CONCEPTS • Constructs • Apps • Charts • ApiObjects
10. Basic building blocks which render down into representation for target framework - CDK → constructs render down to CFN definitions - CDKTF → constructs render down to HCL expressions - CDK8s → constructs render down to k8s Manifests (yaml/json) CONSTRUCTS
11. An app is a construct Entry/Root of the Tree of Constructs. Contains one or more Charts APPS Construct Chart App Chart Construct Construct Construct import { App } from "cdk8s"; const app = new App(); // do all the things … app.synth();
12. A Chart is a construct - Container for constructs Synthesizes to a single YAML manifest* Chart level namespace / labels will applied to all contained constructs CHARTS Construct MyChart App Construct import { App } from "cdk8s"; import { SampleChart } from "./SampleChart" ; const app = new App(); // register chart into app new SampleChart(app, "Sample", { portNumber: 9899, cpuAutoScaleThreshold: 60, }); app.synth(); enum YamlOutputType { /** All resources are output into a single YAML file */ FILE_PER_APP = 0, /** Resources are split into separate files by chart */ FILE_PER_CHART = 1, /** Each resource is output to its own file */ FILE_PER_RESOURCE = 2, /** Each chart in its own folder and each resource in its own file */ FOLDER_PER_CHART_FILE_PER_RESOURCE = 3 }
13. An ApiObject is a construct Represents an entry in a Kubernetes manifest (level 0). In most cases, you won’t use ApiObject directly but rather classes generated through cdk8s import and which extend ApiObject. API OBJECTS 5 // generated by cdk8s import { ApiObject, ApiObjectMetadata , GroupVersionKind } from 'cdk8s'; import { Construct } from 'constructs'; /** * DatadogAgent Deployment with Datadog Operator. * * @schema DatadogAgent */ export class DatadogAgent extends ApiObject {
14. CDK8s CORE
15. CDK8s CORE
16. Introduction to CDK8s+ How does it relate to CDK8s?
17. High level abstractions for authoring Kubernetes applications. Built on top of the auto-generated building blocks provided by cdk8s, this library includes a hand crafted construct for each native kubernetes object, exposing richer API’s with reduced complexity. cdk8s.io/docs/latest/plus WHAT IS CDK8s+ How? CDK8s+ is a separate library With a release per Kubernetes Version - "cdk8s-plus-25" - "cdk8s-plus-26" - "cdk8s-plus-27" - ...
18. WHY CDK8s+? - Intent driven APIs for k8s objects (read: AWS Blog) - Higher level abstractions on top of CDK8s L0 constructs
19. - Decoupled from k8s Group / Version / Kind changes - Simplifies k8s upgrades… - What is the new spec format for HPA from k8s version 1.23 onwards? v2beta2 to v2 upgrade? - What changed from ingress v1beta1 to v1 deprecation in 1.22? See: AWS Blog + Containers from the Couch, simplifying k8s upgrades CDK8s+ Benefits 5
20. Demo time 🤞 4
21. - Sample CDK8s app from samples - Deployment + Service + Ingress - Add HPA - Sample CDK8s construct library for Datadog Operator CRDs - Add DataDog Monitor to deployment Demo 5
22. CDK8s Advanced 10 Tips and Tricks
23. - You can leverage all existing helm charts! (and customize the result) - Cdk8s integrates well with CDKTF (and CDK) using Resolvers - Mind shift required! from pure declarative to OOP Abstractions (configurable objects) Tips and Tricks
24. - Runs helm template and imports the yaml into memory - Allows manipulation / interaction with all Helm chart resources CDK8s Helm Support export class MyChart extends Chart { constructor(scope: Construct, id: string) { super(scope, id); new Helm(this, "redis", { chart: "bitnami/redis", values: { sentinel: { enabled: true, }, }, }); } } const redis = new Helm(this, "redis", { chart: "bitnami/redis", values: { sentinel: { enabled: true, }, }, }); const master = redis.apiObjects.find( (o) => o.name === "foo-redis-master" ); master?.metadata.addAnnotation( "my.annotation", "hey-there" );
25. Example usage: Fetch deploy time values from IaC CDK8s Resolvers
26. export interface MyChartProps { // constructor props (just like my helm chart values 🎉) readonly tag: string readonly createServiceAccount?: boolean; readonly secretsARN?: string; readonly reportsBucketMap?: Record<string, string>; } export class SampleChart extends Chart { constructor(scope: Construct, id: string, props: MyChartProps) { super(scope, id); const deployment = new Deployment(this, "Deployment"); const container = deployment.addContainer({// ... }); if (props.secretsARN) { const sp = new MySecretsProvider(props.secretsARN); // add volume to deploymand and mount volume to container sp.mountSecrets(container, { path: "/mnt/secrets", }); } if (props.reportsBucketMap) { for (const [k, v] of props.reportsBucketMap) { container.env.addVariable( // ... ); } } Tips and Tricks - Mind Shift import { App } from "cdk8s"; import { SampleChart } from "./SampleChart" ; const app = new App(); // register chart into app new SampleChart(app, "Sample", { tag: "f87dfd1", secretsARN: "arn://...", reportsBucketMap: { fooReports: "s3://...", barReports: "s3://...", bazReports: "s3://...", }); app.synth(); PSEUDO CODE
27. Tips and Tricks - Mind Shift export interface MyChartProps { readonly tag: string } export class SampleChart extends Chart { readonly private deployment: kplus.Deployment readonly private container: kplus.Container readonly private sp: MySecretProvider constructor(scope: Construct, id: string, props: MyChartProps) { super(scope, id); this.deployment = new Deployment(this, "Deployment"); this.container = deployment.addContainer({ // container spec }); } public mountSecrets(secretsARN: string) { if(this.sp){ // unmount if already exists? (or throw error?) this.sp.unmountSecrets(container) } this.sp = new MySecretsProvider(props.secretsARN); // add volume to deployment and mount volume to container this.sp.mountSecrets(container, { path: "/mnt/secrets", }); } import { App } from "cdk8s"; import { SampleChart } from "./SampleChart" ; const app = new App(); const chart = new SampleChart(app, "Sample", { tag: "f87dfd1", }); chart.mountSecrets("arn://...") chart.addReportsBucket("fooReports", "arn://...") chart.addReportsBucket("barReports", "arn://...") app.synth(); PSEUDO CODE
28. CDK8s vs … 4
29. CDK8s vs … 5 CDK8s Kustomize Helm Authoring (IDE + Intellisense) Language of choice Established language servers Yaml Yaml + gotemplates Custom helm IDE plugins Testing Established test frameworks i.e. JestJS ?? Niche, custom tooling Distribution Established package managers. I.e. Npmjs / Pypip / Maven / go pkg / NuGet / … Niche, custom http/git Niche, Helm registries / OCI Versioning Expressive version constraints for dependencies Git ref? Semver, but no version constraint support only pinned CRD support Strongly typed interfaces auto generated from OpenAPI Specs Schemaless yaml Schemaless yaml nindented text templates
30. Thank you