Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Kubernetes clusters with Terraform / Terragrunt...

vincentdesmet
December 12, 2019

Kubernetes clusters with Terraform / Terragrunt and Kops

How do you make Kops and Terraform play nice? What is special about Terraform 0.12? What is Terragrunt? And why would I want to use any of these when I can just use GKE / EKS? This talk is lessons learned from managing k8s clusters with kops since 2017. We will also look at what pros the solution discussed offers versus EKS and what common cluster operations this design takes into account. It may be quite heavily AWS focused, but it should also be useful if you're on other cloud providers.

Sample code: https://github.com/compareasiagroup/terraform-provider-kops
Join us: http://www.compareasiagroup.com/powered-by/
Presented at:
- https://www.meetup.com/Singapore-Kubernetes-User-Group/events/266944233/

vincentdesmet

December 12, 2019
Tweet

More Decks by vincentdesmet

Other Decks in Technology

Transcript

  1. Page 1 12 December 2019 Date / other info December,

    2019 Kops clusters with TF & Terragrunt Vincent De Smet, Systems Engineer
  2. Page 2 12 December 2019 Table of contents 1. Kops

    → Terraform 2. Kops Templates 3. Terraform → Kops Templates 4. Terraform → Kops 5. Why Terragrunt? 6. Why not EKS / GKE? 7. Blue / Green clusters Kops is a community driven tool to create and manage Kubernetes clusters. Terraform 0.12 supports structured objects for more expressive configuration and powerful templating. Terragrunt provides best practices for Terraform code layout and accelerates common operations. Together they allow us to come up with a strategy to handle frequent platform upgrades decoupled from stateful application components Vincent De Smet Systems Engineer EXEC SUMMARY
  3. Page 3 12 December 2019 - Re-use Terraform expertise -

    Treat as module: - wrap modifications around code generated files, or - drop tf files into folder (no encapsulation…) - No auto-prune Kops → Terraform (v1 ~2017): Kops output Terraform
  4. Page 4 12 December 2019 Problems with kops default networking:

    Routing Tables / Network Access Control Lists / ... • VPC Peering • Direct Connections • Egress control Kops Templates (v2 ~ 2018) Templated kops manifests
  5. Page 9 12 December 2019 - Reduce Copy Paste -

    Let Kops handle Kops Manifests TF → Kops Values → Kops Templates → TF AWS Resource Identifiers are in Terraform
  6. Page 10 12 December 2019 - Prior work: - Nov-Nov

    2018 - MongoDB - Colin Huglund - PoC - TF 0.12 - no expand / flatten - abandoned - Nov-Jan 2019 - Wandera - Jakub Coufal - TF 0.12 - Based on Colin Hugland - Added expand / flatten for kops schema - Alternative to WanderaOrg/karch local-exec - Abandoned ~ January 2019 - compareasiagroup/terraform-provider-kops - Based on Colin Huglund work - No expand / flatten - Implemented missing functions - Actively being used, PRs welcome! Terraform → Kops (v1 ~2019): Terraform provider Kops
  7. Page 11 12 December 2019 - Manage Kops manifests through

    Terraform - Template AWS Resource Identifiers Terraform → Kops (v1 ~2019): Terraform provider Kops
  8. Page 12 12 December 2019 - TF 0.12 built-in Templating

    Terraform → Kops (v1 ~2019): Terraform provider Kops
  9. Page 13 12 December 2019 - Recommended read: Terraform Up

    And Running, second Edition! Why Terragrunt? Terragrunt for TF 0.12
  10. Page 14 12 December 2019 - Easily split state -

    Modules on steroids - Dependencies between modules - Layered infra approach Why Terragrunt? Terragrunt for TF 0.12
  11. Page 15 12 December 2019 - Existing Kops expertise -

    Self Managed clusters in foreseeable future Why no EKS / GKE ? Hosted solutions simplify Operations
  12. Page 16 12 December 2019 - Isolated Environments with dedicated

    stateful tier - Decouple cluster from data and exposed interfaces Blue / Green Clusters Simplify Cluster upgrades by keeping them stateless
  13. Page 18 12 December 2019 Page 18 We are Hiring!

    Please get in touch with Thulasi for private and confidential discussion or visit our Career Site for more details