HashiDays London 2017 - Evolving your Infrastructure with Terraform

Talk given at HashiDays London 2017. Associated video can be found here: https://www.youtube.com/watch?v=wgzgVm7Sqlk

Transcript

1. EVOLVING YOUR INFRASTRUCTURE WITH TERRAFORM Nicki Watt - CTO
 @techiewatt 12-06-2017
2. ABOUT ME / OPENCREDO ▸OpenCredo CTO ▸Premiere HashiCorp partner ▸Hands on software development consultancy ▸Cloud, Data Engineering, DevSecOps 2
3. AGENDA ▸Evolving your Terraform ▸Orchestrating your Terraform ▸Conclusion 3
4. 4 Evolving your Terraform
 (a journey from a client’s perspective)
5. 6 Example: E-Commerce System in AWS
 (delivered as a Micro-services architecture) 

6. 7 Sample System 
 Simple 
 Kubernetes (K8S) 
 Environment
7. 8 public DMZ & Bastion Box k8s cluster Sample System 
 Simple 
 Kubernetes (K8S) 
 Environment database (RDS)
8. 9 Pass #1 - In the beginning …
9. 10 https://github.com/mycompany/myproject terraform.tf ## Test VPC resource "aws_vpc" "test" { cidr_block = "10.0.0.0/21" enable_dns_support = true enable_dns_hostnames = true } ## Staging Bastion resource "aws_instance" “test_bastion" { ami = "ami-7abd5555" instance_type = "t2.large" . . . } 
 - terraform-prod.tf
 - terraform.tf
 - terraform.tfvars
 - terraform.tfstate
10. 11 We must to go to production this week …
11. terraform.tf ## Test VPC resource "aws_vpc" "test" { cidr_block = "10.0.0.0/21" enable_dns_support = true enable_dns_hostnames = true } ## Staging Bastion resource "aws_instance" “test_bastion" { ami = "ami-7abd5555" instance_type = "t2.large" . . . } ## Prod VPC resource "aws_vpc" "prod" { cidr_block = "172.16.0.0/21" enable_dns_support = true enable_dns_hostnames = true } 12 https://github.com/mycompany/myproject 
 - terraform-prod.tfbkp
 - terraform.tf
 - terraform.tfvars
 - terraform.tfstate
12. terraform-test.tf ## Test VPC resource "aws_vpc" "test" { cidr_block = "10.0.0.0/21" enable_dns_support = true enable_dns_hostnames = true } ## Staging Bast-ion resource "aws_instance" “test_bastion" { ami = "ami-7abd5555" instance_type = "t2.large" . . . } ## Prod VPC resource "aws_vpc" "prod" { cidr_block = "10.0.0.3/24" enable_dns_support = true enable_dns_hostnames = true } 13 https://github.com/mycompany/myproject terraform-prod.tf ## Prod VPC resource "aws_vpc" "prod" { cidr_block = "172.16.0.0/21" enable_dns_support = true enable_dns_hostnames = true } ## Staging Bastion resource "aws_instance" “prod_bastion" { ami = "ami-7abd5555" instance_type = "t2.large" . . . 
 - terraform-prod.tf
 - terraform-test.tf
 - terraform.tfvars
 - terraform.tfstate
13. 14 Need an upgraded CIDR range in TEST …
14. 15 
 - terraform-prod.tfbkp
 - terraform-test.tf
 - terraform.tfvars
 - terraform.tfstate https://github.com/mycompany/myproject terraform-test.tf ## Test VPC resource "aws_vpc" "test" { cidr_block = "10.0.0.0/21" enable_dns_support = true enable_dns_hostnames = true } ## Staging Bast-ion resource "aws_instance" “test_bastion" { ami = "ami-7abd5555" instance_type = "t2.large" . . . } ## Prod VPC resource "aws_vpc" "prod" { cidr_block = "10.0.0.3/24" enable_dns_support = true enable_dns_hostnames = true } terraform-prod.tf ## Prod VPC resource "aws_vpc" "prod" { cidr_block = "172.16.0.0/21" enable_dns_support = true enable_dns_hostnames = true } ## Staging Bastion resource "aws_instance" “prod_bastion" { ami = "ami-7abd5555" instance_type = "t2.large" . . . 15
15. 16 Help! I seem to have deleted production
16. 17 “terralith" https://sites.google.com/site/laurenmcnanyspln/magnetic-fields
17. ▸Single state file ▸Single definition file ▸Hard coded config ▸Local state Terralith: Characteristics 18
18. ▸Can’t manage environments separately ▸Config not that intuitive 
 (big ball of mud) ▸Maintenance challenge: Duplicate Defs (not DRY) Terralith - Pain points 19
19. 20 Pass #2
20. 21 “multi terralith" https://sites.google.com/site/laurenmcnanyspln/magnetic-fields
21. ▸Envs - Separate State Management ▸Multiple Terraform Definition Files ▸Better Use of Variables Multi Terralith: Characteristics 22
22. + test
 - networks.tf
 - vms.tf
 - terraform.tfvars
 - terraform.tfstate 23 https://github.com/mycompany/myproject networks.tf resource "aws_vpc" "core" { cidr_block = "${var.cidr}" enable_dns_support = true enable_dns_hostnames = true } vms.tf resource "aws_instance" "node" { count = "${var.node_count}" ami = "ami-7abd5555" instance_type = “${var.vm_type}” . . . } + prod
 - networks.tf
 - vms.tf
 - terraform.tfvars
 - terraform.tfstate
23. 24 https://github.com/mycompany/myproject networks.tf resource "aws_vpc" "core" { cidr_block = “${var.cidr}” enable_dns_support = true enable_dns_hostnames = true } vms.tf resource "aws_instance" "node" { count = "${var.node_count}" ami = "ami-7abd5555" instance_type = “${var.vm_type}” . . . } + test
 - networks.tf
 - vms.tf
 - terraform.tfvars
 - terraform.tfstate + prod
 - networks.tf
 - vms.tf
 - terraform.tfvars
 - terraform.tfstate
24. + test
 - networks.tf
 - vms.tf
 - terraform.tfvars
 - terraform.tfstate 25 https://github.com/mycompany/myproject networks.tf resource "aws_vpc" "core" { cidr_block = “${var.cidr}” enable_dns_support = true enable_dns_hostnames = true } vms.tf resource "aws_instance" "node" { count = "${var.node_count}" ami = "ami-7abd5555" instance_type = “${var.vm_type}” . . . } + prod
 - networks.tf
 - vms.tf
 - terraform.tfvars
 - terraform.tfstate
25. Terralith - (recap) 26 ▸Can’t manage environments separately
 ▸Config not that intuitive 
 (big ball of mud) ▸Maintenance challenge: Duplicate Defs (not DRY)
26. Multi Terralith 27 ▸Manage environment separately
 (separate state files per env) ▸More intuitive configuration
 (multiple files) ▸Maintenance challenge: Duplicate Defs (not DRY) ✅ "
27. 28 Pass #3
28. 29 “terramod" Alan Chia (https://commons.wikimedia.org/wiki/File:Lego_Color_Bricks.jpg)
29. ▸Reusable modules ▸Envs compose themselves from modules ▸Restructuring of repo 30 Terramod: Characteristics
30. 31 database core k8s-cluster
31. 32 database core k8s-cluster - VPC
 - All Subnets
 - Core Routing & Gateways
 - Bastion Host (OpenVPN server) - Instances
 - Security Groups - Amazon RDS
 - DB Subnet Group
32. 33 + envs/[test|prod]
 - config.tf
 - terraform.tf
 - terraform.tfvars
 - terraform.tfstate + modules
 + core
 - input.tf
 - core.tf
 - output.tf
 + k8s-cluster
 - input.tf
 - dns.tf
 - vms.tf
 - output.tf
 https://github.com/mycompany/myproject separate env management & module defs
33. 34 + envs/[test|prod]
 - config.tf
 - terraform.tf
 - terraform.tfvars
 - terraform.tfstate + modules
 + core
 - input.tf
 - core.tf
 - output.tf
 + k8s-cluster
 - input.tf
 - dns.tf
 - vms.tf
 - output.tf
 https://github.com/mycompany/myproject define logical components as re-usable modules
34. + envs/[test|prod]
 - config.tf
 - terraform.tf
 - terraform.tfvars
 - terraform.tfstate + modules
 + core
 - input.tf
 - core.tf
 - output.tf
 + k8s-cluster
 - input.tf
 - dns.tf
 - vms.tf
 - output.tf
 35 https://github.com/mycompany/myproject core.tf resource "aws_vpc" "core" { cidr_block = "${var.cidr}" enable_dns_support = "${var.dns}" enable_dns_hostnames = "${var.dnsh}" } resource "aws_subnet" "dmz" { vpc_id = "${aws_vpc.core.id}" cidr_block = "${var.dmz_cidr}" map_public_ip_on_launch = 1 ... } resource "aws_subnet" "private" { vpc_id = "${aws_vpc.core.id}" cidr_block = "${var.priv_cidr}" ... }
35. + envs/[test|prod]
 - config.tf
 - terraform.tf
 - terraform.tfvars
 - terraform.tfstate + modules
 + core
 - input.tf
 - core.tf
 - output.tf
 + k8s-cluster
 - input.tf
 - dns.tf
 - vms.tf
 - output.tf
 36 https://github.com/mycompany/myproject core.tf resource "aws_vpc" "core" { cidr_block = "${var.cidr}" enable_dns_support = "${var.dns}" enable_dns_hostnames = "${var.dnsh}" } resource "aws_subnet" "dmz" { vpc_id = "${aws_vpc.core.id}" cidr_block = "${var.dmz_cidr}" map_public_ip_on_launch = 1 ... } resource "aws_subnet" "private" { vpc_id = "${aws_vpc.core.id}" cidr_block = "${var.priv_cidr}" ... } input.tf variable "cidr" {} variable "dns” {} variable "dnsh" {} variable "dmz_cidr" {} variable "priv_cidr" {} ...
36. + envs/[test|prod]
 - config.tf
 - terraform.tf
 - terraform.tfvars
 - terraform.tfstate + modules
 + core
 - input.tf
 - core.tf
 - output.tf
 + k8s-cluster
 - input.tf
 - dns.tf
 - vms.tf
 - output.tf
 37 https://github.com/mycompany/myproject core.tf resource "aws_vpc" "core" { cidr_block = "${var.cidr}" enable_dns_support = "${var.dns}" enable_dns_hostnames = "${var.dnsh}" } resource "aws_subnet" "dmz" { vpc_id = "${aws_vpc.core.id}" cidr_block = "${var.dmz_cidr}" map_public_ip_on_launch = 1 ... } resource "aws_subnet" "private" { vpc_id = "${aws_vpc.core.id}" cidr_block = "${var.priv_cidr}" ... } input.tf variable "cidr" {} variable "dns” {} variable "dnsh" {} variable "dmz_cidr" {} variable "priv_cidr" {} ... output.tf output "priv_subnet_id" { value ="${aws_subnet.private.id}" } ...
37. + envs/[test|prod]
 - config.tf
 - terraform.tf
 - terraform.tfvars
 - terraform.tfstate + modules
 + core
 - input.tf
 - core.tf
 - output.tf
 + k8s-cluster
 - input.tf
 - dns.tf
 - vms.tf
 - output.tf
 38 https://github.com/mycompany/myproject defines the contract of the module
38. + envs/[test|prod]
 - config.tf
 - terraform.tf
 - terraform.tfvars
 - terraform.tfstate + modules
 + core
 - input.tf
 - core.tf
 - output.tf
 + k8s-cluster
 - input.tf
 - dns.tf
 - vms.tf
 - output.tf
 39 https://github.com/mycompany/myproject terraform.tf module "core" { source = "../../modules/core" cidr = "${var.vpc_cidr}" dmz_cidr = "${var.dmz_cidr}" priv_cidr = "${var.priv_cidr}" } module "k8s-cluster" { source = "../../modules/k8s-cluster" num_nodes = "${var.k8s_nodes}" priv_subnet = 
 "${module.core.priv_subnet_id}" }
39. + envs/[test|prod]
 - config.tf
 - terraform.tf
 - terraform.tfvars
 - terraform.tfstate + modules
 + core
 - input.tf
 - core.tf
 - output.tf
 + k8s-cluster
 - input.tf
 - dns.tf
 - vms.tf
 - output.tf
 40 https://github.com/mycompany/myproject terraform.tf module "core" { source = "../../modules/core" cidr = "${var.vpc_cidr}" dmz_cidr = "${var.dmz_cidr}" priv_cidr = "${var.priv_cidr}" } module "k8s-cluster" { source = "../../modules/k8s-cluster" num_nodes = "${var.k8s_nodes}" priv_subnet = 
 "${module.core.priv_subnet_id}" }
40. + envs/[test|prod]
 - config.tf
 - terraform.tf
 - terraform.tfvars
 - terraform.tfstate + modules
 + core
 - input.tf
 - core.tf
 - output.tf
 + k8s-cluster
 - input.tf
 - dns.tf
 - vms.tf
 - output.tf
 41 https://github.com/mycompany/myproject terraform.tf module "core" { source = "../../modules/core" cidr = "${var.vpc_cidr}" dmz_cidr = "${var.dmz_cidr}" priv_cidr = "${var.priv_cidr}" } module "k8s-cluster" { source = "../../modules/k8s-cluster" num_nodes = "${var.k8s_nodes}" priv_subnet = 
 "${module.core.priv_subnet_id}" }
41. ▸Manage environment separately
 (separate state files per env) ▸More intuitive configuration
 (multiple files) ▸Maintenance challenge: Duplicate Defs (not DRY) Multi Terralith 42 ✅ "
42. ▸Manage environment separately
 (separate state files per env) ▸Intuitive configuration
 (reusable modules) ▸Reduced Duplicate Definitions 
 (DRYer) Terramod 43 ✅ " ✅
43. 44 Pass #4
44. 45 Alan Chia (https://commons.wikimedia.org/wiki/File:Lego_Color_Bricks.jpg) terramod Marcos Leal (https://commons.wikimedia.org/wiki/File:Army_(2995294027).jpg) n
45. ▸Nested modules ▸base modules
 (low level infrastructure specific) ▸logical modules
 (system specific) ▸Sometimes dedicated module repo 46 Terramod : Characteristics n
46. 47 https://github.com/mycompany/myproject + envs
 + modules
 + project
 + core
 - input.tf
 - core.tf
 - output.tf
 + k8s-cluster
 - input.tf
 - k8s.tf
 - output.tf
 logical (system specific) modules
47. 48 https://github.com/mycompany/myproject + envs
 + modules
 + common
 + aws
 + network
 + vpc
 + pub_subnet
 + priv_subnet
 + comps
 + instance
 + db-instance
 + envs
 + modules
 + project
 + core
 - input.tf
 - core.tf
 - output.tf
 + k8s-cluster
 - input.tf
 - k8s.tf
 - output.tf
 logical (system specific) modules base (infra specific) modules
48. 49 https://github.com/mycompany/myproject + envs
 + modules
 + common
 + aws
 + network
 + vpc
 + pub_subnet
 + priv_subnet
 + comps
 + instance
 + db-instance
 + envs
 + modules
 + project
 + core
 - input.tf
 - core.tf
 - output.tf
 + k8s-cluster
 - input.tf
 - k8s.tf
 - output.tf
 modules/project/core/core.tf resource "aws_vpc" "core" { cidr_block = "${var.cidr}" enable_dns_support = "${var.dns}" enable_dns_hostnames = "${var.dnsh}" } resource "aws_subnet" "dmz" { vpc_id = "${aws_vpc.core.id}" cidr_block = "${var.dmz_cidr}" map_public_ip_on_launch = 1 ... }
49. modules/project/core/core.tf resource "aws_vpc" "core" { cidr_block = "${var.cidr}" enable_dns_support = "${var.dns}" enable_dns_hostnames = "${var.dnsh}" } resource "aws_subnet" "dmz" { vpc_id = "${aws_vpc.core.id}" cidr_block = "${var.dmz_cidr}" map_public_ip_on_launch = 1 ... } + envs
 + modules
 + project
 + core
 - input.tf
 - core.tf
 - output.tf
 + k8s-cluster
 - input.tf
 - k8s.tf
 - output.tf
 50 https://github.com/mycompany/myproject
50. modules/project/core/core.tf resource "aws_vpc" "core" { cidr_block = "${var.cidr}" enable_dns_support = "${var.dns}" enable_dns_hostnames = "${var.dnsh}" } resource "aws_subnet" "dmz" { vpc_id = "${aws_vpc.core.id}" cidr_block = "${var.dmz_cidr}" map_public_ip_on_launch = 1 ... } + envs
 + modules
 + common
 + aws
 + network
 + vpc
 + pub_subnet
 + priv_subnet
 + comps
 + instance
 + db-instance
 50 https://github.com/mycompany/myproject modules/project/core/core.tf module "vpc" { source = "../../common/aws/net/vpc" cidr = "${var.vpc_cidr}" } module "dmz-subnet" { source = "../../common/aws/net/pub-subnet" vpc_id = "${module.vpc.vpc_id}" subnet_cidrs = [ “${var.dmz_cidr}” ] } module "priv-subnet" { source = "../../common/aws/net/priv-subnet" vpc_id = "${module.vpc.vpc_id}" subnet_cidrs = [ “${var.priv_cidr}” ]
51. + envs
 + modules
 + common
 + aws
 + network
 + vpc
 + pub_subnet
 + priv_subnet
 + comps
 + instance
 + db-instance
 modules/project/core/core.tf resource "aws_vpc" "core" { cidr_block = "${var.cidr}" enable_dns_support = "${var.dns}" enable_dns_hostnames = "${var.dnsh}" } resource "aws_subnet" "dmz" { vpc_id = "${aws_vpc.core.id}" cidr_block = "${var.dmz_cidr}" map_public_ip_on_launch = 1 ... } 51 https://github.com/mycompany/myproject modules/project/core/core.tf module "vpc" { source = "../../common/aws/net/vpc" cidr = "${var.vpc_cidr}" } module "dmz-subnet" { source = "../../common/aws/net/pub-subnet" vpc_id = "${module.vpc.vpc_id}" subnet_cidrs = [ “${var.dmz_cidr}” ] } module "priv-subnet" { source = "../../common/aws/net/priv-subnet" vpc_id = "${module.vpc.vpc_id}" subnet_cidrs = [ “${var.priv_cidr}” ] BUT … Issue #953 - Support the count parameter for modules
52. ▸Manage environment separately
 (separate state files per env) ▸Intuitive configuration
 (reusable modules) ▸Reduced Duplicate Definitions 
 (DRYer) Terramod (recap) 52 ✅ " ✅
53. ▸Manage environment separately
 (separate state files per env) ▸Intuitive configuration
 (reusable modules) ▸Reduced Duplicate Definitions further 
 (as DRY as possible given restrictions) Terramod 53 n ✅ " ✅
54. 54 Time goes on …
55. 55 Maintenance required …
 - Make bastion box smaller -
56. + envs/prod
 - config.tf
 - terraform.tf
 - terraform.tfvars
 - terraform.tfstate + modules
 + core
 - input.tf
 - core.tf
 - output.tf
 + k8s-cluster
 - input.tf
 - dns.tf
 - vms.tf
 - output.tf
 56 terraform.tf module "core" { source = "../../modules/core" cidr = "${var.vpc_cidr}" dmz_cidr = "${var.dmz_cidr}" priv_cidr = "${var.priv_cidr}" bastion_flav = "${var.bastion_flav}" } module "k8s-cluster" { source = "../../modules/k8s-cluster" num_nodes = "${var.k8s_nodes}" node_flavour = "${var.bastion_flav}" } terraform.tfvars vpc_cidr = “10.0.0.0/21” bastion_flav = “r4.large” node_flavour = “m4.4xlarge”
57. + envs/prod
 - config.tf
 - terraform.tf
 - terraform.tfvars
 - terraform.tfstate + modules
 + core
 - input.tf
 - core.tf
 - output.tf
 + k8s-cluster
 - input.tf
 - dns.tf
 - vms.tf
 - output.tf
 57 terraform.tf module "core" { source = "../../modules/core" cidr = "${var.vpc_cidr}" dmz_cidr = "${var.dmz_cidr}" priv_cidr = "${var.priv_cidr}" bastion_flav = "${var.bastion_flav}" } module "k8s-cluster" { source = "../../modules/k8s-cluster" num_nodes = "${var.k8s_nodes}" node_flavour = "${var.bastion_flav}" } terraform.tfvars vpc_cidr = “10.0.0.0/21” bastion_flav = “m4.large” node_flavour = “m4.4xlarge”
58. + envs/prod
 - config.tf
 - terraform.tf
 - terraform.tfvars
 - terraform.tfstate + modules
 + core
 - input.tf
 - core.tf
 - output.tf
 + k8s-cluster
 - input.tf
 - dns.tf
 - vms.tf
 - output.tf
 58 terraform.tf module "core" { source = "../../modules/core" cidr = "${var.vpc_cidr}" dmz_cidr = "${var.dmz_cidr}" priv_cidr = "${var.priv_cidr}" bastion_flav = "${var.bastion_flav}" } module "k8s-cluster" { source = "../../modules/k8s-cluster" num_nodes = "${var.k8s_nodes}" node_flavour = "${var.bastion_flav}" } terraform.tfvars vpc_cidr = “10.0.0.0/21” bastion_flav = “m4.large” node_flavour = “m4.4xlarge”
59. 59 Help! I seem to be rebuilding the K8S cluster!
60. + envs/prod
 - config.tf
 - terraform.tf
 - terraform.tfvars
 - terraform.tfstate + modules
 + core
 - input.tf
 - core.tf
 - output.tf
 + k8s-cluster
 - input.tf
 - dns.tf
 - vms.tf
 - output.tf
 60 terraform.tf module "core" { source = "../../modules/core" cidr = "${var.vpc_cidr}" dmz_cidr = "${var.dmz_cidr}" priv_cidr = "${var.priv_cidr}" bastion_flav = "${var.bastion_flav}" } module "k8s-cluster" { source = "../../modules/k8s-cluster" num_nodes = "${var.k8s_nodes}" node_flavour = "${var.bastion_flav}" } terraform.tfvars vpc_cidr = “10.0.0.0/21” bastion_flav = “m4.large” node_flavour = “m4.4xlarge” OOPS! Typo
61. ▸Can’t manage logical parts of our infrastructure independently Next set of pain! 61
62. 62 Pass #5
63. 63 “terraservices" https://commons.wikimedia.org/wiki/File:Caffeine_Molecule.png
64. ▸ Independent management of logical comps ▸ Isolates & Reduces Risk ▸ Aids with Multi Team Setups ▸Distributed (Remote State) ▸Requires additional orchestration effort Terraservices - Characteristics 64
65. 65 database core k8s-cluster - VPC
 - All Subnets
 - Core Routing & Gateways
 - Bastion Host (OpenVPN server) - Instances
 - Security Groups - Amazon RDS
 - DB Subnet Group
66. + envs
 + test
 
 - ...
 - ...
 - ...
 + k8s-cluster
 - ...
 
 + prod
 + core
 - ...
 - ...
 - ...
 + k8s-cluster
 - ... 66 - terraform.tfstate
 - terraform.tfvars
 - xxx.tf Terraservices - Repo Structure From
67. 67 - terraform.tfstate
 - terraform.tfvars
 - xxx.tf To + envs
 + test
 + core
 - ...
 + database
 - ...
 + k8s-cluster
 - ...
 
 + prod
 + core
 - ...
 + database
 - ...
 + k8s-cluster
 - ... Terraservices - Repo Structure
68. + envs
 + test
 
 - ...
 - ...
 - ...
 + k8s-cluster
 - ...
 
 + prod
 + core
 - ...
 - ...
 - ...
 + k8s-cluster
 - ... 68 envs/test/terraform.tf module "core" { source = "../../modules/core" cidr = "${var.vpc_cidr}" dmz_cidr = "${var.dmz_cidr}" priv_cidr = "${var.priv_cidr}" } module "k8s-cluster" { source = "../../modules/k8s-cluster" num_nodes = "${var.k8s_nodes}" priv_subnet = 
 "${module.core.priv_subnet_id}" } Terramod - Connecting (recap) From
69. 69 + envs
 + test
 + core
 - ...
 + database
 - ...
 + k8s-cluster
 - ...
 
 + prod
 + core
 - ...
 + database
 - ...
 + k8s-cluster
 - ... envs/test/core/terraform.tf # Optional but explicit! (Needs 0.9+) terraform { backend "local" { path = "terraform.tfstate" } } module "core" { source = "../../modules/core" cidr = "${var.vpc_cidr}" envs/test/core/outputs.tf output "priv_subnet_id" { value ="${module.core.priv_subnet_id}" } Terraservices - Connecting To
70. 70 + envs
 + test
 + core
 - ...
 + database
 - ...
 + k8s-cluster
 - ...
 
 + prod
 + core
 - ...
 + database
 - ...
 + k8s-cluster
 - ... envs/test/core/terraform.tf # Optional but explicit! (Needs 0.9+) terraform { backend "local" { path = "terraform.tfstate" } } module "core" { source = "../../modules/core" cidr = "${var.vpc_cidr}" envs/test/core/outputs.tf output "priv_subnet_id" { value ="${aws_subnet.private.id}" } envs/test/k8s-cluster/terraform.tf data "terraform_remote_state" "core" { backend = "local" config { path = “../core/terraform.tfstate" } } module "k8s-cluster" { source = "../../modules/k8s-cluster" num_nodes = "${var.k8s_nodes}" priv_subnet = “${data.terraform_remote_ state.core.priv_subnet_id}" }
 } Terraservices - Connecting To
71. Terraservices - Characteristics 71 ▸ Independent management of logical comps ▸ Isolates & Reduces Risk ▸ Aids with Multi Team Setups ▸Distributed (Remote State) ▸Requires additional orchestration effort
72. 72 + envs
 + test
 + core
 - ...
 + database
 - ...
 + k8s-cluster
 - ...
 
 + prod
 + core
 - ...
 + database
 - ...
 + k8s-cluster
 - ... envs/test/core/terraform.tf # Optional but explicit! (Needs 0.9+) terraform { backend "local" { path = "terraform.tfstate" } } module "core" { source = "../../modules/core" cidr = "${var.vpc_cidr}" envs/test/core/outputs.tf output "priv_subnet_id" { value ="${module.core.priv_subnet_id}" } Terraservices - Distributed (Remote State) From
73. 73 + envs
 + test
 + core
 - ...
 + database
 - ...
 + k8s-cluster
 - ...
 
 + prod
 + core
 - ...
 + database
 - ...
 + k8s-cluster
 - ... envs/test/core/terraform.tf # Optional but explicit! (Needs 0.9+) terraform { backend "s3" { region = "eu-west-1" bucket = "myco/myproj/test" key = "core/terraform.tfstate" encrypt = "true" } } envs/test/core/outputs.tf output "priv_subnet_id" { value ="${module.core.priv_subnet_id}" } Terraservices - Distributed (Remote State) To
74. 74 + envs
 + test
 + core
 - ...
 + database
 - ...
 + k8s-cluster
 - ...
 
 + prod
 + core
 - ...
 + database
 - ...
 + k8s-cluster
 - ... envs/test/core/terraform.tf # Optional but explicit! (Needs 0.9+) terraform { backend "s3" { region = "eu-west-1" bucket = "myco/myproj/test" key = "core/terraform.tfstate" encrypt = "true" } } envs/test/core/outputs.tf output "priv_subnet_id" { value ="${module.core.priv_subnet_id}" } Terraservices - Distributed (Remote State) To envs/test/k8s-cluster/terraform.tf data "terraform_remote_state" "core" { backend = "s3" config { region = "eu-west-1" bucket = "myco/myproj/test" key = "core/terraform.tfstate" encrypt = "true" } } module "k8s-cluster" { source = "../../modules/k8s-cluster" num_nodes = "${var.k8s_nodes}" priv_subnet = “${data.terraform_remote_ state.core.priv_subnet_id}"
75. 75 + envs
 + test|prod
 + core
 - ...
 + database
 - ...
 + k8s-cluster
 - ...
 
 + modules
 + common
 + aws
 + network
 + vpc
 Terraservices - Repo Isolation (Optional) https://github.com/myco/myproj From
76. 76 + envs
 + test|prod
 + core
 - ...
 + database
 - ...
 + k8s-cluster
 - ...
 
 + modules
 + common
 + aws
 + network
 + vpc
 Terraservices - Repo Isolation (Optional) https://github.com/myco/myproj https://github.com/myco/myproj-core https://github.com/myco/myproj-db https://github.com/myco/myproj-k8s https://github.com/myco/tf-modcomm To
77. Terraservices - Characteristics 77 ▸ Independent management of logical comps ▸ Isolates & Reduces Risk ▸ Aids with Multi Team Setups ▸Distributed (Remote State) ▸Requires additional orchestration effort
78. 78 Orchestrating your Terraform
79. 79 database core k8s-cluster
80. 80 database core k8s-cluster
81. 81 Orchestration System
82. 82 Orchestration System Laptop, Local State 
 & READMEs
83. 83 Orchestration System Laptop, Local State 
 & READMEs
84. 84 Orchestration System Laptops, Local State 
 & READMEs
85. 85 Orchestration System Laptops, Local State 
 & READMEs Remote State
86. 86 Orchestration System Laptops, Local State 
 & READMEs Remote State
87. 87 Orchestration System Laptops, Remote State, 
 Shared Services,
 & READMEs
88. 88 Orchestration System Who builds the infrastructure that builds the infrastructure ?
89. 89 Orchestration System Jenkins, Remote State,
 Custom Scripts, 
 Shared Services,
 & READMEs
90. 90 Orchestration System Custom Systems 
 & Tooling
91. 91 Orchestration System SaaS Offerings (HashiCorp Enterprise Products)
92. 92 It’s not just about the structure of the code … You also need to evolve your supporting orchestration system & processes
93. 93 Conclusion
94. 94 Evolving Terraform Setup
95. 95 EVOLVING YOUR TERRAFORM SETUP ▸Terralith ▸Multi Terralith - Envs: Independent management ▸Terramod ▸Terramod - Modules : Maintainability & Reuse ▸Terraservices - Logical Components: Independent
 management n
96. 96 EVOLVING YOUR TERRAFORM SETUP ▸Terralith ▸Multi Terralith - Envs: Independent management ▸Terramod ▸Terramod - Modules : Maintainability & Reuse ▸Terraservices - Logical Components: Independent
 management n
97. 97 EVOLVING YOUR TERRAFORM SETUP ▸Terralith ▸Multi Terralith - Envs: Independent management ▸Terramod ▸Terramod - Modules : Maintainability & Reuse ▸Terraservices - Logical Components: Independent
 management n
98. 98 EVOLVING YOUR TERRAFORM SETUP ▸Terralith ▸Multi Terralith - Envs: Independent management ▸Terramod ▸Terramod - Modules : Maintainability & Reuse ▸Terraservices - Logical Components: Independent
 management n
99. 99 EVOLVING YOUR TERRAFORM SETUP ▸Terralith ▸Multi Terralith - Envs: Independent management ▸Terramod ▸Terramod - Modules : Maintainability & Reuse ▸Terraservices - Logical Components: Independent
 management n
100. 100 Also need to consider how to evolve the management & orchestration of Terraform
101. 101 Thanks! @techiewatt