Share
Embed
What's new in Terraform 0.10 - HUG
In this talk we cover an introduction to Terraform, including it's primary use cases of Infrastructure as Code, composing multiple providers (such as IaaS, SaaS, and PaaS), and enabling self service of infrastructure. We show how Terraform works and the basics of the workflow. Then we talk about new features of Terraform 0.10, with a focus on the split between core and providers, the Kubernetes provider, and workspaces. Terraform 0.9 is reviewed with a focus on destroy provisioners, remote backends, state locking.
Armon Dadgar
August 16, 2017
Transcript
86 slides
-
AGENDA HASHICORP USE CASES INTRO TO TERRAFORM NEW FEATURES Terraform
0.10 Terraform 0.9 -
PROVISION, SECURE AND RUN ANY INFRASTRUCTURE Nomad Consul Vault Vagrant
Packer Terraform Consul Enterprise Terraform Enterprise Vault Enterprise PRODUCT SUITE OSS TOOL SUITE RUN Applications SECURE Application Infrastructure PROVISION Infrastructure FOR INDIVIDUALS FOR TEAMS Nomad Enterprise -
SEP 18-20, 2017 | AUSTIN, TEXAS One day of training
followed by two days of talks on the future of infrastructure. Discount Code: HUG20 -
6 HashiCorp confidential do not distribute USE CASES Easily combine
and build complex infrastructure Self-Serve Infrastructure Common Workflow for Any Infrastructure Hybrid Cloud Management Automate Infrastructure Provisioning Infrastructure as Code -
7 HashiCorp confidential do not distribute COLLABORATE on infrastructure as
code using a version control system (VCS) VALIDATE with a common integrated workflow AUTOMATE public cloud, private cloud, and external services THE IDEAL STATE -
Terraform Goals • Unified view of infrastructure • Infrastructure as
code • Compose multiple tiers (IaaS to PaaS to SaaS) • Safely change/iterate infrastructure over time • One workflow -
Terraform Features • Open Source • Infrastructure as Code •
Resource Providers • Plan and Apply • Collaboration, History [Enterprise] -
File resource "google_compute_instance" "server" { name = "server" machine_type =
"g1-small" zone = "us-central1-a" disk { image = "ubuntu-1404-trusty-v20160114e" } } resource "dnsimple_record" "hello" { domain = "example.com" name = "server" value = "${google_compute_instance.server.network_interface.0.address}" type = "A" } -
File resource "google_compute_instance" "server" { name = "server" machine_type =
"g1-small" zone = "us-central1-a" disk { image = "ubuntu-1404-trusty-v20160114e" } } resource "dnsimple_record" "hello" { domain = "example.com" name = "server" value = "${google_compute_instance.server.network_interface.0.address}" type = "A" } -
File resource "google_compute_instance" "server" { name = "server" machine_type =
"g1-small" zone = "us-central1-a" disk { image = "ubuntu-1404-trusty-v20160114e" } } resource "dnsimple_record" "hello" { domain = "example.com" name = "server" value = "${google_compute_instance.server.network_interface.0.address}" type = "A" } -
Infrastructure as Code • Human-readable (HCL), Machine-editable (JSON) • Declarative
• Text makes it versionable and VCS-friendly • Can be split into multiple files and modules -
Resource Providers • Integration point • Expose resources, such as
"aws_instance", etc. • CRUD API to implement • Core vs. Providers simplifies logic for external contributors -
Resource Providers Amazon BitBucket CenturyLink Cloud CloudFlare CloudStack Cobbler Consul
Datadog DigitalOcean DNSMadeEasy DNSimple Docker Dyn GitHub Fastly Google Heroku Librato MicrosoB Azure MySQL OpenStack Packet PostgreSQL SoBLayer UltraDNS VMware Sphere and more... -
Resource Providers Amazon BitBucket CenturyLink Cloud CloudFlare CloudStack Cobbler Consul
Datadog DigitalOcean DNSMadeEasy DNSimple Docker Dyn GitHub Fastly Google Heroku Librato MicrosoB Azure MySQL OpenStack Packet PostgreSQL SoBLayer UltraDNS VMware Sphere and more... -
Resource Providers Amazon BitBucket CenturyLink Cloud CloudFlare CloudStack Cobbler Consul
Datadog DigitalOcean DNSMadeEasy DNSimple Docker Dyn GitHub Fastly Google Heroku Librato MicrosoB Azure MySQL OpenStack Packet PostgreSQL SoBLayer UltraDNS VMware Sphere and more... -
Resource Providers Amazon BitBucket CenturyLink Cloud CloudFlare CloudStack Cobbler Consul
Datadog DigitalOcean DNSMadeEasy DNSimple Docker Dyn GitHub Fastly Google Heroku Librato MicrosoB Azure MySQL OpenStack Packet PostgreSQL SoBLayer UltraDNS VMware Sphere and more... -
Resource Providers Amazon BitBucket CenturyLink Cloud CloudFlare CloudStack Cobbler Consul
Datadog DigitalOcean DNSMadeEasy DNSimple Docker Dyn GitHub Fastly Google Heroku Librato MicrosoB Azure MySQL OpenStack Packet PostgreSQL SoBLayer UltraDNS VMware Sphere and more... -
File resource "google_compute_instance" "server" { name = "server" machine_type =
"g1-small" zone = "us-central1-a" disk { image = "ubuntu-1404-trusty-v20160114e" } } resource "dnsimple_record" "hello" { domain = "example.com" name = "server" value = "${google_compute_instance.server.network_interface.0.address}" type = "A" } -
Plan • Plan shows you what will happen • Plans
can be saved to guarantee what will happen • Plans show reasons for certain actions (such as re-create) • Not equivalent to "noop" due to the ability to save a plan -
Terminal $ terraform plan + google_compute_instance.server can_ip_forward: "false" disk.#: "1"
disk.0.auto_delete: "true" disk.0.image: "ubuntu-1404-trusty-..." machine_type: "g1-small" metadata_fingerprint: "<computed>" name: "server" network_interface.#: "1" network_interface.0.address: "<computed>" network_interface.0.name: "<computed>" network_interface.0.network: "default" self_link: "<computed>" tags_fingerprint: "<computed>" zone: "us-central1-a" ... Plan: 3 to add, 0 to change, 0 to destroy. -
Apply • Executes changes in order based on dependencies •
Parallelizes changes when possible • Handles and recovers transient errors -
Terminal $ terraform apply google_compute_instance.server: Creating... can_ip_forward: "" => "false"
disk.#: "" => "1" disk.0.auto_delete: "" => "true" disk.0.image: "" => "ubuntu-1404-trusty..." machine_type: "" => "g1-small" metadata_fingerprint: "" => "<computed>" name: "" => "server" network_interface.#: "" => "1" network_interface.0.address: "" => "<computed>" network_interface.0.name: "" => "<computed>" network_interface.0.network: "" => "default" self_link: "" => "<computed>" tags_fingerprint: "" => "<computed>" zone: "" => "us-central1-a" ... Apply complete! Resources: 3 added, 0 changed, 0 destroyed. -
Apply for Changes • Not only creation, but changes over
time • Plan will show you what will happen • The `-target` flag can be used for fine-grained change -
Terraform 0.10 (August 2, 2017) • Split of Terraform Core
/ Providers • Kubernetes Provider • Workspaces renamed from State Environments -
Terraform Core vs Provider Core AWS Provider Fastly Provider HCL
Files API Integration Terraform -
Terraform Parts • Core Responsible for: • Parsing Configuration •
Resource Graph • Interpolation • State Management • Execution • Providers responsible for: • Resource Definition & Validation • API Integration -
Terraform Repository • Everything lived in a single repository •
Released ~2 weeks • Version Sprawl (10+ releases per major version) -
Mono Repo Issues • Terraform Core moves slowly • Weeks
to implement new features • Terraform providers move fast • 70+ Providers today • 1000+ Resources • Hours to implement new resources -
Mono Repo Issues • Hard to triage issues • 2000+
issues • Core Bugs to Provider Feature Requests • Hard to implement Core features • Hard to scale providers • Impossible to update Provider without updating Core -
Divide and Conquer • Grand Split proposed months ago •
Split Providers into individual repositories • Release Providers on independent cadence • Download Providers as needed • Version constrain Providers • "terraform init” -
File provider aws { version = "~> v0.1.3" region =
"us-west-2" } provider fastly { # Implicit: version = “latest” api_key = “exampleapikey" } -
Terminal $ terraform init Initializing provider plugins... - Checking for
available provider plugins on https:// releases.hashicorp.com... - Downloading plugin for provider "fastly" (0.1.2)... - Downloading plugin for provider "aws" (0.1.4)... The following providers do not have any version constraints in configuration, so the latest version was installed. To prevent automatic upgrades to new major versions that may contain breaking changes, it is recommended to add version = "..." constraints to the corresponding provider blocks in configuration, with the constraint strings suggested below. * provider.fastly: version = "~> 0.1" Terraform has been successfully initialized! You may now begin working with Terraform. Try running "terraform plan" to see any changes that are required for your infrastructure. All Terraform commands -
Terraform Core vs Provider Core 0.10 AWS Provider 0.1.4 Fastly
Provider 0.1.2 Plugin Interface -
End Goal • Independent Versioning and Release for Providers /
Core • Live documentation for multiple versions • Make it easier to add and support Providers • Infrastructure as Code for all the things! -
Kubernetes Provider • Terraform manages IaaS, SaaS, and PaaS •
Kubernetes has sub-resources to manage (Pods, Controllers) • Compose other resources (load balancers, DNS, CDN) with K8S • Common workflow for everything -
Kubernetes Provider • Data Source: kubernetes_service • Data Source: kubernetes_storage_class
• Resource: kubernetes_config_map • Resource: kubernetes_horizontal_pod_autoscalar • Resource: kubernetes_limit_range • Resource: kubernetes_namespace • Resource: kubernetes_persistent_volume • Resource: kubernetes_persistent_volume_claim • Resource: kubernetes_pod • Resource: kubernetes_replication_controller • Resource: kubernetes_resource_quota • Resource: kubernetes_secret • Resource: kubernetes_service • Resource: kubernetes_service_account • Resource: kubernetes_storage_class • … -
Using Schedulers with Terraform • Kubernetes • Nomad • Docker
Swarm • AWS ECS • Manage Application Lifecycle • Compose with other resources -
Workspaces • A state namespace • Allows single folder of
TF config to manage multiple distinct sets of infrastructure resources -
Terminal $ terraform workspace list * default $ terraform workspace
new mitchellh-test Created and switched to workspace "mitchellh-test"! $ terraform workspace list default * mitchellh-test -
File resource "aws_instance" "example" { count = “${terraform.workspace == "default"
? 5 : 1}" tags { Name = "web - ${terraform.workspace}" } # ... other fields } -
Terraform 0.9 (March 15, 2017) • Destroy provisioners • Remote
backends • State locking • Workspaces (formerly State Environments) -
Provisioners (Terraform <= 0.8) • Run arbitrary code locally or
remotely on resource creation • If provisioner fails, resource is tainted and scheduled for recreation on the next apply -
File resource "null_resource" "example" { provisioner "local-exec" { command =
"echo foo" } provisioner "local-exec" { command = "echo destroying" when = "destroy" } } -
Terminal $ terraform apply null_resource.example: Creating... null_resource.example: Provisioning with 'local-exec'...
null_resource.example (local-exec): Executing: /bin/sh -c "echo foo" null_resource.example (local-exec): foo null_resource.example: Creation complete (ID: 1965091882910923448) Apply complete! Resources: 1 added, 0 changed, 0 destroyed. -
Terminal $ terraform destroy -force null_resource.example: Refreshing state... (ID: 1965091882910923448)
null_resource.example: Destroying... (ID: 1965091882910923448) null_resource.example: Provisioning with 'local-exec'... null_resource.example (local-exec): Executing: /bin/sh -c "echo destroying" null_resource.example (local-exec): destroying null_resource.example: Destruction complete -
Destroy Provisioners • Configured with when = "destroy" • Run
on resource destroy (not just "terraform destroy") • Failure cancels physical resource destruction by default • Can allow failure with on_failure = "continue" -
File resource "null_resource" "example" { provisioner "local-exec" { command =
"exit 1" when = "destroy" } } -
Terminal $ terraform destroy -force null_resource.example: Refreshing state... (ID: 8665586891184105369)
null_resource.example: Destroying... (ID: 8665586891184105369) null_resource.example: Provisioning with 'local-exec'... null_resource.example (local-exec): Executing: /bin/sh -c "exit 1" Error applying plan: 1 error(s) occurred: * null_resource.example (destroy): 1 error(s) occurred: * Error running command 'exit 1': exit status 1. -
File resource "null_resource" "example" { provisioner "local-exec" { command =
"exit 1" when = "destroy" on_failure = "continue" } } -
Terminal $ terraform destroy -force null_resource.example: Refreshing state... (ID: 8665586891184105369)
null_resource.example: Destroying... (ID: 8665586891184105369) null_resource.example: Provisioning with 'local-exec'... null_resource.example (local-exec): Executing: /bin/sh -c "exit 1" null_resource.example: Destruction complete Destroy complete! Resources: 1 destroyed. -
Destroy Provisioners • Useful for resource cleanup • Can SSH
into machine (any machine!) prior to destruction • Recommend resource cleanup live as part of the resource itself, but destroy provisioners give you another option -
Before Remote Backends (TF <= 0.8) • Awkward "remote config"
command • Users could accidentally run Terraform without remote init • Configuration only via CLI • Local cache of state stored in .terraform/terraform.tfstate • Changed remote configuration was manual -
Terminal $ # TERRAFORM <= 0.8, BEFORE REMOTE BACKENDS $
terraform remote config \ -backend=S3 \ -backend-config="bucket=<bucket>" \ -backend-config="key=<path to file>" ... -
Remote Backends • Subsumes "remote state", enables locking, workspaces, more
• Configure from tf files, external configuration, or CLI • Detects configuration change • Forces new users of a TF configuration to initialize • One command to init them all: `terraform init` -
File terraform { backend "s3" { bucket = "<bucket>" key
= "<path to file>" } } -
Terminal $ terraform init Initializing the backend... Successfully configured the
backend "s3"! Terraform will automatically use this backend unless the backend configuration changes. Terraform has been successfully initialized! -
Terminal $ # New user, didn't run init $ terraform
console Backend reinitialization required. Please run "terraform init". Reason: Initial configuration of the requested backend "s3" ... -
File terraform { backend "s3" { bucket = "CHANGED-THIS-SETTING" key
= "<path to file>" } } -
Terminal $ terraform console Backend reinitialization required. Please run "terraform
init". Reason: Backend configuration changed for "s3" ... -
Terminal $ terraform console Backend reinitialization required. Please run "terraform
init". Reason: Unsetting the previously set backend "s3" ... -
Remote Backends • One command to init: `terraform init` •
Automatic detection of backend change (set, change, unset) • No state stored locally at all • Always gitignore ".terraform" folder -
A Focus on Safety • Common complaint: easy to corrupt
remote state • Remote backends add new layer of safety: detecting changes, checking "lineage", disallowing writing unsafe state, more. -
A New "Init" • Init has existed since Terraform 0.1
• Used to just setup folder structure for new projects • Now the single source of init, safe to run multiple times • Initializes backend, downloads modules, creates folders • One day: downloads providers, verifies versions, more... -
State Locking • For supported backends, Terraform automatically locks state
on write operations • If unlock fails, error is shown with lock ID to allow a force unlock • Doesn't lock against concurrent reads -
File terraform { backend "consul" {} } resource "null_resource" "example"
{ provisioner "local-exec" { command = "sleep 10" } } -
Terminal $ terraform apply null_resource.example: Creating... null_resource.example: Provisioning with 'local-exec'...
null_resource.example (local-exec): Executing: /bin/sh -c "sleep 10" -
Terminal $ terraform apply Error loading state: failed to lock
state in Consul: Lock Info: ID: 5c0b66d6-018f-59b4-5536-499fec947fb2 Path: foo Operation: OperationTypeApply Who: [email protected] Version: 0.9.1 Created: 2017-04-04 16:16:59.733058195 +0000 UTC Info: $ terraform console > -
SEP 18-20, 2017 | AUSTIN, TEXAS One day of training
followed by two days of talks on the future of infrastructure. Discount Code: HUG20