Managing GCP Projects with Terraform (devfest Pisa 2018)

Create and evolve simple and complex Google Cloud architectures with your text editor and Terraform.

Transcript

1. Managing GCP Projects with Terraform GDG DevFest Pisa 0.1 Giovanni Toraldo @gionn cloudesire.com
2. About me Giovanni Toraldo Open Source Enthusiast, Java coder, writer of the OpenNebula book, lead developer & co-founder at @Cloudesire, shooting 2 euro coin at 36 meters with crossbow 2
3. Monetization & Brokering Platform for immediate SaaSification and automated distribution of business applications and services. 3
5. Infrastructure as code? Design, implement, and deploy applications infrastructure with known software best practices: • Code versioning • Code reuse (modularization/abstraction) • Code sharing In order to achieve: • Repeatability • Speed • Reliability 5
6. Cloud services are easy? Complexity is increasing Composition is a must Velocity is the key of success 6
8. What is Terraform? Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. • Declarative approach ◦ Infrastructure is described using a high-level configuration syntax • Incremental changes ◦ Automatically discover the steps required from current to desired infrastructure state • Execution plan ◦ See what Terraform will do when you apply configuration • Dependency graph ◦ automatically decide the order in which action are executed • Automation 8
9. What is NOT Terraform? • Configuration management tool ◦ You still need Puppet, Chef, Ansible to manage software on VM • Cloud abstraction layer ◦ Doesn’t expose any API, just a CLI mean to be used by humans ◦ Doesn’t hide the inner characteristics of each cloud provider via abstraction • A solution to your lack of cloud knowledge 9
10. Use cases • Manage production environments by ops • Manage disposable test/qa environments by dev • Multi-tier (complex) infrastructures • Setup demo environments • Multi-cloud deployment 10
11. Install Terraform Terraform is distributed as a single Go binary without external dependencies. Download, unpack, execute. • cd /tmp • wget <url> • unzip <file> || tar xvf <file> • sudo mv terraform /usr/local/bin • sudo chmod +x /usr/local/bin/terraform 11
12. Create a Google Cloud Platform account 12
13. Create a new Google Cloud project 13
14. Navigate to Compute Engine to activate 14
15. Create Service account key and download 15
16. Create a new project Just create an empty folder with a file auth.tf: // Configure the Google Cloud provider provider "google" { credentials = "${file("account.json")}" project = "terraform-test-197317" region = "europe-west1 " } And run: $ terraform init 16
17. Initialize terraform project And run: $ terraform init 17
18. Apply changes to infrastructure Plan command to evaluate changes Apply command to modify infrastructure 18
19. Terraform project files structure There isn’t any enforcement on how to arrange resources inside a terraform project: • All .tf files in the current folder are automatically sources and merged together in memory $ ls -la total 108 drwxr-xr-x 4 gionn dev 4096 mar 8 15:47 . drwxr-xr-x 3 gionn dev 4096 mar 7 18:03 .. drwxr-xr-x 7 gionn dev 4096 mar 8 15:47 .git drwxr-xr-x 3 gionn dev 4096 mar 7 18:22 .terraform -rw-r--r-- 1 gionn dev 2333 mar 7 18:18 account.json -rw-r--r-- 1 gionn dev 175 mar 7 18:19 auth.tf -rw-r--r-- 1 gionn dev 406 mar 8 15:46 vm.tf 19
20. Create a new VM 20
21. Plan output $ terraform plan Refreshing Terraform state in-memory prior to plan... The refreshed state will be used to calculate this plan, but will not be persisted to local or remote state storage. --------------------------------------------- An execution plan has been generated and is shown below. 21
22. Plan output + create Terraform will perform the following actions: + google_compute_instance.default id: <computed> boot_disk.0.initialize_params.0.image: "debian-cloud/debian-8" cpu_platform: <computed> instance_id: <computed> label_fingerprint: <computed> machine_type: "n1-standard-1" metadata.%: "1" metadata.ssh-keys: "debian:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDnX70/3FB4j7UvewR3T... 22
23. Plan output Plan: 1 to add, 0 to change, 0 to destroy. ------------------------------------------------- - Note: You didn't specify an "-out" parameter to save this plan, so Terraform can't guarantee that exactly these actions will be performed if "terraform apply" is subsequently run. 23
24. Apply output $ terraform apply An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: + google_compute_instance.default 24
25. Apply output Summary of the actions to perform, waiting for confirmation Plan: 1 to add, 0 to change, 0 to destroy. Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value: ___ 25
26. After an apply successful operation State is persisted to terraform.tfstate: 26
28. terraform.tfstate A state file is required in order to: • Map managed resources to terraform resources • Persist additional metadata • Cache, useful for large infrastructures JSON structure that can be modified (bugs happens) or inspected (custom integrations). State should be persistent and shared among developers. 28
29. Terraform.tfstate sharing and locking For solo developers or small teams: just commit to GIT. For bigger teams, tfstate locking is required. Multiple backends supported: • Google cloud storage • S3 • Consul • Etcd • Terraform enterprise 29
30. Terraform.tfstate sample output { "version": 3, "terraform_version": "0.11.3", "serial": 2, "lineage": "d495f1c0-3c8b-45c2-bda2-f40d28382f30", "modules": [ { "path": [ "root" ], "outputs": {}, "resources": { "google_compute_instance.default": { "type": "google_compute_instance", "depends_on": [], "primary": { "id": "test-vm", "attributes": { "attached_disk.#": "0", "boot_disk.#": "1", "boot_disk.0.auto_delete": "true", "boot_disk.0.device_name": "persistent-disk-0", "boot_disk.0.disk_encryption_key_raw": "", "boot_disk.0.disk_encryption_key_sha256": "", 30
31. Terraform.tfstate: where is the IP address? "network_interface.#": "1", "network_interface.0.access_config.#": "1", "network_interface.0.access_config.0.assigned_nat_ip": "104.155.126.70", "network_interface.0.access_config.0.nat_ip": "104.155.126.70", "network_interface.0.address": "10.132.0.2", "network_interface.0.alias_ip_range.#": "0", "network_interface.0.name": "nic0", "network_interface.0.network": "https://www.googleapis.com/compute/v1/projects/terraform-test-19731 7/global/networks/default", "network_interface.0.network_ip": "10.132.0.2", "network_interface.0.subnetwork": "https://www.googleapis.com/compute/v1/projects/terraform-test-19731 7/regions/europe-west1/subnetworks/default", "network_interface.0.subnetwork_project": "terraform-test-197317", "project": "terraform-test-197317", 31
32. Automatically print VM ip address Add to output.tf: output "ip" { value = "${google_compute_instance.default. network_interface.0.access_config.0.nat_ip}" } And run terraform apply: Outputs: ip = 104.155.126.70 32
33. Change infrastructure example: VM upscale Just change the terraform attribute machine_type (and configure allow_stopping_for_update) Now, when terraform apply is run, terraform discover that the machine_type of the existing resource doesn’t correspond to the desiderata. 33
36. Attach a new data disk We’ll create a new data disk resource, and reference it insiude the instance resource. 36
40. 40 Manage multiple instances with disks • Define a variable resource ◦ Set a default • Define unique resource names to avoid conflicts ◦ Leverage count.index variable • Reference a difference disk for each instance ◦ Use count.index variable as disk reference • Override variable value via environment variable ◦ TF_VAR_my_counter
42. 42 github.com/ gionn/terraform-example
43. Questions? 43