Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Creating a Terraform Provider for Just About An...

Creating a Terraform Provider for Just About Anything

Terrafom is an amazing tool that lets you define your infrastructure as code. Under the hood it's an incredibly powerful state machine that makes API requests and marshals resources. In this talk we'll dive into the inner workings of Terraform and examine all the elements of a provider — from the documentation to the test suite. You'll walk away with the knowledge of how to contribute to an existing provider or create your own from scratch. We'll also take a look at some of the things we encountered while working on the DigitalOcean provider and the lessons learned from the community.

https://youtu.be/noxwUVet5RE

Eddie Zaneski

October 23, 2018
Tweet

More Decks by Eddie Zaneski

Other Decks in Technology

Transcript

  1. provider "digitalocean" { token = "DIGITALOCEAN_API_TOKEN" } resource "digitalocean_droplet" "web"

    { name = "web" region = "sfo1" size = "s-1vcpu-1gb" image = "ubuntu-18-04-x64" }
  2. provider "kubernetes" { config_context_auth_info = "ops" config_context_cluster = "mycluster" }

    resource "kubernetes_service" "dope_web_server" { metadata { name = "dope_web_server" labels = { app = "web" } } spec { type = "LoadBalancer" selector { app = "${kubernetes_pod.web.metadata.0.labels.app}" } port { port = 8080 target_port = 80 } } }
  3. provider "github" { token = "${var.github_token}" organization = "${var.github_organization}" }

    resource "github_repository" "codez" { name = "uber-for-tacos" description = "My new startup" private = true }
  4. provider "googlecalendar" {} resource "googlecalendar_event" "talk" { summary = "This

    Terraform talk" location = "Gold Room" start = "2018-10-23T14:35:00-08:00" end = "2018-10-23T15:10:00-08:00" attendee { email = "[email protected]" } } @sethvargo https://www.hashicorp.com/blog/managing-google-calendar-with-terraform
  5. provider "hue" { token = "YOUR_HUE_TOKEN" bridge_ip = "192.168.1.137" }

    data "hue_light" "kitchen" { light_id = "1" } resource "hue_light_state" "kitchen" { light_id = "${data.hue_light.kitchen.light_id}" brightness = 42 red = 246 blue = 74 green = 138 }
  6. provider "todoist" { api_key = "TODOIST_API_KEY" } resource "todoist_task" "pack"

    { content = "Pack to move to Denver" due_string = "Friday" responsible_uid = "my_wife" completed = false } https://github.com/eddiezane/terraform-provider-todoist https://github.com/eddiezane/todoist-rest-go
  7. Your Company ◦ You're a cloud provider ◦ Your customers

    ask for one ◦ You have a resource in an API that makes sense to manage with Terraform ◦ Skip a front end ◦ Don't need to write duct tape apps ◦ Internal
  8. Maintain a Fork ◦ Org specific defaults or needs ◦

    Missing features ◦ Unresponsive maintainer
  9. Go API Client ◦ Separate your API logic from your

    provider logic ◦ Error handling ◦ Logging
  10. return &schema.Provider{ Schema: map[string]*schema.Schema{ "api_key": &schema.Schema{ Type: schema.TypeString, Description: "Your

    Todoist API key", Required: true, DefaultFunc: schema.EnvDefaultFunc("TODOIST_API_KEY", nil), }, }, ConfigureFunc: configureFunc(), ... } Schema o Used in all the things o Type enforcement
  11. "Terraform stdlib" ◦ https://github.com/hashicorp/terraform/tree/master/helper ◦ Read it! ◦ schema ◦

    Provider ◦ Resource ◦ ResourceData ◦ Validators ◦ IntBetween ◦ Testing ◦ RandString(strlen int)
  12. func configureFunc() func(*schema.ResourceData) (interface{}, error) { return func(d *schema.ResourceData) (interface{},

    error) { client := todoistRest.NewClient(d.Get("api_key").(string)) return client, nil } } ConfigureFunc (meta) o Return the meta
  13. func resourceTodoistTask() *schema.Resource { return &schema.Resource{ Schema: map[string]*schema.Schema{ "content": &schema.Schema{

    Type: schema.TypeString, Required: true, }, "completed": &schema.Schema{ Type: schema.TypeBool, Optional: true, Default: false, }, }, Create: resourceTodoistTaskCreate, Read: resourceTodoistTaskRead, Update: resourceTodoistTaskUpdate, Delete: resourceTodoistTaskDelete, } }
  14. func resourceTodoistTaskCreate(d *schema.ResourceData, meta interface{}) error { client := meta.(*todoistRest.Client)

    content := d.Get("content").(string) newTask := &todoistRest.NewTask{ Content: content, } task, err := client.CreateTask(newTask) if err != nil { return err } d.SetId(task.Id) return resourceTodoistTaskRead(d, meta) }
  15. func resourceTodoistTaskRead(d *schema.ResourceData, meta interface{}) error { client := meta.(*todoistRest.Client)

    id := d.Id() task, err := client.GetTask(id) if err != nil { return err } ... d.Set("content", t.Content) d.Set("completed", t.Completed) return nil }
  16. func resourceTodoistTaskUpdate(d *schema.ResourceData, meta interface{}) error { client := meta.(*todoistRest.Client)

    id := d.Id() if d.HasChange("content") { content := d.Get("content").(string) err := client.UpdateTask(... } return nil }
  17. func resourceTodoistTaskDelete(d *schema.ResourceData, meta interface{}) error { client := meta.(*todoistRest.Client)

    id := d.Id() err := client.DeleteTask(id) if err != nil { return err } return nil }
  18. ◦ Extremely robust to failures and fault tolerant ◦ Log

    all the things! ◦ TF_LOG=INFO terraform plan ◦ Quickly identify what's your API's fault vs Terraform vs random ◦ Partial state If the Create callback returns with or without an error and an ID has been set, the resource is assumed created and all state is saved with it. Error Handling https://www.terraform.io/docs/extend/writing-custom-providers.html#error-handling-amp-partial-state
  19. func TestAccTodoistTask_basic(t *testing.T) { var task todoistRest.Task content := "test

    task content" resource.Test(t, resource.TestCase{ PreCheck: func() { testAccPreCheck(t) }, Providers: testAccProviders, CheckDestroy: testAccCheckTodoistTaskDestroy, Steps: []resource.TestStep{ { Config: testAccCheckTodoistTaskConfig_basic(content), Check: resource.ComposeTestCheckFunc( testAccCheckTodoistTaskExists("todoist_task.test", &task), testAccCheckTodoistTaskAttributes(&task, content), resource.TestCheckResourceAttr("todoist_task.test", "content", content), ), }, }, }) }
  20. ◦ Precheck > Test Steps > Destroy > CheckDestroy ◦

    One config per step ◦ Use steps to simulate updates and deletes ◦ Reuse as much as possible—especially early on ◦ Makefile ◦ make testacc TESTARGS='-run=TestAccDigitalOceanDomain_Basic' Testing
  21. Process Notes ◦ Engage HashiCorp early ◦ Travis CI for

    CI/CD ◦ Releases via Slack ◦ TeamCity for releasing https://www.terraform.io/guides/terraform-provider-development-program.html
  22. Tips ◦ Learn a bit of Go first ◦ Get

    familiar with `strconv` ◦ Custom `UnmarshalJSON` functions ◦ Have a solid Go API client library ◦ Fix design problems with your API if possible ◦ Understand Terraform Modules ◦ Don't be afraid to copy/pasta ◦ Focus on composability and reusable functions ◦ Sweepers https://www.terraform.io/guides/terraform-provider-development-program.html
  23. Resources ◦ Other providers ◦ Terraform docs, guides, and source

    code ◦ Videos ◦ How to Extend the Terraform Provider List - Paul Stack ▪ https://youtu.be/2BvpqmFpchI ◦ Terraform and The Extensible Provider Architecture - Clint Shryock ▪ https://youtu.be/TMmovxyo5sY ◦ OpenCredo