Share
Embed
TER301: Terraforming Safely and Collaboratively on AWS with Atlantis
Presented during AWS Summit 2019 Developer Lounge and AWS User Group Meetup May 2019
Steve Teo
April 10, 2019
Transcript
62 slides
-
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T TER301: Terraforming AWS safely and collaboratively with Atlantis Steve Teo Director of Cloud Security Engineering Horangi Cyber Security www.linkedin.com/in/steveteo D e v e l o p e r L o u n g e -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Steve ”Potay” Teo • CloudDevSecOps Fanatic • Currently building products at • Progression => UI/UX Designer / Developer => Full Stack Developer => Build & Release Engineer => Infrastructure & Tools Architect • AWS Areas of Interests: AWS Multi-Account Strategy, Cloud Security • Totally uncertified and proud :P -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Communities I serve www.meetup.com/AWS-SG/ www.meetup.com/ Atlassian-User-Group-Singapore/ -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T AWS User Group Singapore - Monthly -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T AWS User Group Singapore - Monthly -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T AWS User Group Singapore - Monthly -
Before we start, a quick poll. How many of you
know what Terraform is? How many of you are already using Terraform? How many of you are using Terraform in a team setting? -
If you start using Terraform alone, it is definitely going
to look like this. -
However, if you are starting to use Terraform in a
team setting, it is definitely going to look like this. -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T What this talk is about • Quick Introduction to Terraform • Challenges around Terraform State Management and Collaboration • How Atlantis solves these challenges safely and collaboratively • Demo What this talk is not about • Every other advanced Terraform topic • Terraform vs CloudFormation • Every Little Atlantis Feature! RTFM -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Providers • Multi-Provider Orchestration • Abstracts APIs as resources • Context for resources -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Providers • Multi-Provider Orchestration • Abstracts APIs as resources • Context for resources aws_instance aws_ec2_transit_gateway aws_db_instance -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Providers • Multi-Provider Orchestration • Abstracts APIs as resources • Context for resources -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Write Desired State • Think of how the world should be • Infrastructure as Code • Versionable • Specify dependencies -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Terraform Plan • Preview changes before applying • Reduces mistakes & uncertainty • Map resource dependencies -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Terraform Plan • Preview changes before applying • Reduces mistakes & uncertainty • Map resource dependencies -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Terraform Apply • Reach the desired state • Live result log • Safe Orchestration -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T One Simple Workflow -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Desired State vs Current State -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Desired State vs Current State -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T State File -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T State file is Sacred! • Real world mapping of resources • Track metadata of resource and its dependency with respect to other resources • State is exposed - Protect your state file at all cost! • Teams collaborate via state files -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Local Terraform Usage Pros • Simple Issues • No backup of state files • Not even possible to collaborate in a team setting • Different developers have different Terraform environments • Doesn’t integrate into a CI/CD workflow -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Local Terraform Usage Pros • Simple Issues • No backup of state files • Not even possible to collaborate in a team setting • Different developers have different Terraform environments • Doesn’t integrate into a CI/CD workflow -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T { Ops } { Dev } { Ops } { Dev } { Dev } Solo Development -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Remote Backend: S3 • Durable Storage • Enable Bucket Versioning • DynamoDB Locking - State can be accessed safely -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Remote Backend: S3 • Durable Storage • Enable Bucket Versioning • DynamoDB Locking - State can be accessed safely -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Remote Backend Terraform Usage Cons • No backup of state files • Not even really possible to collaborate in a team setting • Different developers have different Terraform environments • Doesn’t integrate into a CI/CD workflow our -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T { Ops } { Dev } { Ops } { Dev } { Dev } Now only Ops can perform change -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T { Ops } { Dev } { Ops } { Dev } { Dev } Now only Ops can perform change -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Workflow before Atlantis -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Atlantis • Started in HootSuite by Anubhav Mishra and currently maintained by Luke Kysow • Battle-tested at HootSuite with 100 developers working on 600 Terraform Repositories • Self-hosted • Open-source, supported by Hashicorp • Well written docs • Active maintainer -
Atlantis monitors git pull requests and centralizes Terraform Plans and
Applies and enforces collaboration through pull requests -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Monitors git pull requests -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Monitors git pull requests -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Centralizes Terraform Plans and Applies -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Centralizes Terraform Plans and Applies -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Centralizes Terraform Plans and Applies -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Centralizes Terraform Plans and Applies • Issue: Pull requests can become really noisy through multiple plan iterations • Solution: Iterate locally against a Sandbox account or against read only state in s3 bucket -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Enforces collaboration through pull requests • Pull Request Approval Check • Merge-ability Check -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Enforces collaboration through pull requests • Pull Request Approval Check • Merge-ability Check -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Enforces collaboration through pull requests -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Standardized Workflow after Atlantis -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T { Ops } { Dev } { Ops } { Dev } { Dev } Developers Writing Infrastructure Code -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Developers Writing Infrastructure Code { Ops } { Dev } { Ops } { Dev } { Dev } -
S U M M I T © 2019, Amazon Web
Services, Inc. or its affiliates. All rights reserved. -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T Gotchas I Found - Approval / mergability checks doesn’t really work in solo based scenarios in a team based setting - Works in a very plan / apply centric model. No out of the box support for other terraform state manipulation commands, or even destroy. All this needs to be handled - Security models for working within Atlantis workflows are still being worked out, eg. https://github.com/runatlantis/atlantis/issues/308. Definitely evaluate this very carefully if you are using this for production system within a low-trust team model -
S U M M I T © 2019, Amazon Web
Services, Inc. or its affiliates. All rights reserved. -
S U M M I T © 2019, Amazon Web
Services, Inc. or its affiliates. All rights reserved. -
© 2019, Amazon Web Services, Inc. or its affiliates. All
rights reserved. S U M M I T References • www.speakerdeck.com/anubhavmishra/taming-terraform-workflow- using-terraform-modules-and-github-1?slide=67 • www.speakerdeck.com/anubhavmishra/taming-infrastructure- workflow-at-scale-with-terraform • www.medium.com/runatlantis/introducing-atlantis-6570d6de7281 • www.medium.com/runatlantis/putting-the-dev-into-devops-why- your-developers-should-write-terraform-too-d3c079dfc6a8 • https://medium.com/runatlantis/moving-atlantis-to-runatlantis- atlantis-on-github-4efc025bb05f -
Thank you! S U M M I T © 2019,
Amazon Web Services, Inc. or its affiliates. All rights reserved. Steve Teo Director of Cloud Security Engineering Horangi Cyber Security www.linkedin.com/in/steveteo -
S U M M I T © 2019, Amazon Web
Services, Inc. or its affiliates. All rights reserved.