Upgrade to Pro — share decks privately, control downloads, hide ads and more …

AWS CloudFormation Level-Up

Chuck Meyer
February 13, 2019

AWS CloudFormation Level-Up

CloudFormation is AWS' native infrastructure as code service. This presentation walks you through the process of authoring, deploying and testing your CloudFormation-driven AWS infrastructure, with opportunities to level-up your skill set along the way.

Chuck Meyer

February 13, 2019
Tweet

More Decks by Chuck Meyer

Other Decks in Technology

Transcript

  1. Who I am Chuck Meyer [email protected] Sr Developer Advocate, AWS

    CloudFormation 5+ years at AWS Major: Infrastructure as code and DevOps Minor: Security automation / DevSecOps 20+ Years in Technology Bass player @chuckm AWS CloudFormation Level-Up 4 / 42
  2. Infrastructure as code Declarative or imperative statements describing hardware, software

    and services and their relationships. AWS CloudFormation Level-Up 5 / 42
  3. Infrastructure as code Declarative or imperative statements describing hardware, software

    and services and their relationships. Resource: MyWebServer Class: Server Type: ExtraBig Ports: - 443 AWS CloudFormation Level-Up 5 / 42
  4. Infrastructure as code Declarative or imperative statements describing hardware, software

    and services and their relationships. Resource: MyWebServer Class: Server Type: ExtraBig Ports: - 443 server_names = [ 'Red', 'Blue', 'Green'] for name in server_names: launch_server(name, 'web') AWS CloudFormation Level-Up 5 / 42
  5. Infrastructure as code Single source of truth for provisioning and

    configuration Infrastructure that you can replicate, re-deploy, and re-purpose Control versioning on your infrastructure and your application together Roll back to the last good state on failures Build and deploy your infrastructure through your CI/CD pipeline AWS CloudFormation Level-Up 6 / 42
  6. A simplified way to create and manage a collection of

    AWS resources Enables orderly and predictable provisioning and updating of resources Fully managed service Integrates with the AWS Management Console, the AWS Command Line Interface (CLI), or AWS APIs Only pay for the resources you create AWS CloudFormation AWS CloudFormation Level-Up 7 / 42
  7. CloudFormation at a glance Enables provisioning and management of your

    infrastructure as code AWS CloudFormation Level-Up 8 / 42
  8. JSON JavaScript Object Notation Attribute/Value pairs Similar to XML Designed

    to be machine readable CloudFormation syntax AWS CloudFormation Level-Up 10 / 42
  9. YAML YAML ain't a markup language Human readable data serialization

    standard Comments (use #) No } or ; CloudFormation syntax AWS CloudFormation Level-Up 11 / 42
  10. Template Anatomy 1. Format version 2. Transforms 3. Description 4.

    Metadata 5. Parameters 6. Mappings 7. Conditions 8. Resources* (required) 9. Outputs CloudFormation syntax https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-anatomy.html AWS CloudFormation Level-Up 12 / 42
  11. --- AWSTemplateFormatVersion: '2010-09-09' Description: Create an EC2 instance running the

    latest Amazon Linux AMI. Parameters: KeyPair: Description: The EC2 Key Pair to allow SSH access to the instance Type: String Resources: Ec2Instance: Properties: ImageId: ami-9d23aeea InstanceType: m3.medium KeyName: !Ref 'KeyPair' Type: AWS::EC2::Instance Outputs: InstanceId: Description: The InstanceId of the newly created EC2 instance Value: !Ref 'Ec2Instance' AWS CloudFormation Level-Up 13 / 42
  12. Simple template – create EC2 instance Parameters: KeyPair: Description: 'The

    EC2 Key Pair to allow SSH access to the instance' Type: 'AWS::EC2::KeyPair::KeyName' You enter a value for the KeyPair parameter when you create your stack. AWS CloudFormation Level-Up 14 / 42
  13. Simple template – create EC2 instance Resources: Ec2Instance: Type: 'AWS::EC2::Instance'

    Properties: ImageId: 'ami-9d23aeea' InstanceType: 'm3.medium' KeyName: !Ref 'KeyPair' Includes statically defined properties (ImageId and InstanceType) and a reference to the KeyPair parameter. ImageId is the AMI specific to the region that you want to launch this stack in (eu- west-1 region in this example). AWS CloudFormation Level-Up 15 / 42
  14. Simple template – create EC2 instance Outputs: InstanceId: Description: 'The

    InstanceId of the newly created EC2 instance' Value: !Ref 'Ec2Instance' These outputs are returned after the template has completed execution. AWS CloudFormation Level-Up 16 / 42
  15. CloudFormation syntax – Resources The only section of the template

    that is required AWS services that will be created, updated, or deleted from your account Supports 342 resource types (and growing) Resources: Ec2Instance: Type: 'AWS::EC2::Instance' Properties: ImageId: 'ami-9d23aeea' InstanceType: 'm3.medium' KeyName: !Ref 'KeyPair' AWS CloudFormation Level-Up 17 / 42
  16. CloudFormation syntax – Parameters Enable you to input custom values

    to your template each time you create or update a stack with input validation and restrictions Parameter types: String, Number, List<Number>, CommaDelimitedList, Parameter Store values, and AWS-specific types (AWS::EC2::Image::Id, AWS::Route53::HostedZone::Id) Use the Ref and Fn::Sub intrinsic functions to reference parameters Pseudo-Parameters are predefined by AWS CloudFormation and used just like normal parameters (AWS::Region) AWS CloudFormation Level-Up 18 / 42
  17. Intrinsic functions Basic programmatic functions available in-line for your declarative

    templates. Retrieve external values (Ref, Fn::Sub, Fn::FindInMap, Fn::GetAtt, Fn::GetAZs) Manipulate strings (Fn::Sub, Fn::Split, Fn::Join, Fn::Base64, Fn::Transform) Conditional logic (Fn::If, Fn::Equals, Fn::Not) AWS CloudFormation Level-Up 19 / 42
  18. Dynamic references Inject values from SSM Parameter Store and Secrets

    Manager KMS encrypted strings Versioned and secured by IAM MyIAMUser: Type: AWS::IAM::User Properties: UserName: 'MyUserName’ LoginProfile: Password: '{{resolve:ssm-secure:IAMUserPassword:10}}' AWS CloudFormation Level-Up 20 / 42
  19. CloudFormation syntax - Conditions Resource creation can depend on logical

    conditions: Conditions: isProd: Fn::Equals [ !Ref EnvType, prod ] Resources: EC2Instance: Type: "AWS::EC2::Instance" Condition: isProd Properties: ImageId: Fn::FindInMap [RegionMap, !Ref "AWS::Region", AMI] You can use conditions with intrinsic functions (Fn::If, Fn::Equals, Fn::Not) to create complex logic for property values. AWS CloudFormation Level-Up 21 / 42
  20. CloudFormation syntax - Outputs Outputs from successful operations View them

    in the console or pass them along as inputs to other stacks Used with nested stacks and cross stack references Outputs: Environment: Description: 'Environment type' Value: Fn:If: [ isProd, 'Production', 'Development' ] AWS CloudFormation Level-Up 22 / 42
  21. Authoring Level-up Use a linter (cfn-python-lint) Decompose architecture by lifecycle

    (short vs. long lived) AWS CloudFormation Level-Up 23 / 42
  22. Authoring Level-up Use a linter (cfn-python-lint) Decompose architecture by lifecycle

    (short vs. long lived) Isolate stateful resources (databases, caches) AWS CloudFormation Level-Up 23 / 42
  23. Authoring Level-up Use a linter (cfn-python-lint) Decompose architecture by lifecycle

    (short vs. long lived) Isolate stateful resources (databases, caches) Don't write, Recycle! AWS CloudFormation Level-Up 23 / 42
  24. Authoring Level-up Use a linter (cfn-python-lint) Decompose architecture by lifecycle

    (short vs. long lived) Isolate stateful resources (databases, caches) Don't write, Recycle! AWS Quick Starts Documentation examples AWS CloudFormation Level-Up 23 / 42
  25. Authoring Level-up Use a linter (cfn-python-lint) Decompose architecture by lifecycle

    (short vs. long lived) Isolate stateful resources (databases, caches) Don't write, Recycle! AWS Quick Starts Documentation examples Use DSLs or the CDK (Troposphere, Sparkleformation, GoFormation) AWS CloudFormation Level-Up 23 / 42
  26. CloudFormation uses your template as a blueprint to provision resources

    into a construct called a stack. On create, the CloudFormation service: 1. Retrieves template from S3 (or API) 2. Parses template and validates parameters 3. Provisions resources in parallel or serial based on dependencies 4. Waits for resources to stabilize 5. Populates outputs and signals completion -or- Rolls back and signals failure Creating a stack AWS CloudFormation Level-Up 25 / 42
  27. CloudFormation evaluates your changes to the template or parameters against

    the last known state of the provisioned resources in the stack. At then modifies the resources as needed. On update, the CloudFormation service: 1. Retrieves template from S3 (or API) 2. Compares the template ands parameters to the last known state 3. Changes resources in place or creates new immutable resources 4. Waits for resources to stabilize 5. Updates outputs and signals completion -or- Rolls back and signals failure Updating a stack Deleting a stack AWS CloudFormation Level-Up 26 / 42
  28. CloudFormation change sets Preview the impact to your stack of

    changes by comparing the new template and parameters to the last known state of the stack. CloudFormation makes the changes to your stack only when you decide to execute the change set. AWS CloudFormation Level-Up 27 / 42
  29. Network Stack Outputs: VPC Description: 'VPC ID' Value: !Ref 'VPC'

    Export: Name: 'ProdVPC' =======> App Stack Resources: myTargetGroup: Type: 'AWS::ELBV2::TargetGroup' Properties: VpcId: Fn::ImportValue: 'ProdVPC' Cross-stack references (Exports) Allows you to share information between independent stacks. Export a stack’s output values. Other stacks in the same account and region can import the exported values. AWS CloudFormation Level-Up 28 / 42
  30. Application Resources: NetworkResources: Type: 'AWS::CloudFormation::Stack' ContainerResources: Type: 'AWS::CloudFormation::Stack' Network Resources

    Resources: MyVPC Type: 'AWS::EC2::VPC' Create a hieararchy of stacks composed of multiple templates. Re-use templates with frequently used resources. Reference resources across stacks. Nested stacks AWS CloudFormation Level-Up 29 / 42
  31. Drift detection Compares the last known state of the stack

    to current resource configurations. Shows if configuration changes were made to your stack resources outside of CloudFormation. AWS CloudFormation Level-Up 30 / 42
  32. CloudFormation StackSets Create, update, and delete stacks in multiple accounts

    and regions using a single operation AWS CloudFormation Level-Up 31 / 42
  33. Deployment Level-up Build in guard rails Termination protection Stack policies

    UpdateReplace and Deletion policies IAM AWS CloudFormation Level-Up 32 / 42
  34. Deployment Level-up Build in guard rails Termination protection Stack policies

    UpdateReplace and Deletion policies IAM Use an orchestration tool to promote environments Jenkins, CodePipeline AWS CloudFormation Level-Up 32 / 42
  35. Deployment Level-up Build in guard rails Termination protection Stack policies

    UpdateReplace and Deletion policies IAM Use an orchestration tool to promote environments Jenkins, CodePipeline Use other services to manage configuration SSM Parameter Store or Secrets Manager AWS CloudFormation Level-Up 32 / 42
  36. Deployment Level-up Build in guard rails Termination protection Stack policies

    UpdateReplace and Deletion policies IAM Use an orchestration tool to promote environments Jenkins, CodePipeline Use other services to manage configuration SSM Parameter Store or Secrets Manager Use changesets whenever possible AWS CloudFormation Level-Up 32 / 42
  37. Infrastructure as is code! Template code should be in a

    repo Track issues and history Commits can trigger test suites and builds Use tools and utilities for validation Hook into Jenkins, Bamboo, Ansible, Chef, Puppet... AWS CloudFormation Level-Up 34 / 42
  38. The challenge If this is ”infrastructure as code” why are

    we still testing by continually deploying and fixing failures? Our goal: Catch errors early to reduce authoring time. AWS CloudFormation Level-Up 35 / 42
  39. We are living in a golden age of tools cfn-lint

    Validate AWS CloudFormation yaml/json templates against the AWS CloudFormation spec and additional checks cfn-nag Look for patterns in templates that may indicate insecure infrastructure. Taskcat Catch problems that aren’t obvious in a single template/stack AWS CloudFormation Level-Up 36 / 42
  40. cfn-lint "Can I deploy this template?" Community-driven open source tool

    to validate CloudFormation YAML/JSON templates against the CloudFormation resource specification + additional checks. IDE plugins (VS Code, Atom, Sublime, IntelliJ, vim) https://github.com/awslabs/cfn-python-lint pip install cfn-lint AWS CloudFormation Level-Up 37 / 42
  41. cfn-nag "Should I deploy this template?" Looks for patterns in

    CloudFormation templates that may indicate insecure infrastructure. IAM rule wildcards Security group wildcards Access logs that aren't enabled Encryption that isn't enabled https://github.com/stelligent/cfn_nag gem install cfn-nag AWS CloudFormation Level-Up 38 / 42
  42. taskcat "Will this template deploy everywhere?" Catches problems that aren’t

    obvious in a single template/stack. Tests your templates by creating stacks in multiple AWS regions simultaneously. Generates a report with a pass/fail grade for each region AWS CloudFormation Level-Up 39 / 42
  43. Testing Level-up Static analysis saves time (cfn-lint, cfn_nag)... ...but can't

    catch everything (taskcat) AWS CloudFormation Level-Up 41 / 42
  44. Testing Level-up Static analysis saves time (cfn-lint, cfn_nag)... ...but can't

    catch everything (taskcat) Shorter templates are easier to test and maintain AWS CloudFormation Level-Up 41 / 42
  45. Testing Level-up Static analysis saves time (cfn-lint, cfn_nag)... ...but can't

    catch everything (taskcat) Shorter templates are easier to test and maintain More reuse == less testing AWS CloudFormation Level-Up 41 / 42