Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Infrastructure IS Code on AWS

Darko Mesaros
September 10, 2020

Infrastructure IS Code on AWS

In the age of The Cloud, Serverless, Containers and Microservices, our infrastructure is part of our application more than ever. So why do we still treat it as something else? Our Application is Code, our Business Logic is code, well Infrastructure also IS code. One of the parts of doing things properly at scale is being able to describe your infrastructure as code and deploy it as such. If we already treat our infrastructure as code, why not apply all the best practices of software delivery to infrastructure delivery. In this session we look into Infrastructure as Code solutions, best practices and patterns on AWS.

Darko Mesaros

September 10, 2020
Tweet

More Decks by Darko Mesaros

Other Decks in Technology

Transcript

  1. © 2020, Amazon Web Services, Inc. or its Affiliates. Infrastructure

    IS Code on AWS Darko Meszaros Developer Advocate – Amazon Web Services @darkosubotica ln/darko-mesaros twitch.tv/ruptwelve youtube.com/ruptwelve
  2. © 2020, Amazon Web Services, Inc. or its Affiliates. Agenda

    for today • What is Infrastructure as Code • AWS CloudFormation • AWS Serverless Application Model (SAM) • AWS Cloud Development Kit (CDK) • Other Tools out there • Some Best Practices • Wrap up
  3. © 2020, Amazon Web Services, Inc. or its Affiliates. Infrastructure

    as code ✓ Make infrastructure changes repeatable and predictable ✓ Release infrastructure changes using the same tools as code changes ✓ Replicate production in a staging environment to enable continuous testing
  4. © 2020, Amazon Web Services, Inc. or its Affiliates. Infrastructure

    as code Declarative I tell you what I need I tell you what to do Imperative
  5. © 2020, Amazon Web Services, Inc. or its Affiliates. $(whoami)

    Darko Mesaroš / Darko Meszaros / Дарко Месарош ! → " → # → $ → % Berlin ! @darkosubotica ln/darko-mesaros twitch.tv/ruptwelve youtube.com/ruptwelve
  6. © 2020, Amazon Web Services, Inc. or its Affiliates. AWS

    CloudFormation • Infrastructure as code (IaC) • Provides a common language for you to describe and provision all the infrastructure resources in your cloud environment • Build and rebuild your infrastructure and applications, without having to perform manual actions or write custom scripts https://aws.amazon.com/cloudformation/
  7. © 2020, Amazon Web Services, Inc. or its Affiliates. Sample

    AWS CloudFormation code • Code is written in files called templates • A stack is generated from a template • Templates primarily define resources for an application • AWS CloudFormation can create over 490 types of resources • Each resource is configured based on its available properties • Dependencies can be explicitly declared or implicitly discovered AWSTemplateFormatVersion: "2010-09-09" Description: A CodeCommit Repo and Cloud9 Environment Resources: MyRepo: Type: "AWS::CodeCommit::Repository" Properties: RepositoryName: MyRepo RepositoryDescription: Sample Repository for Demo MyC9Environment: Type: "AWS::Cloud9::EnvironmentEC2" Properties: Repositories: - PathComponent: /cfn RepositoryUrl: !GetAtt MyRepo.CloneUrlHttp InstanceType: t2.micro
  8. © 2020, Amazon Web Services, Inc. or its Affiliates. Anatomy

    of an AWS CloudFormation template • Resources • Parameters and Mappings • Conditions • Outputs
  9. © 2020, Amazon Web Services, Inc. or its Affiliates. Testing

    via pipelines • As you would with other application code, templates should be version controlled and tested via CI/CD pipelines • The linter can be run in an AWS CodeBuild step to ensure that teams comply with rules and standards • Additional tools, like taskcat (available on GitHub), allow tests across regions Git push Templates AWS CodeCommit AWS CodePipeline AWS CodeBuild AWS CloudFormation Region AWS CodeDeploy
  10. © 2020, Amazon Web Services, Inc. or its Affiliates. CloudFormation

    registry Open CLI Open providers Introducing the AWS CloudFormation registry An open approach to managing external resources
  11. © 2020, Amazon Web Services, Inc. or its Affiliates. AWS

    CloudFormation registry and CLI • Allows AWS CloudFormation to support native and non-AWS resources while inheriting many core benefits like rollbacks • Use the AWS CloudFormation CLI tool to create resource providers using JSON schema-driven development, generating many of the code assets for you • Use third-party resource providers as you would use native AWS resource types
  12. © 2020, Amazon Web Services, Inc. or its Affiliates. Model

    function environments with AWS Serverless Application Model (SAM) • Open source framework for building serverless applications on AWS • Shorthand syntax to express functions, APIs, databases, and event source mappings • Transforms and expands SAM syntax into AWS CloudFormation syntax on deployment • Supports all AWS CloudFormation resource types https://aws.amazon.com/serverless/sam/
  13. © 2020, Amazon Web Services, Inc. or its Affiliates. SAM

    template AWSTemplateFormatVersion: '2010-09-09’ Transform: AWS::Serverless-2016-10-31 Resources: MySimpleTableFunction: Type: AWS::Serverless::Function Properties: Handler: mySimpleTableFunction.handler Runtime: nodejs12.x CodeUri: ./functions Policies: - DynamoDBReadPolicy: TableName: !Ref MySimpleTable Events: MySimpleFunctionApi: Type: Api Properties: Path: /simpleTable Method: GET MySimpleTable: Type: AWS::Serverless::SimpleTable Just 20 lines to create: • Lambda function • IAM role • API Gateway
  14. © 2020, Amazon Web Services, Inc. or its Affiliates. AWS

    SAM CLI • Create, build, test, and deploy AWS SAM applications • Step-through debugging and IDE support • Open source! • https://github.com/awslabs/aw s-sam-cli
  15. © 2020, Amazon Web Services, Inc. or its Affiliates. Use

    SAM CLI to package and deploy SAM templates pip install --user aws-sam-cli # Or even better use native installers sam init --name my-app --runtime python cd my-app/ sam local ... # generate-event/invoke/start-api/start-lambda sam validate # The SAM template sam build # Depending on the runtime sam package --s3-bucket my-packages-bucket \ --output-template-file packaged.yaml sam deploy --template-file packaged.yaml \ --stack-name my-stack-prod sam logs -n MyFunction --stack-name my-stack-prod -t # Tail sam publish # To the Serverless Application Repository CodePipeline Use CloudFormation deployment actions with any SAM application Jenkins Use SAM CLI plugin O pen Source
  16. © 2020, Amazon Web Services, Inc. or its Affiliates. Shorten

    the learning curve with AWS Cloud Development Kit (CDK) Late 2020 Brings cloud infrastructure to developers in ways the can understand • Build cloud infrastructure with the languages they already know • Use their existing tools and workflows • Helpful abstractions that remove the need to learn the details • Vibrant and fast-growing community of developers
  17. © 2020, Amazon Web Services, Inc. or its Affiliates. AWS

    Cloud Development Kit (AWS CDK) A multi-language development framework for modeling infrastructure as reusable components
  18. © 2020, Amazon Web Services, Inc. or its Affiliates. How

    do we do testing with CDK? • Snapshot tests • Fine-grained assertions • Validation tests npm install --save-dev jest @types/jest @aws-cdk/assert
  19. © 2020, Amazon Web Services, Inc. or its Affiliates. Best

    practices (1/5) • Layer your application to reduce blast radius when updating resources • Use multiple, isolated environments for testing, production, development, staging, etc. • Smaller files are easier to write, test, and troubleshoot Instances, Auto Scaling groups API endpoints, functions Alarms, dashboards VPCs, NAT gateways, VPNs, subnets IAM users, groups, roles, policies Front-end resources Backend services Stateful resources Base network Identity & security Monitoring resources Databases and clusters, queues
  20. © 2020, Amazon Web Services, Inc. or its Affiliates. Quotes

    “Please for the love of everything you hold dear, separate critical persistent storage from the rest of your IAC so you don’t accidentally remove it. Please.” - An Infrastructure as Code Developer with scars to prove it
  21. © 2020, Amazon Web Services, Inc. or its Affiliates. Best

    practices (2/5) • Start small and don’t try to boil the ocean • Work out simple resources first to get the hang of it. • Do not specify every little detail right from the start.
  22. © 2020, Amazon Web Services, Inc. or its Affiliates. Best

    practices (3/5) • It’s okay to repeat yourself • Do not engineer a whole new construct/library just so you prevent yourself from typing twice. • Do not overengineer things – this will help you out in the long run
  23. © 2020, Amazon Web Services, Inc. or its Affiliates. Quotes

    “It’s okay to repeat yourself in CDK. It’s not normal code, Don’t engineer a whole new construct just to prevent yourself from typing something twice” - An experience CDK developer “Keep in mind the operational aspects coming after you build the infra and make it as simple as possible to support. And more importantly, straightforward to troubleshoot. Operations will pay dearly for crazy abstractions and dependencies in your code” - Someone who had a run in with Operations
  24. © 2020, Amazon Web Services, Inc. or its Affiliates. Best

    practices (4/5) • Parameters and Mappings • Secrets Manager and SSM Parameter store • Do not hardcode sensitive information Resources: MyRDSDB: Type: "AWS::RDS::DBInstance" Properties: DBInstanceClass: db.t2.medium AllocatedStorage: ’20’ Engine: mariadb EngineVersion: ’10.2’ MasterUsername: appadmin MasterUserPassword: ‘{{resolve:ssm-secure:ssbRDSmEcntl:1}}'
  25. © 2020, Amazon Web Services, Inc. or its Affiliates. Best

    practices (5/5) • Test, Test, Test • Put proper guidelines in place before. • Introduce peer reviews of your infrastructure code.
  26. © 2020, Amazon Web Services, Inc. or its Affiliates. Quotes

    “Automation makes it easy to destroy your entire org with a few lines of yaml. Be paranoid about peer review, promoting changes through test environments, and privilege segregation" - An IaC Developer who, apparently, destroyed an entire org. “Automation let’s you mess up, at scale!” - Developer whor an the ‘* destroy’ command
  27. © 2020, Amazon Web Services, Inc. or its Affiliates. Takeaways!

    • Get into Infrastructure as Code Early, it will help you manage scale in thelong run. ☁ • Treat infrastructure as code as any other code! • Use the tools that best fit your needs! ⚒
  28. © 2020, Amazon Web Services, Inc. or its Affiliates. @darkosubotica

    ln/darko-mesaros twitch.tv/ruptwelve youtube.com/ruptwelve