Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building a CI/CD pipeline on AWS

Building a CI/CD pipeline on AWS

An overview of why and how to build a CI/CD pipeline using the Code-suite AWS tools

Milan Cermak

April 09, 2019
Tweet

More Decks by Milan Cermak

Other Decks in Technology

Transcript

  1. WHAT TO BUILD Holy grail A git-push-to-production pipeline, with automated

    rollbacks, with separated environments, with separated accounts, distributed in multiple regions Start simple Automate deployments for EVERYTHING Photo by Kobby Mendez on Unsplash
  2. WHY BOTHER? Speed Focus Peace of mind GOOD FOR BUSINESS

    Photo by Taha Mazandarani on Unsplash
  3. CODEPIPELINE ARTIFACTS Artifact is a fancy word for a zip

    file Code from the repository Output of CodeBuild … Accessible to actions Stored in an S3 bucket
  4. Name: FetchSource Actions: - Name: CodeCommit ActionTypeId: Category: Source Owner:

    AWS Provider: CodeCommit Version: '1' Configuration: RepositoryName: !GetAtt CodeRepository.Name BranchName: master OutputArtifacts: - Name: SourceOutput
  5. Name: UpdatePipeline Actions: - Name: PipelineStack ActionTypeId: Category: Deploy Owner:

    AWS Provider: CloudFormation Version: '1' Configuration: ActionMode: CREATE_UPDATE Capabilities: CAPABILITY_IAM RoleArn: !GetAtt CloudformationRole.Arn StackName: !Ref AWS::StackName TemplatePath: SourceOutput::infrastructure/ pipeline.yml ParameterOverrides: !Sub '{"Service": "$ {Service}"}' InputArtifacts: - Name: SourceOutput
  6. Name: Build Actions: - Name: Functions ActionTypeId: Category: Build Owner:

    AWS Provider: CodeBuild Version: '1' Configuration: ProjectName: !Ref CodeBuildProject InputArtifacts: - Name: SourceOutput OutputArtifacts: - Name: BuildOutput
  7. CodeBuildProject: Type: AWS::CodeBuild::Project Properties: Artifacts: Type: CODEPIPELINE Cache: Location: !Sub

    '${ArtifactsBucket}/buildcache/ functions' Type: S3 Environment: ComputeType: BUILD_GENERAL1_SMALL EnvironmentVariables: - Name: ARTIFACTS_BUCKET Value: !Ref ArtifactsBucket Type: PLAINTEXT Image: aws/codebuild/python:3.7.1 Type: LINUX_CONTAINER ServiceRole: !GetAtt CodeBuildRole.Arn Source: BuildSpec: infrastructure/buildspec.yml Type: CODEPIPELINE TimeoutInMinutes: 15
  8. BUILDSPEC.YML Collection of build commands in YAML Creates artifacts that

    are passed further along the pipeline Tips: Print useful progress messages, run verbose programs in quiet mode Use a cache Put “complex” multiline commands in a .sh file Update aws-cli if you need newish services support Debug your build process locally with Docker
  9. version: 0.2 env: variables: # ARTIFACTS_BUCKET is available from pipeline.yml

    PIP_CACHE: pip_cache phases: install: commands: - printenv - pip install -r requirements-dev.txt —cache-dir $PIP_CACHE pre_build: commands: - pytest -vrf tests build: commands: - aws cloudformation package --template-file infrastructure/functions.yml --output-template-file packaged_functions.yml --s3-bucket "${ARTIFACTS_BUCKET}" artifacts: files: - packaged_functions.yml cache: paths: - 'pip_cache/**/*'
  10. Name: Deploy Actions: - Name: Beta ActionTypeId: Category: Deploy Owner:

    AWS Provider: CloudFormation Version: '1' Configuration: ActionMode: CREATE_UPDATE Capabilities: CAPABILITY_IAM,CAPABILITY_AUTO_EXPAND RoleArn: !GetAtt CloudformationRole.Arn StackName: !Sub '${Service}-beta-functions' TemplatePath: BuildOutput::packaged_functions.yml ParameterOverrides: !Sub '{"Service": "${Service}", "Stage": "beta"}' InputArtifacts: - Name: BuildOutput
  11. Actions: - Name: Prod ActionTypeId: Category: Deploy Owner: AWS Provider:

    CloudFormation Version: '1' Configuration: ActionMode: CREATE_UPDATE Capabilities: CAPABILITY_IAM,CAPABILITY_AUTO_EXPAND RoleArn: !GetAtt CloudformationRole.Arn StackName: !Sub '${Service}-prod-functions' TemplatePath: BuildOutput::packaged_functions.yml ParameterOverrides: !Sub '{"Service": "${Service}", "Stage": “prod"}' InputArtifacts: - Name: BuildOutput
  12. USING CODEDEPLOY Connect Lambda with CodeDeploy via a CloudWatch Alarm

    Typically monitor the error rate Easy with AWS SAM, but there’s too much magic Tips: Make sure the alarms works as expected! Skip gradual rollout for non-production stacks Deploy “smaller regions” (i.e. not us-east-1) first
  13. Parameters: Stage: Description: Environment stage (deployment phase) Type: String AllowedValues:

    - beta - prod Conditions: IsBeta: !Equals [!Ref Stage, beta] Resources: HelloWorld: Type: AWS::Serverless::Function Properties: # ... AutoPublishAlias: !Ref Stage DeploymentPreference: Type: !If [IsBeta, AllAtOnce, Canary10Percent5Minutes] Alarms: - !Ref HelloWorldErrorsAlarm
  14. HelloWorldErrorsAlarm: Type: AWS::CloudWatch::Alarm Properties: ComparisonOperator: GreaterThanThreshold Dimensions: - Name: Resource

    Value: !Sub '${HelloWorld}:${Stage}' - Name: FunctionName Value: !Ref HelloWorld - Name: ExecutedVersion Value: !GetAtt HelloWorld.Version.Version EvaluationPeriods: 2 MetricName: Errors Namespace: AWS/Lambda Period: 60 Statistic: Sum Threshold: 0
  15. WHAT’S NEXT? Dynamic feature pipelines Separate accounts per stage Multi-region

    deployments Pipeline performance metrics Deploying dependent services Photo by Samuel Zeller on Unsplash