CI CD for serverless applications - AWS UG Thames Valley March 31 2021

Presented at AWS User Group Thames Valley, March 31st, 2021.

@gunnargrosch

To get the most out of the agility afforded by serverless, it is essential to build CI/CD pipelines that help teams iterate on code and quickly release features.
This session breaks down the CI/CD pipeline and looks at different technologies that can help you achieve automated deployments. You learn how container image support for AWS Lambda makes deployment simpler.
You also see how the AWS Serverless Application Model, infrastructure as code framework, and other AWS tools like AWS CodeBuild, AWS CodePipeline, and AWS CodeDeploy can perform safe deployments of your serverless applications.

AWS SAM Managed Policies
http://bit.ly/sam-policy-template

Parameter override functions with CodePipeline
https://amzn.to/3lKXCfR

Lambda function as an action in your pipeline
https://amzn.to/2SVWZlW

Transcript

1. © 2021, Amazon Web Services, Inc. or its Affiliates. Gunnar Grosch Senior Developer Advocate, AWS @gunnargrosch CI/CD for serverless applications
2. © 2021, Amazon Web Services, Inc. or its Affiliates. What is CI/CD? CI: Continuous integration CD: Continuous delivery or CD: Continuous deployment
3. © 2021, Amazon Web Services, Inc. or its Affiliates. Stages of the software release process Source Build Test Production
4. © 2021, Amazon Web Services, Inc. or its Affiliates. Stages of the software release process Source Build Test Production Developers’ version – review and commit code changes
5. © 2021, Amazon Web Services, Inc. or its Affiliates. Stages of the software release process Source Build Test Production Build tasks and unit testing Developers’ version – review and commit code changes
6. © 2021, Amazon Web Services, Inc. or its Affiliates. Stages of the software release process Source Build Test Production Deploy to testing environments and run integration tests Developers’ version – review and commit code changes Build tasks and unit testing
7. © 2021, Amazon Web Services, Inc. or its Affiliates. Stages of the software release process Source Build Test Production Deploy to production environment Developers’ version – review and commit code changes Build tasks and unit testing Deploy to testing environments and run integration tests
8. © 2021, Amazon Web Services, Inc. or its Affiliates. Stages of the software release process Source Build Test Production Continuous integration
9. © 2021, Amazon Web Services, Inc. or its Affiliates. Stages of the software release process Source Build Test Production Continuous integration Continuous delivery Approved deploy
10. © 2021, Amazon Web Services, Inc. or its Affiliates. Stages of the software release process Source Build Test Production Continuous deployment Continuous integration Continuous delivery Automated deploy Approved deploy
11. © 2021, Amazon Web Services, Inc. or its Affiliates. Choosing the right tools
12. © 2021, Amazon Web Services, Inc. or its Affiliates. AWS CodePipeline • Continuous delivery service for fast and reliable application updates • Model and visualize your software release process • Builds, tests, and deploys your code every time there is a code change • Integrates with third-party tools and AWS • https://aws.amazon.com/codepipeline/
13. © 2021, Amazon Web Services, Inc. or its Affiliates. Example of minimal developer’s pipeline • Three stages • Builds code artifact • One development environment • Uses AWS SAM/AWS CloudFormation to deploy artifact and other AWS resources • Has Lambda custom actions for testing functions Source Source GitHub i Build test-build-source CodeBuild i MyDev-Deploy create-changeset CloudFormation i execute-changeset CloudFormation i Run-stubs Lambda i
14. © 2021, Amazon Web Services, Inc. or its Affiliates. Example of production pipeline • Five stages • Builds code artifact • Three deployed to “environments” • Uses AWS SAM / AWS CloudFormation to deploy artifact and other AWS resources • AWS Lambda custom actions for testing functions • Integrates with a third-party tool / service • Has a manual approval before deploying to production Source Source GitHub i Build test-build-source CodeBuild i Deploy-Testing create-changeset CloudFormation i execute-changeset CloudFormation i Run-stubs Lambda i Deploy-Staging Deploy-Prod create-changeset CloudFormation i execute-changeset CloudFormation i Post-Deploy-Stack Lambda i create-changeset CloudFormation i execute-changeset CloudFormation i QA-Sign-Off Manual approval i Review Run-API-test Runscope i
15. © 2021, Amazon Web Services, Inc. or its Affiliates. Best practice Keep one environment per AWS account Source Source GitHub i Build test-build-source CodeBuild i Deploy-Testing create-changeset CloudFormation i execute-changeset CloudFormation i Run-stubs Lambda i Deploy-Staging Deploy-Prod create-changeset CloudFormation i execute-changeset CloudFormation i Post-Deploy-Stack Lambda i create-changeset CloudFormation i execute-changeset CloudFormation i QA-Sign-Off Manual approval i Review Run-API-test Runscope i Dev account Staging account Production account
16. © 2021, Amazon Web Services, Inc. or its Affiliates. CodePipeline demo
17. © 2021, Amazon Web Services, Inc. or its Affiliates. AWS CodePipeline: Supported sources A U T O M A T I C A L L Y K I C K O F F R E L E A S E A N D P U L L L A T E S T S O U R C E C O D E Pick object or folder Amazon Simple Storage Service (Amazon S3) Pick Docker tag Amazon Elastic Container Registry (Amazon ECR) Pick branch AWS CodeCommit GitHub
18. © 2021, Amazon Web Services, Inc. or its Affiliates. Release process stages Source Build Test Production Infrastructure Code
19. © 2021, Amazon Web Services, Inc. or its Affiliates. Infrastructure as code
20. © 2021, Amazon Web Services, Inc. or its Affiliates. AWS CloudFormation • Infrastructure as code • Provides a common language for you to describe and provision all the infrastructure resources in your cloud environment • Build and rebuild your infrastructure and applications, without having to perform manual actions or write custom scripts • https://aws.amazon.com/cloudformation/
21. © 2021, Amazon Web Services, Inc. or its Affiliates. AWS Serverless Application Model (AWS SAM) • Open source framework for building serverless applications on AWS • Shorthand syntax to express functions, APIs, databases, and event source mappings • Transforms and expands SAM syntax into AWS CloudFormation syntax on deployment • Supports all AWS CloudFormation resource types https://aws.amazon.com/serverless/sam/
22. © 2021, Amazon Web Services, Inc. or its Affiliates. AWS SAM template AWSTemplateFormatVersion: '2010-09-09’ Transform: AWS::Serverless-2016-10-31 Resources: MySimpleTableFunction: Type: AWS::Serverless::Function Properties: Handler: mySimpleTableFunction.handler Runtime: nodejs12.x CodeUri: ./functions Policies: - DynamoDBReadPolicy: TableName: !Ref MySimpleTable Events: MySimpleFunctionApi: Type: Api Properties: Path: /simpleTable Method: GET MySimpleTable: Type: AWS::Serverless::SimpleTable
23. © 2021, Amazon Web Services, Inc. or its Affiliates. AWS SAM template AWS Cloud Amazon API Gateway Lambda function Table Role === To become this Allowing this AWSTemplateFormatVersion: '2010-09-09’ Transform: AWS::Serverless-2016-10-31 Resources: MySimpleTableFunction: Type: AWS::Serverless::Function Properties: Handler: mySimpleTableFunction.handler Runtime: nodejs12.x CodeUri: ./functions Policies: - DynamoDBReadPolicy: TableName: !Ref MySimpleTable Events: MySimpleFunctionApi: Type: Api Properties: Path: /simpleTable Method: GET MySimpleTable: Type: AWS::Serverless::SimpleTable
24. © 2021, Amazon Web Services, Inc. or its Affiliates. AWSTemplateFormatVersion: '2010-09-09’ Transform: AWS::Serverless-2016-10-31 Resources: MySimpleTableFunction: Type: AWS::Serverless::Function Properties: Handler: mySimpleTableFunction.handler Runtime: nodejs12.x CodeUri: ./functions Policies: - DynamoDBReadPolicy: TableName: !Ref MySimpleTable Events: MySimpleFunctionApi: Type: Api Properties: Path: /simpleTable Method: GET MySimpleTable: Type: AWS::Serverless::SimpleTable AWS SAM template Tells AWS CloudFormation that this is a SAM template it needs to “transform”
25. © 2021, Amazon Web Services, Inc. or its Affiliates. AWSTemplateFormatVersion: '2010-09-09’ Transform: AWS::Serverless-2016-10-31 Resources: MySimpleTableFunction: Type: AWS::Serverless::Function Properties: Handler: mySimpleTableFunction.handler Runtime: nodejs12.x CodeUri: ./functions Policies: - DynamoDBReadPolicy: TableName: !Ref MySimpleTable Events: MySimpleFunctionApi: Type: Api Properties: Path: /simpleTable Method: GET MySimpleTable: Type: AWS::Serverless::SimpleTable AWS SAM template Just 20 lines to create: • Lambda function • IAM role • API gateway
26. © 2021, Amazon Web Services, Inc. or its Affiliates. AWSTemplateFormatVersion: '2010-09-09’ Transform: AWS::Serverless-2016-10-31 Resources: MySimpleTableFunction: Type: AWS::Serverless::Function Properties: Handler: mySimpleTableFunction.handler Runtime: nodejs12.x CodeUri: ./functions Policies: - DynamoDBReadPolicy: TableName: !Ref MySimpleTable Events: MySimpleFunctionApi: Type: Api Properties: Path: /simpleTable Method: GET MySimpleTable: Type: AWS::Serverless::SimpleTable AWS SAM template Function configuration
27. © 2021, Amazon Web Services, Inc. or its Affiliates. AWSTemplateFormatVersion: '2010-09-09’ Transform: AWS::Serverless-2016-10-31 Resources: MySimpleTableFunction: Type: AWS::Serverless::Function Properties: Handler: mySimpleTableFunction.handler Runtime: nodejs12.x CodeUri: ./functions Policies: - DynamoDBReadPolicy: TableName: !Ref MySimpleTable Events: MySimpleFunctionApi: Type: Api Properties: Path: /simpleTable Method: GET MySimpleTable: Type: AWS::Serverless::SimpleTable AWS SAM template Permissions for the function
28. © 2021, Amazon Web Services, Inc. or its Affiliates. AWSTemplateFormatVersion: '2010-09-09’ Transform: AWS::Serverless-2016-10-31 Resources: MySimpleTableFunction: Type: AWS::Serverless::Function Properties: Handler: mySimpleTableFunction.handler Runtime: nodejs12.x CodeUri: ./functions Policies: - DynamoDBReadPolicy: TableName: !Ref MySimpleTable Events: MySimpleFunctionApi: Type: Api Properties: Path: /simpleTable Method: GET MySimpleTable: Type: AWS::Serverless::SimpleTable AWS SAM policy templates Policies: - CloudWatchPutMetricPolicy: {} Policies: - DynamoDBReadPolicy: TableName: !Ref MySimpleTable
29. © 2021, Amazon Web Services, Inc. or its Affiliates. AWSTemplateFormatVersion: '2010-09-09’ Transform: AWS::Serverless-2016-10-31 Resources: MySimpleTableFunction: Type: AWS::Serverless::Function Properties: Handler: mySimpleTableFunction.handler Runtime: nodejs12.x CodeUri: ./functions Policies: - DynamoDBReadPolicy: TableName: !Ref MySimpleTable Events: MySimpleFunctionApi: Type: Api Properties: Path: /simpleTable Method: GET MySimpleTable: Type: AWS::Serverless::SimpleTable AWS SAM policy templates All the available policies can be found here: http://bit.ly/sam-policy-template
30. © 2021, Amazon Web Services, Inc. or its Affiliates. AWSTemplateFormatVersion: '2010-09-09’ Transform: AWS::Serverless-2016-10-31 Resources: MySimpleTableFunction: Type: AWS::Serverless::Function Properties: Handler: mySimpleTableFunction.handler Runtime: nodejs12.x CodeUri: ./functions Policies: - DynamoDBReadPolicy: TableName: !Ref MySimpleTable Events: MySimpleFunctionApi: Type: Api Properties: Path: /simpleTable Method: GET MySimpleTable: Type: AWS::Serverless::SimpleTable AWS SAM template API gateway
31. © 2021, Amazon Web Services, Inc. or its Affiliates. AWSTemplateFormatVersion: '2010-09-09’ Transform: AWS::Serverless-2016-10-31 Resources: MySimpleTableFunction: Type: AWS::Serverless::Function Properties: Handler: mySimpleTableFunction.handler Runtime: nodejs12.x CodeUri: ./functions Policies: - DynamoDBReadPolicy: TableName: !Ref MySimpleTable Events: MySimpleFunctionApi: Type: Api Properties: Path: /simpleTable Method: GET MySimpleTable: Type: AWS::Serverless::SimpleTable AWS SAM template Creates an Amazon DynamoDB table with 5 read / write units
32. © 2021, Amazon Web Services, Inc. or its Affiliates. One artifact – Multiple stages DEV STAGE PROD
33. © 2021, Amazon Web Services, Inc. or its Affiliates. One artifact – Multiple stages I N A W S S A M T E M P L A T E Parameters: MyEnvironment: Type: String Default: dev AllowedValues: - dev - staging - prod Mappings: ResourcesName: dev: tableName: MyDynamoDBTable-dev staging: tableName: MyDynamoDBTable-staging prod: tableName: MyDynamoDBTable-prod Resources: MyDynamoDBTable: Type: "AWS::DynamoDB::Table" Properties: . . . TableName: !FindInMap [ResourcesName, !Ref MyEnvironment, tableName]
34. © 2021, Amazon Web Services, Inc. or its Affiliates. Parameters: MyEnvironment: Type: String Default: dev AllowedValues: - dev - staging - prod Mappings: ResourcesName: dev: tableName: MyDynamoDBTable-dev staging: tableName: MyDynamoDBTable-staging prod: tableName: MyDynamoDBTable-prod Resources: MyDynamoDBTable: Type: "AWS::DynamoDB::Table" Properties: . . . TableName: !FindInMap [ResourcesName, !Ref MyEnvironment, tableName] One artifact – Multiple stages I N A W S S A M T E M P L A T E
35. © 2021, Amazon Web Services, Inc. or its Affiliates. Parameters: MyEnvironment: Type: String Default: dev AllowedValues: - dev - staging - prod Mappings: ResourcesName: dev: tableName: MyDynamoDBTable-dev staging: tableName: MyDynamoDBTable-staging prod: tableName: MyDynamoDBTable-prod Resources: MyDynamoDBTable: Type: "AWS::DynamoDB::Table" Properties: . . . TableName: !FindInMap [ResourcesName, !Ref MyEnvironment, tableName] One artifact – Multiple stages I N A W S S A M T E M P L A T E
36. © 2021, Amazon Web Services, Inc. or its Affiliates. Parameters: MyEnvironment: Type: String Default: dev AllowedValues: - dev - staging - prod Mappings: ResourcesName: dev: tableName: MyDynamoDBTable-dev staging: tableName: MyDynamoDBTable-staging prod: tableName: MyDynamoDBTable-prod Resources: MyDynamoDBTable: Type: "AWS::DynamoDB::Table" Properties: . . . TableName: !FindInMap [ResourcesName, !Ref MyEnvironment, tableName] One artifact – Multiple stages I N A W S S A M T E M P L A T E
37. © 2021, Amazon Web Services, Inc. or its Affiliates. One artifact – Multiple stages I N A W S C O D E P I P E L I N E We need to pass the stage name as a parameter when we create the AWS CloudFormation change set https://amzn.to/3lKXCfR
38. © 2021, Amazon Web Services, Inc. or its Affiliates. AWS Lambda environment variables I N A W S S A M T E M P L A T E MySimpleFunction: Type: "AWS::Serverless::Function" Properties: Handler: mySimpleFunction.handler Policies: - DynamoDBCrudPolicy: TableName: !Ref MyDynamoDBTable Environment: Variables: TABLE_NAME: !Ref MyDynamoDBTable Events: MySimpleFunctionAPI: Type: Api Properties: RestApiId: !Ref MyApi Path: /simple Method: GET
39. © 2021, Amazon Web Services, Inc. or its Affiliates. AWS Lambda environment variables I N T H E H A N D L E R O F T H E F U N C T I O N MySimpleFunction: Type: "AWS::Serverless::Function" Properties: Handler: mySimpleFunction.handler Policies: - DynamoDBCrudPolicy: TableName: !Ref MyDynamoDBTable Environment: Variables: TABLE_NAME: !Ref MyDynamoDBTable Events: MySimpleFunctionAPI: Type: Api Properties: RestApiId: !Ref MyApi Path: /simple Method: GET 'use strict'; const TABLE_NAME = process.env.TABLE_NAME; module.exports.getItem = async (propertyId, sortKey) => { console.log('getItem'); ...
40. © 2021, Amazon Web Services, Inc. or its Affiliates. Function code
41. © 2021, Amazon Web Services, Inc. or its Affiliates. Release process stages Source Build Test Production Code Infrastructure .zip Container images
42. © 2021, Amazon Web Services, Inc. or its Affiliates. Function code as container image • Use your favorite tools to build your apps • Pack function code and dependencies as a container image up to 10 GB • Lambda function only runs when triggered • Best of two worlds • Serverless operational simplicity • Flexibility of containers
43. © 2021, Amazon Web Services, Inc. or its Affiliates. AWS SAM INIT $sam init Which template source would you like to use? 1 - AWS Quick Start Templates 2 - Custom Template Location Choice: 1 What package type would you like to use? 1 - Zip (artifact is a zip uploaded to S3) 2 - Image (artifact is an image uploaded to an ECR image repository) Package type: 2
44. © 2021, Amazon Web Services, Inc. or its Affiliates. AWS SAM INIT Which base image would you like to use? 1 - amazon/nodejs12.x-base 2 - amazon/nodejs10.x-base 3 - amazon/python3.8-base 4 - amazon/python3.7-base 5 - amazon/python3.6-base 6 - amazon/python2.7-base 7 - amazon/ruby2.7-base 8 - amazon/ruby2.5-base 9 - amazon/go1.x-base 10 - amazon/java11-base 11 - amazon/java8.al2-base 12 - amazon/java8-base 13 - amazon/dotnetcore3.1-base 14 - amazon/dotnetcore2.1-base Project name [sam-app]: sam-lambda-containers Cloning app templates from https://github.com/awslabs/aws-sam-cli-app- templates.git ----------------------- Generating application: ----------------------- Name: sam-lambda-containers Base Image: amazon/nodejs12.x-base Dependency Manager: npm Output Directory: . Next steps can be found in the README file at ./sam-lambda-containers/README.md Base image: 1
45. © 2021, Amazon Web Services, Inc. or its Affiliates. Function code: Docker config FROM 628053151772.dkr.ecr.sa-east- 1.amazonaws.com/awslambda/nodejs12.x-runtime:beta COPY app.js package.json ./ RUN npm install # Command can be overwritten by providing a different command in the template directly. CMD ["app.lambdaHandler"]
46. © 2021, Amazon Web Services, Inc. or its Affiliates. AWS SAM with container images In the template.yml Resources: MyFunction: Type: 'AWS::Serverless::Function’ Properties: PackageType: Image ImageUri: '123456789012.dkr.ecr.us-west-2.amazonaws.com/my-function:latest'
47. © 2021, Amazon Web Services, Inc. or its Affiliates. Container image demo
48. © 2021, Amazon Web Services, Inc. or its Affiliates. Release process stages Source Build Test Production
49. © 2021, Amazon Web Services, Inc. or its Affiliates. AWS CodeBuild • Fully managed build service that can compile source code, run tests, and produce software packages • Scales continuously and processes multiple builds concurrently • No build servers to manage • Pay by the minute, for only the compute resources you use • https://aws.amazon.com/codebuild/
50. © 2021, Amazon Web Services, Inc. or its Affiliates. phases: install: commands: - pip install --upgrade pip - pip install pipenv –user - pipenv install awscli aws-sam-cli - npm install pre_build: commands: - eslint *.js - npm test build: commands: - sam build post_build: commands: - sam package –template-file template.yaml –s3-bucket $BUCKET_NAME –output-template packaged.yaml artifacts: type: zip files: - packaged.yaml Lambda buildspec using SAM CLI A W S C O D E B U I L D
51. © 2021, Amazon Web Services, Inc. or its Affiliates. phases: install: commands: - pip install --upgrade pip - pip install pipenv –user - pipenv install awscli aws-sam-cli - npm install pre_build: commands: - eslint *.js - npm test build: commands: - sam build post_build: commands: - sam package –template-file template.yaml –s3-bucket $BUCKET_NAME –output-template packaged.yaml artifacts: type: zip files: - packaged.yaml Lambda buildspec using SAM CLI A W S C O D E B U I L D Use install phase to install packages or any prereqs you may need throughout the build
52. © 2021, Amazon Web Services, Inc. or its Affiliates. phases: install: commands: - pip install --upgrade pip - pip install pipenv –user - pipenv install awscli aws-sam-cli - npm install pre_build: commands: - eslint *.js - npm test build: commands: - sam build post_build: commands: - sam package –template-file template.yaml –s3-bucket $BUCKET_NAME –output-template packaged.yaml artifacts: type: zip files: - packaged.yaml Lambda buildspec using SAM CLI A W S C O D E B U I L D Use pre-build phase to run unit tests
53. © 2021, Amazon Web Services, Inc. or its Affiliates. phases: install: commands: - pip install --upgrade pip - pip install pipenv –user - pipenv install awscli aws-sam-cli - npm install pre_build: commands: - eslint *.js - npm test build: commands: - sam build post_build: commands: - sam package –template-file template.yaml –s3-bucket $BUCKET_NAME –output-template packaged.yaml artifacts: type: zip files: - packaged.yaml Lambda buildspec using SAM CLI A W S C O D E B U I L D In the build phase, we build the project using AWS SAM
54. © 2021, Amazon Web Services, Inc. or its Affiliates. phases: install: commands: - pip install --upgrade pip - pip install pipenv –user - pipenv install awscli aws-sam-cli - npm install pre_build: commands: - eslint *.js - npm test build: commands: - sam build post_build: commands: - sam package –template-file template.yaml –s3-bucket $BUCKET_NAME –output-template packaged.yaml artifacts: type: zip files: - packaged.yaml Lambda buildspec using SAM CLI A W S C O D E B U I L D Use post-build for creating the package of the artifact
55. © 2021, Amazon Web Services, Inc. or its Affiliates. phases: install: commands: - pip install --upgrade pip - pip install pipenv –user - pipenv install awscli aws-sam-cli - npm install pre_build: commands: - eslint *.js - npm test build: commands: - sam build post_build: commands: - sam package –template-file template.yaml –s3-bucket $BUCKET_NAME –output-template packaged.yaml artifacts: type: zip files: - packaged.yaml Lambda buildspec using SAM CLI A W S C O D E B U I L D Where AWS CodeBuild can find the output artifacts
56. © 2021, Amazon Web Services, Inc. or its Affiliates. Lambda buildspec using SAM CLI and container images A W S C O D E B U I L D post_build: commands: - sam package –image-repository <ECR-URL> –output-template-file packaged.yaml –resolve-s3 ONLY CHANGE NEEDED Package with the ECR image
57. © 2021, Amazon Web Services, Inc. or its Affiliates. CodeBuild demo
58. © 2021, Amazon Web Services, Inc. or its Affiliates. Release process stages Source Build Test Production
59. © 2021, Amazon Web Services, Inc. or its Affiliates. Where and what to test • Code review via pull request • Lint / syntax check • Unit test pass • Code successfully compiles • Application deploys successfully • Mocked / stubbed integration tests • Application deploys successfully • Test against real services (potentially against production dependencies) • Run pre-traffic Lambda tests • Deploy canaries • Complete wait period successfully • Deploy 100% • Run post-traffic Lambda tests Source Source CodeCommit i Build test-build-source CodeBuild i Deploy-Testing create-changeset CloudFormation i execute-changeset CloudFormation i Run-stubs Lambda i Deploy-Staging Deploy-Prod create-changeset CloudFormation i execute-changeset CloudFormation i Post-Deploy-Stack Lambda i create-changeset CloudFormation i execute-changeset CloudFormation i QA-Sign-Off Manual approval i Review Run-API-test Runscope i
60. © 2021, Amazon Web Services, Inc. or its Affiliates. AWS Lambda to perform integration tests exports.handler = async (event, context) => { let jobId = event["CodePipeline.job"].id; // DO SOME INTEGRATION TESTS if (testsPassed) { let params = { jobId: jobId }; return codepipeline.putJobSuccessResult(params).promise(data => { context.succeed('Test passed'); }).catch(error => { context.fail(error); }); } else { let params = { jobId: jobId, failureDetails: { message: JSON.stringify('Test failed’), type: 'JobFailed’, externalExecutionId: context.invokeid } }; return codepipeline.putJobFailureResult(params).promise(data => { context.fail(message); }); } };
61. © 2021, Amazon Web Services, Inc. or its Affiliates. AWS Lambda to perform integration tests exports.handler = async (event, context) => { let jobId = event["CodePipeline.job"].id; // DO SOME INTEGRATION TESTS if (testsPassed) { let params = { jobId: jobId }; return codepipeline.putJobSuccessResult(params).promise(data => { context.succeed('Test passed'); }).catch(error => { context.fail(error); }); } else { let params = { jobId: jobId, failureDetails: { message: JSON.stringify('Test failed’), type: 'JobFailed’, externalExecutionId: context.invokeid } }; return codepipeline.putJobFailureResult(params).promise(data => { context.fail(message); }); } }; Write some integration tests
62. © 2021, Amazon Web Services, Inc. or its Affiliates. AWS Lambda to perform integration tests exports.handler = async (event, context) => { let jobId = event["CodePipeline.job"].id; // DO SOME INTEGRATION TESTS if (testsPassed) { let params = { jobId: jobId }; return codepipeline.putJobSuccessResult(params).promise(data => { context.succeed('Test passed'); }).catch(error => { context.fail(error); }); } else { let params = { jobId: jobId, failureDetails: { message: JSON.stringify('Test failed’), type: 'JobFailed’, externalExecutionId: context.invokeid } }; return codepipeline.putJobFailureResult(params).promise(data => { context.fail(message); }); } }; Needs to call the putJobSuccessResult
63. © 2021, Amazon Web Services, Inc. or its Affiliates. AWS Lambda to perform integration tests exports.handler = async (event, context) => { let jobId = event["CodePipeline.job"].id; // DO SOME INTEGRATION TESTS if (testsPassed) { let params = { jobId: jobId }; return codepipeline.putJobSuccessResult(params).promise(data => { context.succeed('Test passed'); }).catch(error => { context.fail(error); }); } else { let params = { jobId: jobId, failureDetails: { message: JSON.stringify('Test failed’), type: 'JobFailed’, externalExecutionId: context.invokeid } }; return codepipeline.putJobFailureResult(params).promise(data => { context.fail(message); }); } }; Needs to call the putJobFailureResult
64. © 2021, Amazon Web Services, Inc. or its Affiliates. AWS Lambda to perform integration tests exports.handler = async (event, context) => { let jobId = event["CodePipeline.job"].id; // DO SOME INTEGRATION TESTS if (testsPassed) { let params = { jobId: jobId }; return codepipeline.putJobSuccessResult(params).promise(data => { context.succeed('Test passed'); }).catch(error => { context.fail(error); }); } else { let params = { jobId: jobId, failureDetails: { message: JSON.stringify('Test failed’), type: 'JobFailed’, externalExecutionId: context.invokeid } }; return codepipeline.putJobFailureResult(params).promise(data => { context.fail(message); }); } }; Find more information on how to configure this here: https://amzn.to/2SVWZlW
65. © 2021, Amazon Web Services, Inc. or its Affiliates. AWS CodeDeploy • Automate code deployments to any instance and AWS Lambda • Handle the complexity of updating your applications • Avoid downtime during application deployment • Roll back automatically if failure is detected • Deploy to Amazon EC2, AWS Lambda, AWS Fargate, Amazon EC2, or on-premises servers
66. © 2021, Amazon Web Services, Inc. or its Affiliates. Safe deployments Canary10Percent30Minutes Canary10Percent5Minutes Canary10Percent10Minutes Canary10Percent15Minutes Linear10PercentEvery10Minutes Linear10PercentEvery1Minute Linear10PercentEvery2Minutes Linear10PercentEvery3Minutes AllAtOnce BeforeAllowTraffic AfterAllowTraffic AllowTraffic Deployment preference
67. © 2021, Amazon Web Services, Inc. or its Affiliates. Lambda canary deployment A W S C O D E D E P L O Y API gateway Lambda function weighted alias “live” v1 Lambda function code 100%
68. © 2021, Amazon Web Services, Inc. or its Affiliates. API gateway Lambda function weighted alias “live” v1 code 100% Run PreTraffic hook against v2 code before it receives traffic v2 code 0% Lambda canary deployment A W S C O D E D E P L O Y
69. © 2021, Amazon Web Services, Inc. or its Affiliates. API gateway Lambda function weighted alias “live” v1 code 90% Wait 10 minutes, roll back in case of alarm v2 code 10% Lambda canary deployment A W S C O D E D E P L O Y
70. © 2021, Amazon Web Services, Inc. or its Affiliates. API gateway Lambda function weighted alias “live” v1 code 0% Run PostTraffic hook and complete deployment v2 code 100% Lambda canary deployment A W S C O D E D E P L O Y
71. © 2021, Amazon Web Services, Inc. or its Affiliates. AWS CodeDeploy safe deployments A W S S A M T E M P L A T E HelloFunction: Type: 'AWS::Serverless::Function’ Properties: Handler: handler.hello Runtime: nodejs12.x CodeUri: ./hello AutoPublishAlias: live DeploymentPreference: Type: Linear10PercentEvery10Minutes Alarms: - !Ref AliasErrorMetricGreaterThanZeroAlarm - !Ref LatestVersionErrorMetricGreaterThanZeroAlarm Hooks: PreTraffic: !Ref PreTrafficLambdaFunction PostTraffic: !Ref PostTrafficLambdaFunction
72. © 2021, Amazon Web Services, Inc. or its Affiliates. HelloFunction: Type: 'AWS::Serverless::Function’ Properties: Handler: handler.hello Runtime: nodejs12.x CodeUri: ./hello AutoPublishAlias: live DeploymentPreference: Type: Linear10PercentEvery10Minutes Alarms: - !Ref AliasErrorMetricGreaterThanZeroAlarm - !Ref LatestVersionErrorMetricGreaterThanZeroAlarm Hooks: PreTraffic: !Ref PreTrafficLambdaFunction PostTraffic: !Ref PostTrafficLambdaFunction Instructs AWS SAM to publish a new version of the function for each new deployment and to link it to the live alias AWS CodeDeploy safe deployments A W S S A M T E M P L A T E
73. © 2021, Amazon Web Services, Inc. or its Affiliates. HelloFunction: Type: 'AWS::Serverless::Function’ Properties: Handler: handler.hello Runtime: nodejs12.x CodeUri: ./hello AutoPublishAlias: live DeploymentPreference: Type: Linear10PercentEvery10Minutes Alarms: - !Ref AliasErrorMetricGreaterThanZeroAlarm - !Ref LatestVersionErrorMetricGreaterThanZeroAlarm Hooks: PreTraffic: !Ref PreTrafficLambdaFunction PostTraffic: !Ref PostTrafficLambdaFunction Canary10Percent30Minutes Canary10Percent5Minutes Canary10Percent10Minutes Canary10Percent15Minutes Linear10PercentEvery10Minutes Linear10PercentEvery1Minute Linear10PercentEvery2Minutes Linear10PercentEvery3Minutes AllAtOnce AWS CodeDeploy safe deployments A W S S A M T E M P L A T E
74. © 2021, Amazon Web Services, Inc. or its Affiliates. HelloFunction: Type: 'AWS::Serverless::Function’ Properties: Handler: handler.hello Runtime: nodejs12.x CodeUri: ./hello AutoPublishAlias: live DeploymentPreference: Type: Linear10PercentEvery10Minutes Alarms: - !Ref AliasErrorMetricGreaterThanZeroAlarm - !Ref LatestVersionErrorMetricGreaterThanZeroAlarm Hooks: PreTraffic: !Ref PreTrafficLambdaFunction PostTraffic: !Ref PostTrafficLambdaFunction A list of alarms that you want to monitor; you can specify a maximum of 10 alarms AWS CodeDeploy safe deployments A W S S A M T E M P L A T E
75. © 2021, Amazon Web Services, Inc. or its Affiliates. HelloFunction: Type: 'AWS::Serverless::Function’ Properties: Handler: handler.hello Runtime: nodejs12.x CodeUri: ./hello AutoPublishAlias: live DeploymentPreference: Type: Linear10PercentEvery10Minutes Alarms: - !Ref AliasErrorMetricGreaterThanZeroAlarm - !Ref LatestVersionErrorMetricGreaterThanZeroAlarm Hooks: PreTraffic: !Ref PreTrafficLambdaFunction PostTraffic: !Ref PostTrafficLambdaFunction Validation Lambda functions that run before and after traffic shifting AWS CodeDeploy safe deployments A W S S A M T E M P L A T E
76. © 2021, Amazon Web Services, Inc. or its Affiliates. Safe deployment demo
77. © 2021, Amazon Web Services, Inc. or its Affiliates. Different strategies for different stages DEV STAGE PROD Linear10PercentEvery10Minutes AWS Lambda function
78. © 2021, Amazon Web Services, Inc. or its Affiliates. Different strategies for different stages Parameters: MyEnvironment: Type: String Default: dev AllowedValues: - dev - staging - prod Conditions: IsProd: !Equals [!Ref MyEnvironment, prod] MySafeDeployedFunction: Type: "AWS::Serverless::Function" Properties: Handler: mySafeDeployedFunction.handler AutoPublishAlias: live DeploymentPreference: Type: !If [IsProd, Linear10PercentEvery1Minute, AllAtOnce]
79. © 2021, Amazon Web Services, Inc. or its Affiliates. Different strategies for different stages Parameters: MyEnvironment: Type: String Default: dev AllowedValues: - dev - staging - prod Conditions: IsProd: !Equals [!Ref MyEnvironment, prod] MySafeDeployedFunction: Type: "AWS::Serverless::Function" Properties: Handler: mySafeDeployedFunction.handler AutoPublishAlias: live DeploymentPreference: Type: !If [IsProd, Linear10PercentEvery1Minute, AllAtOnce]
80. © 2021, Amazon Web Services, Inc. or its Affiliates. Different strategies for different stages Parameters: MyEnvironment: Type: String Default: dev AllowedValues: - dev - staging - prod Conditions: IsProd: !Equals [!Ref MyEnvironment, prod] MySafeDeployedFunction: Type: "AWS::Serverless::Function" Properties: Handler: mySafeDeployedFunction.handler AutoPublishAlias: live DeploymentPreference: Type: !If [IsProd, Linear10PercentEvery1Minute, AllAtOnce]
81. © 2021, Amazon Web Services, Inc. or its Affiliates. Release process stages Source Build Test Production
82. © 2021, Amazon Web Services, Inc. or its Affiliates. Monitoring and observability Amazon CloudWatch Collects monitoring and operational data in the form of logs, metrics, and events AWS X-Ray Provides a trace or an end-to-end view of requests traveling through your application
83. © 2021, Amazon Web Services, Inc. or its Affiliates. AWS Developer Tools for modern software delivery MONITOR DEPLOY TEST BUILD SOURCE / ARTIFACT AUTHOR AWS Cloud9 AWS IDE Toolkits AWS X-Ray AWS CodeBuild AWS CodeCommit AWS CodeDeploy AWS CodeBuild + Third Party Amazon CloudWatch AWS CodePipeline MODEL AWS CloudFormation AWS SAM AWS Cloud Development Kit AWS CodeArtifact NEW
84. © 2021, Amazon Web Services, Inc. or its Affiliates. “Create application” experience Create a serverless application from a starter template • AWS SAM template for infrastructure as code • CI / CD pipeline • Clone locally for development with AWS SAM CLI
85. © 2021, Amazon Web Services, Inc. or its Affiliates. Thank you! Gunnar Grosch Senior Developer Advocate, AWS @gunnargrosch