Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Do more with less code in a serverless world

Do more with less code in a serverless world

There are many software engineering practices that can and should be applied to Lambda functions: Single Responsibility Principle (from SOLID), You Ain't Gonna Need It (YAGNI), Keep It Simply Stupid (KISS). In this presentation, I'll go through the different ways to apply those principles in the AWS serverless world and even to avoid the usage of Lambda functions sometimes.

Tweet

More Decks by Jérôme Van Der Linden

Other Decks in Technology

Transcript

  1. Less is more Do more with less code in a

    serverless world Jerome Van Der Linden Geneva Serverless Meetup - 26/05/2020
  2. About me • In a previous life, ”Mr Cut Cut”

    (M. Coupe coupe) • Developer & software craftsman • And now, Solutions Architect @ AWS 2 linkedin.com/in/jeromevdl/ Jerome Van Der Linden
  3. SOLID 4 S Single Responsibility Principle O Open / closed

    Principle L Liskov substitution Principle I Interface segregation Principle D Dependency Inversion Principle
  4. Single Responsibility Principle 5 “A class or module should have

    one, and only one, reason to be changed” - Robert C. Martin, aka Uncle Bob
  5. YAGNI (You Ain’t Gonna Need It) 6 Just because you

    can, doesn’t mean you should…
  6. 8 And many more… Clean Code DRY Don’t Repeat Yourself

    Law of Demeter Broken window theory Boy scout rule Not invented here
  7. AWS Serverless world 12 AWS Lambda AWS Fargate Amazon API

    Gateway Amazon SNS Amazon SQS COMPUTE DATA STORES INTEGRATION Amazon Aurora Serverless Amazon S3 Amazon DynamoDB Amazon EventBridge FAAS AWS Step Functions AWS AppSync
  8. Single Responsibility Principle ü Do’s ✘ Don’ts * - Input

    validation - Business logic /!\ - Transform data - Return result - Event/Input Filtering - Transport data - Orchestration & long transactions - Retry/Failure handling * Most of the time
  9. Event Filtering 15 SNS Topic Publisher if event_type == 'order_created’:

    lambda_client.invoke('OrderCreation', ...) elif event_type == 'order_placed’: lambda_client.invoke('OrderPlacement', ...) elif event_type == 'order_cancelled’: lambda_client.invoke('OrderCancelation', ...) OrderCreation OrderPlacement OrderCancellation { "event_type": "order_placed", "order": { "id": "232134", "amount": "2341,45", "stock_ref": "AMZN” } } the wrong way function code Input event OrderFiltering
  10. Event Filtering with SNS 16 OrderCreation OrderPlacement OrderCancellation Publisher OrderCreationEvent:

    Type: AWS::SNS::Subscription Properties: TopicArn: 'arn:aws:sns:eu-central-1:123456789:OrdersTopic’ Protocol: lambda Endpoint: 'arn:aws:lambda:eu-central-1:123456789:function:OrderCreation’ FilterPolicy: event_type: - order_created SNS Topic https://docs.aws.amazon.com/sns/latest/dg/sns-subscription-filter-policies.html Cloudformation template { "event_type": [ "order_created" ] } Filter policy
  11. Event Filtering with SNS 17 OrderCreation OrderPlacement OrderCancellation Publisher OrderPlacementEvent:

    Type: AWS::SNS::Subscription Properties: TopicArn: 'arn:aws:sns:eu-central-1:123456789:OrdersTopic’ Protocol: lambda Endpoint: 'arn:aws:lambda:eu-central-1:123456789:function:OrderPlacement’ FilterPolicy: event_type: - order_placed SNS Topic Cloudformation template
  12. Event Filtering with SNS 18 OrderCreation OrderPlacement OrderCancellation Publisher OrderCancellationEvent:

    Type: AWS::SNS::Subscription Properties: TopicArn: 'arn:aws:sns:eu-central-1:123456789:OrdersTopic’ Protocol: lambda Endpoint: 'arn:aws:lambda:eu-central-1:123456789:function:OrderCancellation’ FilterPolicy: event_type: - order_cancelled SNS Topic Cloudformation template
  13. Event Filtering with EventBridge 19 https://aws.amazon.com/blogs/compute/reducing-custom-code-by-using-advanced-rules-in-amazon-eventbridge/ { "Source": "custom.myATMapp", "EventBusName":

    "default", "DetailType": "transaction", "Time": "Wed Jan 29 2020 08:03:18 GMT-0500", "Detail":{ "action": "withdrawal", "location": "NY-NYC-001", "amount": 300, "result": "approved", "transactionId": "123456", "cardPresent": true, "partnerBank": "Example Bank", "remainingFunds": 722.34 } } { "source": [ "custom.myATMapp" ], "detail-type": [ "transaction" ], "detail": { "amount": [ { "numeric": [ ">", 300 ] } ] } } { "source": [ "custom.myATMapp" ], "detail-type": [ "transaction" ], "detail": { "location": [ { "prefix": "NY-NYC-" } ] } } { "source": [ "custom.myATMapp" ], "detail-type": [ "transaction" ], "detail": { "partnerBank": [ { "exists": true } ] } } { "source": [ "custom.myATMapp" ], "detail-type": [ "transaction" ], "detail": { "result": [ "approved" ], "partnerBank": [ { "exists": false } ], "location": [ { "anything-but": "NY-NYC-002" }] } }
  14. Orchestration 20 the wrong way invoke invoke if (… )

    invoke // else if (success) Notify with SNS n * invoke if (failure) Enqueue error notify with SNS need approval App…
  15. Simple orchestration with Lambda destinations 21 invoke invoke if (…

    ) invoke // else if (success) Notify with SNS n * invoke if (failure) Enqueue error notify with SNS need approval App… Lambda Destinations
  16. Simple orchestration with Lambda destinations 22 Amazon SNS Amazon EventBridge

    Amazon Cloudwatch Logs Amazon S3 Amazon SES AWS Config Amazon CloudFormation AWS CodeCommit A S Y N C "DestinationConfig": { "onSuccess": { "Destination": "arn:aws:lambda:..." }, "onFailure": { "Destination": "arn:aws:sqs:..." } } Cloudformation template Amazon SNS Amazon EventBridge Amazon SQS AWS Lambda if success: return {...} else: raise Exception(‘Failure', {...}) function code Lambda function A S Y N C
  17. Advanced orchestration with Step Functions 23 invoke invoke if (…

    ) invoke // else if (success) Notify with SNS n * invoke if (failure) Enqueue error notify with SNS need approval App…
  18. Advanced orchestration with Step Functions 24 invoke invoke if (…

    ) invoke // else if (success) Notify with SNS n * invoke if (failure) Enqueue error notify with SNS need approval App… { "StartAt": "SimpleInvocation", "States": { "SimpleInvocation": { "Type": "Task", "Resource": "arn:aws:lambda:eu-central- 1:123456789012:function:HelloFunction", "Next": "Choose1or2" },
  19. Advanced orchestration with Step Functions 25 invoke invoke if (…

    ) invoke // else if (success) Notify with SNS n * invoke if (failure) Enqueue error notify with SNS need approval App… "Choose1or2": { "Type": "Choice", "Choices": [ { "Variable": "$.foo”, "NumericEquals": 1, "Next": "Lambda1" }, { "Variable": "$.foo", "NumericEquals": 2, "Next": "ParallelInvocation" } ], "Default": "Unmatched" },
  20. Advanced orchestration with Step Functions 26 invoke invoke if (…

    ) invoke // else if (success) Notify with SNS n * invoke if (failure) Enqueue error notify with SNS need approval App… "Lambda1": { "Type": "Task", "Resource": "arn:aws:lambda:eu- central-1:123456789012:function:Lambda1", "Next": "SuccessOrFailure" },
  21. Advanced orchestration with Step Functions 27 invoke invoke if (…

    ) invoke // else if (success) Notify with SNS n * invoke if (failure) Enqueue error notify with SNS need approval App… "SuccessOrFailure": { "Type": "Choice", "Choices": [ { "Variable": "$.status", "StringEquals": "SUCCESS", "Next": "SendNotification" }, { "Variable": "$.status", "StringEquals": "FAILURE", "Next": "QueueError" } ], "Default": "Unmatched" }
  22. Advanced orchestration with Step Functions 28 invoke invoke if (…

    ) invoke // else if (success) Notify with SNS n * invoke if (failure) Enqueue error notify with SNS need approval App… "SendNotification": { "Type": "Succeed" }, "QueueError": { "Type": "Fail" },
  23. Advanced orchestration with Step Functions 29 invoke invoke if (…

    ) invoke // else if (success) Notify with SNS n * invoke if (failure) Enqueue error notify with SNS need approval App… "ParallelInvocation": { "Type": "Parallel", "Branches": [ { "StartAt": "SendApprovalRequest", "States": { "SendApprovalRequest": { // ... } }, { "StartAt": "Loop", "States": { "Loop": { // ... } } }
  24. Advanced orchestration with Step Functions 30 invoke invoke if (…

    ) invoke // else if (success) Notify with SNS n * invoke if (failure) Enqueue error notify with SNS need approval App… "SendApprovalRequest": { "Type": "Task", "Resource": "arn:aws:states:::lambda:invoke.waitForTaskToken", "Parameters": { "FunctionName": "sendMailForApprovalFunction", "Payload": { "step.$": "$$.State.Name", "model.$": "$.data", "token.$": "$$.Task.Token" } }, "ResultPath": "$.output", "Next": "Approved", "Catch": [ { "ErrorEquals": [ "rejected" ], "ResultPath": "$.reason", "Next": "Rejected" } ] } SendTaskSuccess SendTaskFailure
  25. Advanced orchestration with Step Functions 31 invoke invoke if (…

    ) invoke // else if (success) Notify with SNS n * invoke if (failure) Enqueue error notify with SNS need approval App… "Loop": { "Type": "Map", "ItemsPath": "$.loopItems", "Iterator": { "StartAt": "LoopLambda", "States": { "LoopLambda": { "Type": "Task", "Resource": "arn:aws:lambda:us-east- 1:123456789012:function:LoopFunction", "End": true } } }, "End": true }
  26. Advanced orchestration with Step Functions 32 invoke invoke if (…

    ) invoke // else if (success) Notify with SNS n * invoke if (failure) Enqueue error notify with SNS need approval App…
  27. TL;DR Single Responsibility Principle 33 èKeep your code focused on

    the business This is not the responsibility of a to do orchestration
  28. Ex: insert data in DynamoDB 36 Resource: /comments HTTP Method:

    POST HTTP Request Body: { "pageId": "example-page-id", "userName": "ExampleUserName", "message": "Example comment to be added." } Table: Comments commentId: String userName: String message: String pageId: String PK: commentId API Gateway DynamoDB Lambda var AWS = require('aws-sdk'); var ddb = new AWS.DynamoDB({apiVersion: '2012-08-10'}); exports.handler = async function(event, context) { var params = { TableName: 'Comments’, Item: { 'commentId' : { S: 'randomid’ }, 'pageId' : { S: event.pageId }, 'userName' : { S: event.userName }, 'message' : { S: event.message } } }; ddb.putItem(params, function(err, data) { if (err) { console.log("Error", err); } else { console.log("Success", data); } }); }
  29. Ex: insert data in DynamoDB 37 Resource: /comments HTTP Method:

    POST HTTP Request Body: { "pageId": "example-page-id", "userName": "ExampleUserName", "message": "Example comment to be added." } Table: Comments commentId: String userName: String message: String pageId: String PK: commentId API Gateway DynamoDB
  30. Ex: insert data in DynamoDB 38 Resource: /comments HTTP Method:

    POST HTTP Request Body: { "pageId": "example-page-id", "userName": "ExampleUserName", "message": "Example comment to be added." } Table: Comments commentId: String userName: String message: String pageId: String PK: commentId API Gateway DynamoDB
  31. Ex: insert data in DynamoDB 39 Resource: /comments HTTP Method:

    POST HTTP Request Body: { "pageId": "example-page-id", "userName": "ExampleUserName", "message": "Example comment to be added." } Table: Comments commentId: String userName: String message: String pageId: String PK: commentId API Gateway DynamoDB
  32. Ex: insert data in DynamoDB 40 Resource: /comments HTTP Method:

    POST HTTP Request Body: { "pageId": "example-page-id", "userName": "ExampleUserName", "message": "Example comment to be added." } Table: Comments commentId: String userName: String message: String pageId: String PK: commentId API Gateway DynamoDB
  33. Ex: insert data in DynamoDB 41 Resource: /comments HTTP Method:

    POST HTTP Request Body: { "pageId": "example-page-id", "userName": "ExampleUserName", "message": "Example comment to be added." } Table: Comments commentId: String userName: String message: String pageId: String PK: commentId API Gateway DynamoDB
  34. Use with caution 42 • Not a generic design pattern,

    not always applicable • Apply when the lambda is a passthrough (no business code, just mapping) • Mapping can sometimes be complex (Velocity Template Language)
  35. Step Function integrations 43 "QueueError": { "Type": "Task", "Resource": "arn:aws:states:::sqs:sendMessage",

    "Parameters": { "QueueUrl": "https://sqs.eu-central-1.amazonaws.com/123456789012/myQueue", "MessageBody.$": "$.input.message" }, "End": true }, "SendNotification": { "Type": "Task", "Resource": "arn:aws:states:::sns:publish", "Parameters": { "TopicARN": "arn:aws:sns:eu-central- 1:123456789012:myTopic", "Subject": "Lambda1 has successfully finish its job", "Message.$" : "$.input.message" }, "End": true }, SQS SNS
  36. Step Function integrations 44 AWS Batch Amazon DynamoDB Amazon ECS

    / Fargate Amazon EMR AWS Glue Amazon SageMaker Amazon SNS Amazon SQS AWS Step Functions AWS Lambda AWS CodeBuild
  37. TL;DR You Ain’t Gonna Need It 45 è is not

    always needed You can save costs and optimize performance when functions are just passthrough/mapping
  38. Keep your functions simple & stupid nano functions API GW

    /res1 /res2 /res3 = 1 handler (1 function) = 1 file Not that stupid so they need sisters to do the job !
  39. Keep your functions simple & stupid nano functions API GW

    /res1 /res2 /res3 = 1 handler (1 function) = 1 file function-lith API GW /{proxy} /res1 /res2 /res3 That’s really not simple & stupid! = 1 handler (1 function) = 1 file
  40. Keep your functions simple & stupid nano functions API GW

    /res1 /res2 /res3 function-lith API GW /{proxy} /res1 /res2 /res3 = 1 handler (1 function) = 1 file fat functions API GW /res1 /res2 /res3 = 3 handlers (3 functions) = 1 file = 1 handler (1 function) = 1 file
  41. Keep your functions simple & stupid 50 nano functions API

    GW /res1 /res2 /res3 function-lith API GW /{proxy} /res1 /res2 /res3 fat functions = 1 handler (1 function) = 1 file API GW = 3 handler (3 functions) = 1 file micro functions API GW /res1 /res2 /res3 /res1 /res2 /res3 = 1 handler (1 function) = 1 file = 1 handler (1 function) = 1 file
  42. Keep your functions simple & stupid https://github.com/cdk-patterns/serverless/tree/master/the-lambda-trilogy Potential duplicated code

    ++ Cognitive burden ++ Coupled functions Coupled deployments nano functions API GW /res1 /res2 /res3 function-lith API GW /{proxy} /res1 /res2 /res3 fat functions = 1 handler (1 function) = 1 file API GW = 3 handler (3 functions) = 1 file micro functions /res1 /res2 /res3 = 1 handler (1 function) = 1 file API GW /res1 /res2 /res3 = 1 handler (1 function) = 1 file Stupidity Complexity FAAS? Longer cold starts ++ Risk of blast radius Framework dependent Longer cold starts Risk of blast radius Potential duplicated code Cognitive burden
  43. TL;DR Keep it simple stupid 52 Lighter functions = Easier

    to test and maintain Faster to start: less cold start More scalable: higher throughput More secure: less permission needed
  44. Conclusion No function is easier to manage than ‘no function’

    – me è If you can do better without a function, just do it è Else, apply software craftsmanship principles (clean code, tests, code reviews…) “No server is easier to manage than ‘no server’” – Werner Vogels
  45. AWSTemplateFormatVersion: 2010-09-09 Resources: API: Type: AWS::ApiGateway::RestApi Properties: Name: !Sub ${AWS::StackName}-api-

    ${AWS::AccountId} APIDeployment: Type: AWS::ApiGateway::Deployment Properties: RestApiId: !Ref API StageName: prod DependsOn: - ListBucketAPI APIBucketResource: Type: AWS::ApiGateway::Resource Properties: ParentId: !GetAtt API.RootResourceId PathPart: "{bucket}" RestApiId: !Ref API ListBucketAPI: Type: AWS::ApiGateway::Method Properties: HttpMethod: GET ResourceId: !Ref APIBucketResource RestApiId: !Ref API AuthorizationType: NONE Integration: Credentials: 'arn:aws:iam::1234567890:role/role-demo-api-gw-s3-integration’ IntegrationHttpMethod: GET IntegrationResponses: - StatusCode: "200" PassthroughBehavior: WHEN_NO_MATCH RequestParameters: integration.request.header.Content-Type: method.request.header.Content-Type integration.request.header.x-amz-acl: "'authenticated-read’” integration.request.path.bucket: method.request.path.bucket Type: AWS Uri: !Sub arn:aws:apigateway:${AWS::Region}:s3:path/{bucket} MethodResponses: - StatusCode: "200" RequestParameters: method.request.path.bucket: true method.request.header.Content-Type: false API Gateway & S3 sample https://docs.aws.amazon.com/apigateway/latest/developerguide/integrating-api-with-aws-services-s3.html
  46. 57 Advanced orchestration with Step Functions { "StartAt": "SimpleInvocation", "States":

    { "SimpleInvocation": { "Type": "Task", "Resource": "arn:aws:lambda:eu-central-1:123456789012:function:HelloFunction", "Next": "Choose1or2" }, "Choose1or2": { "Type": "Choice", "Choices": [ { "Variable": "$.foo", "NumericEquals": 1, "Next": "Lambda1" }, { "Variable": "$.foo", "NumericEquals": 2, "Next": "ParallelInvocation" } ], "Default": "Unmatched" }, "Lambda1": { "Type": "Task", "Resource": "arn:aws:lambda:eu-central-1:123456789012:function:Lambda1", "Next": "SuccessOrFailure" }, "SuccessOrFailure": { "Type": "Choice", "Choices": [ { "Variable": "$.status", "StringEquals": "SUCCESS", "Next": "SendNotification" }, { "Variable": "$.status", "StringEquals": "FAILURE", "Next": "QueueError" } ], "Default": "Unmatched" }, "SendNotification": { "Type": "Succeed" }, "QueueError": { "Type": "Fail" }, "ParallelInvocation": { "Type": "Parallel", "Branches": [ { "StartAt": "SendApprovalRequest", "States": { "SendApprovalRequest": { "Type": "Task", "Resource": "arn:aws:states:::lambda:invoke.waitForTaskToken", "Parameters": { "FunctionName": "sendMailForApprovalFunction", "Payload": { "step.$": "$$.State.Name", "model.$": "$.data", "token.$": "$$.Task.Token" } }, "ResultPath": "$.output", "Next": "Approved", "Catch": [ { "ErrorEquals": [ "rejected" ], "ResultPath": "$.reason", "Next": "Rejected" } ] }, "Approved": { "Type": "Task", "Resource": "arn:aws:lambda:eu-central-1:123456789012:function:Lambda1", "End": true }, "Rejected": { "Type": "Fail" } } }, { "StartAt": "Loop", "States": { "Loop": { "Type": "Map", "ItemsPath": "$.loopItems", "Iterator": { "StartAt": "LoopLambda", "States": { "LoopLambda": { "Type": "Task", "Resource": "arn:aws:lambda:us-east-1:123456789012:function:LoopFunction", "End": true } } }, "End": true } } } ], "End": true }, "Unmatched": { "Type": "Fail", "Error": "DefaultStateError", "Cause": "No Matches!" } } }