Error Handling in Stateless Environments

Errors happen in every application. In serverless, additional failures exist - timeouts, out of memory, and more. In many cases, errors are handled by the cloud vendor using a retry mechanism. Since the code is stateless, how can you be sure that the application actually works?

Transcript

1. devopsdays Tel Aviv Error Handling in Stateless Environments Nitzan Shapira, Epsagon
2. Nitzan Shapira (@nitzanshapira) Software engineer > 12 years Co-Founder, CEO at Epsagon Tel Aviv 2 > whoami
3. What is serverless? How is it different? What is stateless? Why is it relevant today? How to handle errors in such environments? How will it help my job? 3 Things to discuss
4. 4 [Compute-as-a-Service] FaaS: Function-as-a-Service CaaS: Container-as-a-Service + Managed services (APIs) = Don’t manage infrastructure Focus on business logic What is serverless?
5. 5 Why serverless? Pay-per-use: reduces cloud compute cost by 90% Out-of-the-box auto-scaling DevOps à LowOps ++Developer velocity Focus on business logic – iterate faster Server Utilization
6. 6 The limitations of FaaS Limited memory Limited running time Cold starts Stateless + concurrency limit + some others…
7. 7 The properties of serverless applications Serverless is micro-services Serverless applications are - Highly distributed - Highly event-driven Utilizing managed services via APIs is key
8. A real example – HSBC 8 Source: re:Invent 2018
9. 9 The challenge in serverless SIMPLE COMPLEX Yan Cui
10. 10 Troubleshoot and fix e
11. 11 What the community thinks 2018 Serverless Community Survey, serverless.com, July 2018 2017 results
12. 12 Error handling in traditional environments
13. 13 What do you do when something goes wrong? Take a look at the log – it will tell the story Connect to the host! Run a debugger!
14. 14 Stateless environments challenges Event-drive design No server to connect to – difficult to troubleshoot No current state – difficult to determine system health
15. 15 Error handling in stateless environments
16. 16 Types of failures in serverless Unhandled exception – from your own code Timeout – due to your code or an external service Out-of-memory – due to your code or misconfiguration
17. 17 Retries behavior Synchronous events • The invoking application is in charge of the error Asynchronous events • A retry mechanism is triggered for a certain period of time Stream-based events • A retry mechanism is triggered until the data is expired
18. 18 Retry behavior consequences Retries might change the logical flow of the application In order for retries to succeed, the code must be idempotent: “Idempotence is the property of certain operations in mathematics and computer science that they can be applied multiple times without changing the result beyond the initial application” (Wikipedia).
19. 19 Example of idempotent operations Update the same DB entry to the same value multiple times Authenticate a user Check if a file exists, and if not, create an empty file with that name Confusing to design, difficult to implement!
20. 20 Practical methods for retries Write idempotent code Tough! Use a proper service Example : AWS Step Functions Source: AWS
21. 21 AWS Step Functions – Serverless Orchestration
22. 22 Troubleshooting with retries
23. 23 SNS + AWS Step Functions
24. 24 Deploying using Serverless Framework
25. 25 Observability – why do we need it? Track system health Troubleshoot and fix Optimize performance and cost
26. 26 Observability in serverless and stateless systems Let’s go one by one
27. 27 Track system health System == Functions ?
28. 28 Functions are important - Errors - Timeout - Out-of-memory - Cold start
29. 29 Track system health System > Functions ! Serverless != Functions
30. 30 Track system health System > Functions ! Functions APIs Transactions
31. 31 Troubleshoot and fix Functions are not enough Need: track asynchronous events e
32. 32 Transactions
33. 33 Tracing asynchronous invocations
34. 34 Tracing asynchronous invocations
35. 35 Tracing asynchronous invocations
36. 36 Distributed tracing …a trace tells the story of a transaction or workflow as it propagates through a (potentially distributed) system. Distributed tracing is a method used to profile and monitor applications.
37. 37 Distributed tracing Jaeger
38. 38 Implementing distributed tracing Manual tracing/instrumentation Before/after calls At the end of each micro-service High maintenance High potential of errors
39. 39 Serverless apps are very distributed Complex systems have thousands of functions What about the developer velocity?
40. 40 Can it be done differently in serverless?
41. 41 Automation can help to keep up with the development speed of serverless
42. 42 Example
43. 43 Example
44. 50 Why you should care about external APIs 702ms e
45. 52 Business flows Subscribe Transfer Payment
46. 53 What should I optimize first?
47. 54 Remember… Adopt serverless Think event-driven Use proper orchestration services Automate observability Go to market – faster!
48. nitzan@epsagon.com @nitzanshapira www.epsagon.com Thank you!