Python Serverless Microservices - Workshop PyCon SE 2020

Transcript

1. Python Serverless Microservices Workshop Nilo Ney Coutinho Menezes (nilo@logikraft.be)
2. Plan • Short introduction to serverless • Examples: 1. Simple Example 2. Show IP and country 3. URL Shortener
3. Serverless Application Framework • Written in JavaScript, but works with multiple languages • Uses Node, Python and Chocolatey (alternative Windows Package Manager) • Deploys the serverless infrastructure in the cloud. Compatible with Amazon Web Services, Google Cloud, Microsoft Azure and others • Simplifies configuration and application updates • Configuration files written in YAML • https://www.serverless.com/
4. Server less? • We know the server still there • But we don’t have to keep it running • We don’t have to update it frequently • We can have more as we need them (autoscaling) • We only pay if we use them (per call, memory and run time) • So, it is an industry term to say somebody else is taking care of the server for you.
5. Microservices • Microservices are an architecture for development and deployment of new systems • It antagonizes the traditional monolithic architecture that deploys the full system in a single application • Microservices means we should split our systems into smaller ones • Smaller parts are easy to understand, can be deployed independently • Share no database with other parts • Share data and services via a well-defined API • Can be rewritten one by one (less coupling)
6. Lambdas • Lambdas are Functions as a Service (FaaS) provided by Amazon Web Services (AWS) • We will be using AWS as our example provider • You need to have an account to test the code shown in this presentation • You can create a free account on their web site: https://aws.amazon.com/ • If it is your first time on AWS, I strongly suggest you check their web site and documentation, as you will be paying for the resources used beyond the free tier. • To start using AWS with Python, install their command-line client: pip install awscli
7. Python Serverless Microservices • Because we use Python • Because we use AWS Lambdas, FaaS, serverless • And we would divide each API call in a different lambda, grouped in a domain, following the directives of Microservices and Domain-Driven Design (DDD). Check these books for more info:
8. 1 – Simple example
9. Installing • Download and install node.js from https://nodejs.org/en/ • Install the Serverless Application Framework: npm install –g serverless • Let’s created the application stub type: serverless • Try to answer the questions as in the image below:
10. The stub application: handler.py • Please rename the file to hello.py
11. Serverless.yml • This is all the configuration we need for our simple function. • It says we are going to deploy a Python 3.8 Lambda Function • It defines the handler (the module and function name) • It also specifies the URL path to use and the http method
12. Deploying • Serverless will deploy all we need with a single command sls deploy • Your urls will be different.
13. Visiting our new page Go to: https://dg68p0g319.execute-api.eu-west-1.amazonaws.com/dev/pyconse2020/hello Or https://bit.ly/PyConSE-Hello
14. The super user-friendly page :-D
15. What is in this json? • In the input field, all parameters passed to your function • Copy and paste it to your preferred text editor that supports JSON. Reformat the code to make it easier to read. • You get user location with IP and country information • The URL and path used on this call • Any parameter used (query string and path) if any • You can access all these values as in simple Python dictionary
16. What do we have now
17. 2 – Showing IP and Country
18. Showing our IP and country • Using just the headers available to our lambda, let’s display a simple page with the callers IP and country. • We need to return a html page now. • Better use templates to offload the work to Jinja2 pip install jinja2 • We also must tell serverless to deploy or modules with the lambda function sls plugin install -n serverless-python- requirements
19. Show me the code
20. Let’s test it
21. 3 – URL Shortener
22. URL Shortener • I suppose you will get a short domain name :-D • Simple service  Get URL  Return link with short code  When the short code is accessed: - Count access (increment database field) - Redirect user to the URL - Codes will be deleted after 2 days
23. What we will do • We will reuse the same function to process GET and POST requests • GET will show a simple form with a field to enter the URL • POST will process the user input, create a record in the database and return a page with the links created. • Another function will get the code and redirect to the user url, incrementing the counter.
24. We need a table • Let’s ask serverless to create a table for us.
25. The Database • We are using DynamoDB • It is a key value database in AWS • It is also serverless and we pay for the storage and requests made • Super fast • Scalable • Does not support SQL • Really odd at the beginning
26. Our table • We designed a very simple table with a hash key (primary key) • All other fields are free, no schema • Using PynamoDB to interact with DynamoDB API
27. PynamoDB Operations Checking if a key already exists Creating a new record Querying data Updating the counter
28. What did we do?
29. In serverless • We create two functions • One for create that handles GETs and POST • Another one that handles GETs • Look how their urls are different. The function is chosen by pattern matching • In the useURL, the {short_key} is a parameter passed to the lambda in the event • We are using layers for dependencies • And a role to have access to the database
30. How does it look?
31. Wait, no huge framework? • Nope • You have a very simple function deployed to the cloud • You choose the batteries • You can combine multiple Python libraries depending on your needs • Leverage the cloud provider services like databases, api gateways, queues, etc
32. Advantages • Simple, you can read the function from top to bottom, no hidden magic code • Cheap, millions of invocations for less than 1 euro • Pays as you go (no upfront costs, if you don’t use, you won’t pay either) • Functions can be deployed independently • No downtime, old version keeps running until the new one is deployed
33. Disadvantages • As any new thing can be difficult at the beginning • You get dependent to a cloud provider • More tools to use • Lack of framework to start with
34. What’s next • Add more functions • Play with AWS Cognito to support authenticated users • Read about lambda layers • Other frameworks to help with Lambdas and Python:  AWS Chalice https://github.com/aws/chalice  AWS SAM https://aws.amazon.com/serverless/sam/
35. Thank you • All the code is hosted on GitHub https://github.com/lskbr/pyconse2020 • You may find this presentation on SpeakerDeck https://speakerdeck.com/lskbr • If you have any questions, do not hesitate to contact me: nilo@logikraft.be @lskbr on Twitter and Telegram