Easy Data Processing on Azure with Serverless Functions

Serverless or Function as a Service (FaaS) enables you to focus more on your code while making it easier to deploy your applications. FaaS can be great tools to perform data processing scenarios. Azure functions allow you to leverage Azure's robust, managed, and scalable cloud computing platform. In this workshop, Tania Allard will teach you how you can get started with Azure functions and Python for data processing scenarios.

✨ Check the tutorial at https://aka.ms/pycon2020-azurefunctions

✨ GitHub repository https://github.com/trallard/pycon2020-azure-functions

🐍 PyCon schedule description: https://us.pycon.org/2020/schedule/presentation/230/

Transcript

1. Tania Allard, PhD PyCon 2020 Easy data processing with Python and Azure functions @ixek
2. • Detailed step-by-step tutorial: aka.ms/pycon2020- azurefunctions • GitHub Repository: https:!//github.com/trallard/pycon2020- azure-functions • Slides:aka.ms/pycon2020-azurefunctions-slides Resources: @ixek
3. Serverless computing Are there no servers at all? @ixek
4. Serverless computing Are there no servers at all? Yes there are but: - You are not responsible for the management or maintenance - Your cloud provider takes care of the provisioning @ixek
5. Serverless computing You focus on the code not the infrastructure @ixek
6. Managed Pay only for what you use Scalable Why Serverless computing? @ixek
7. Managed serverless Microsoft Azure functions @ixek Learn more at: https:!//cda.ms/1g7
8. Handles: software, monitoring, scaling, and hardware Host management Microsoft Azure functions @ixek
9. Are responsible for the application code Decide what services to integrate (i.e databases, Blob Storage, email provider) You - the developer @ixek
10. Why functions? Microsoft Azure functions @ixek
11. Serverless computing A.K.A Functions as a Service (FasS) @ixek
12. Microsoft Azure functions Why functions? FaaS Self contained Code snippets In the cloud @ixek
13. Microsoft Azure functions FaaS Self contained Code snippets In the cloud a.k.a function @ixek
14. Event triggered (idle in the meantime) Coupled services Stateless and short-lived Asynchronous (do not wait for response) Serverless characteristics @ixek
15. Image and video processing Internet of Things Data pipelines Good use cases for FaaS @ixek
16. • You want to collect data from the StackExchange API - to monitor questions from a specific topic or tool • The data needs to be stored for later wrangling and reporting • Ideally, you want to get a daily digest in your inbox with the new questions created as well as a plot of the most commonly used tags in said questions The Scenario @ixek
17. • Python 3.x (supported 3.6, 3.7, 3.8) • VS Code • Python VS Code Extension • Azure Functions VS Code Extension • StackExchange API app key https:!//api.stackexchange.com/ Requirements: @ixek
18. Getting started - timer @ixek Timer trigger: Every day at 9am Function gets executed Data is collected from StackExchange
19. Triggers vs bindings @ixek Triggers Idle function Trigger Active function
20. Triggers vs bindings @ixek Bindings Idle function Trigger Active function In Binding Out Bindings Trigger Binding
21. Extend function @ixek Timer trigger: Every day at 9am Function gets executed Data is collected from StackExchange Store in Blob Storage (csv) Trigger Binding
22. Completing the scenario @ixek Function gets executed File added in Blob Storage (csv) Process data / Create plots Save plots Send digest Retrieve Data Trigger Binding
23. • Detailed step-by-step tutorial: aka.ms/pycon2020- azurefunctions • GitHub Repository: https:!//github.com/trallard/pycon2020- azure-functions • Slides:aka.ms/pycon2020-azurefunctions-slides Resources: @ixek
24. Hope you enjoyed the tutorial