Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Easy Data Processing on Azure with Serverless Functions

Easy Data Processing on Azure with Serverless Functions

Serverless or Function as a Service (FaaS) enables you to focus more on your code while making it easier to deploy your applications. FaaS can be great tools to perform data processing scenarios. Azure functions allow you to leverage Azure's robust, managed, and scalable cloud computing platform. In this workshop, Tania Allard will teach you how you can get started with Azure functions and Python for data processing scenarios.

✨ Check the tutorial at https://aka.ms/pycon2020-azurefunctions

✨ GitHub repository https://github.com/trallard/pycon2020-azure-functions

🐍 PyCon schedule description: https://us.pycon.org/2020/schedule/presentation/230/

Tania Allard

April 24, 2020
Tweet

More Decks by Tania Allard

Other Decks in Technology

Transcript

  1. Tania Allard, PhD
    PyCon 2020
    Easy data processing with
    Python and Azure functions
    @ixek

    View full-size slide

  2. • Detailed step-by-step tutorial: aka.ms/pycon2020-
    azurefunctions
    • GitHub Repository: https:!//github.com/trallard/pycon2020-
    azure-functions
    • Slides:aka.ms/pycon2020-azurefunctions-slides
    Resources:
    @ixek

    View full-size slide

  3. Serverless computing
    Are there no servers at all?
    @ixek

    View full-size slide

  4. Serverless computing
    Are there no servers at all?
    Yes there are but:
    - You are not responsible for the management or
    maintenance
    - Your cloud provider takes care of the
    provisioning
    @ixek

    View full-size slide

  5. Serverless computing
    You focus on the code not the infrastructure
    @ixek

    View full-size slide

  6. Managed
    Pay only for what you use
    Scalable
    Why Serverless computing?
    @ixek

    View full-size slide

  7. Managed serverless
    Microsoft Azure functions
    @ixek
    Learn more at: https:!//cda.ms/1g7

    View full-size slide

  8. Handles: software, monitoring, scaling, and
    hardware
    Host management
    Microsoft Azure functions
    @ixek

    View full-size slide

  9. Are responsible for the application code
    Decide what services to integrate (i.e databases,
    Blob Storage, email provider)
    You - the developer
    @ixek

    View full-size slide

  10. Why functions?
    Microsoft Azure functions
    @ixek

    View full-size slide

  11. Serverless computing
    A.K.A Functions as a Service (FasS)
    @ixek

    View full-size slide

  12. Microsoft Azure functions
    Why functions?
    FaaS
    Self contained
    Code snippets
    In the cloud
    @ixek

    View full-size slide

  13. Microsoft Azure functions
    FaaS
    Self contained
    Code snippets
    In the cloud
    a.k.a function
    @ixek

    View full-size slide

  14. Event triggered (idle in the meantime)
    Coupled services
    Stateless and short-lived
    Asynchronous (do not wait for response)
    Serverless characteristics
    @ixek

    View full-size slide

  15. Image and video processing
    Internet of Things
    Data pipelines
    Good use cases for FaaS
    @ixek

    View full-size slide

  16. • You want to collect data from the StackExchange API - to
    monitor questions from a specific topic or tool
    • The data needs to be stored for later wrangling and
    reporting
    • Ideally, you want to get a daily digest in your inbox with
    the new questions created as well as a plot of the most
    commonly used tags in said questions
    The Scenario
    @ixek

    View full-size slide

  17. • Python 3.x (supported 3.6, 3.7, 3.8)
    • VS Code
    • Python VS Code Extension
    • Azure Functions VS Code Extension
    • StackExchange API app key https:!//api.stackexchange.com/
    Requirements:
    @ixek

    View full-size slide

  18. Getting started - timer
    @ixek
    Timer trigger:
    Every day at 9am
    Function gets
    executed
    Data is collected
    from StackExchange

    View full-size slide

  19. Triggers vs bindings
    @ixek
    Triggers
    Idle function
    Trigger
    Active function

    View full-size slide

  20. Triggers vs bindings
    @ixek
    Bindings
    Idle function
    Trigger
    Active function
    In Binding
    Out Bindings
    Trigger
    Binding

    View full-size slide

  21. Extend function
    @ixek
    Timer trigger:
    Every day at 9am
    Function gets
    executed
    Data is collected
    from StackExchange
    Store in Blob
    Storage (csv)
    Trigger
    Binding

    View full-size slide

  22. Completing the scenario
    @ixek
    Function gets
    executed
    File added in Blob
    Storage (csv)
    Process data /
    Create plots
    Save plots
    Send digest
    Retrieve
    Data
    Trigger
    Binding

    View full-size slide

  23. • Detailed step-by-step tutorial: aka.ms/pycon2020-
    azurefunctions
    • GitHub Repository: https:!//github.com/trallard/pycon2020-
    azure-functions
    • Slides:aka.ms/pycon2020-azurefunctions-slides
    Resources:
    @ixek

    View full-size slide

  24. Hope you enjoyed the tutorial

    View full-size slide