Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Google Cloud Serverless for .NET developers

Google Cloud Serverless for .NET developers

Mete Atamel

October 01, 2021
Tweet

More Decks by Mete Atamel

Other Decks in Programming

Transcript

  1. Google Cloud Serverless for .NET developers Mete Atamel Developer Advocate

    at Google @meteatamel atamel.dev speakerdeck.com/meteatamel
  2. Legacy architecture for .NET apps End Users On-Prem Data Center

    Load Balancer IIS (.NET) Backend (.NET) MSMQ SQL Server Active Directory
  3. Compute Engine Windows Server App Engine Flex* Kubernetes Engine Linux

    containers .NET on Windows .NET (Core) on Linux Cloud Run Where should I run my .NET stuff? Cloud Functions Kubernetes Engine Windows containers Serverless * Not really serverless!
  4. Serverless functions Serverless web applications Serverless HTTP containers Source code-based

    event driven functions Source code-based web apps and API backends HTTP containers fully managed HTTP containers on GKE cluster Cloud Functions App Engine Cloud Run Serverless options
  5. Backend functions that trigger in response to an event Microservices

    Cloud "glue" Certain versions of Node.js, Python, Go, Java, .NET, PHP, Ruby Cloud Functions
  6. Cloud Functions as cloud glue Access Google services Cloud Storage

    Cloud Pub/Sub HTTPS Firebase Cloud Scheduler Cloud Tasks
  7. Versions supported by Cloud Functions Node.js 10, 12, 14 Python

    3.7, 3.8, 3.9 Go 1.11, 1.13 Java 11 .NET Core 3.1 PHP 7.4 Ruby 2.6, 2.7
  8. Cloud Run Fully managed, deploy your workloads and don’t see

    the cluster. Cloud Run on Anthos Deploy into Anthos, run serverless side-by-side with your existing workloads. Knative Everywhere Use the same APIs and tooling anywhere you run Kubernetes with Knative. Serverless containers with Knative and Cloud Run
  9. HTTPS Endpoint Public • Website • API endpoint Private •

    Internal services • Async tasks • Mobile backend • Webhook
  10. Container contract Listen on 0.0.0.0 on port $PORT (default 8080)

    HTTP server must start < 4 min (timeout → 504) Request time < 60 min (default → 5 min) Stateless (in-memory file system, doesn’t persist) Computation only within request (No background activity)
  11. Container resources 1 vCPU per container instance (configurable to 4

    vCPU) 256 MiB of memory up to a max of 8 GiB (configurable) 80 concurrent requests per container (configurable 1-1000) 100 max containers by default (configurable 1-1000) Access to a Metadata Server Sandboxed by gVisor
  12. Billable time Instance Billable Time Request 1 Start Request 1

    End Request 2 Start Request 2 End Instance Time Billable Non-billable
  13. Custom Sources And more ... Workflows Orchestrated Cloud Functions Cloud

    Scheduler Eventarc Event-driven Cloud Run Cloud Run for Anthos GKE Compute Engine Pub/Sub Sources BigQuery Cloud Storage Google Sources Targets Serverless integration
  14. Asynchronously deliver events from a wide selection of sources to

    many targets Private alpha since early 2020 as Events for Cloud Run Public GA as Eventarc in Jan 2021 Eventarc
  15. Google Cloud GA Your own apps GA Cloud Run GA

    Cloud Functions Preview Workflows Preview Targets Sources Cloud Run for Anthos on GKE Preview Eventarc Triggers to filter events CloudEvents format
  16. New message in a Pub/Sub topic → Cloud Run gcloud

    eventarc triggers create trigger-pubsub \ --destination-run-service=$SERVICE_NAME \ --destination-run-region=$REGION \ --event-filters="type=google.cloud.pubsub.topic.v1.messagePublished" --transport-topic=projects/$PROJECT_ID/topics/$TOPIC_ID Pub/Sub Trigger
  17. Create a trigger for new object creation in Cloud Storage

    → Cloud Run gcloud eventarc triggers create trigger-gcs \ --destination-run-service=$SERVICE_NAME \ --destination-run-region=$REGION --event-filters="type=google.cloud.storage.object.v1.finalized" \ --event-filters="bucket=$BUCKET_NAME" \ --service-account=$PROJECT_NO-compute@developer.gserviceaccount.com Cloud Storage Trigger
  18. New Compute Engine VM creation → Cloud Run gcloud eventarc

    triggers create trigger-auditlog \ --destination-run-service=$SERVICE_NAME \ --destination-run-region=$REGION --event-filters="type=google.cloud.audit.log.v1.written" \ --event-filters="serviceName=compute.googleapis.com" \ --event-filters="methodName=beta.compute.instances.insert" \ --service-account=$PROJECT_NO-compute@developer.gserviceaccount.com Audit Log Trigger
  19. POST / HTTP/1.1 Content-Type: application/json; charset=utf-8 Content-Length: 33 ce-specversion: 1.0

    ce-type: google.cloud.pubsub.topic.publish ce-time: 2020-09-05T03:56:24Z ce-id: 1234-1234-1234 ce-source: mycontext/subcontext custom-attr: 42 { "message": "Hello Cloud Next!" } CloudEvents cloudevents.io "Data" "Context"
  20. Cloud Run Service CloudEvent SDK HTTP request → CloudEvent in

    your language cloudevents.io Google Events Library Type library for CloudEvent#data Event parsing libraries HTTP POST to Cloud Run URL HTTP body is a CloudEvent with event data ("binary" CloudEvent V1) (optional) (optional) github.com/googleapis/google-cloudevents
  21. const { HTTP } = require("cloudevents"); const {toLogEntryData} = require('@google/events/cloud/storage/v1/StorageObjectData')

    app.post('/', async (req, res) => { // Read CloudEvent using CloudEvents SDK const cloudEvent = HTTP.toEvent({ headers: req.headers, body: req.body }); // Read Cloud Storage event using Google.Events library for Node.js const storageObjectData = toStorageObjectData(cloudEvent.data); // Extract bucket and objectName const bucket = storageObjectData.bucket; const objectName = storageObjectData.name;
  22. using CloudNative.CloudEvents; using Google.Events; using Google.Events.Protobuf.Cloud.PubSub.V1; public async Task<CloudEvent> Read(HttpContext

    context) { var formatter = CloudEventFormatterAttribute.CreateFormatter(typeof(MessagePublishedData)); // Read CloudEvent using CloudEvents SDK var cloudEvent = await context.Request.ToCloudEventAsync(formatter); // Read Pub/Sub message using Google.Events library for .NET var messagePublishedData = (MessagePublishedData)cloudEvent.Data; // Extract the Pub/Sub message var pubSubMessage = messagePublishedData.Message;
  23. Serverless Compute External API’s Google API’s etc... Workflows - orchestrate

    & integrate SaaS API’s Private API’s Other Clouds
  24. - processPayment: call: http.post args: url: https://payment-processor.run.app/... body: input: ${paymentDetails}

    result: processResult - shipItems: call: http.post args: url: https://.../cloudfunctions.net/ship body: input: ${processResult.body} result: shipResult - notifyUser: call: http.post ... Payment Processor Cloud Run Authorize & charge CC Notifier Cloud Run Notify user Shipper Cloud Functions Prepare & ship items YAML or JSON syntax
  25. Payment Processor Cloud Run Authorize & charge CC Notifier Cloud

    Run Notify user Shipper Cloud Functions Prepare & ship items Payment Processor Cloud Run Authorize & charge CC Notifier Cloud Run Notify user Shipper Cloud Functions Prepare & ship items WAIT Payment Processor Cloud Run Authorize & charge CC Notifier Cloud Run Notify user Shipper Cloud Functions Prepare & ship items shipmentDetails userDetails Step Sequencing Serverless Pause Variable passing JSON Parsing Steps
  26. Errors and retries Payment Processor Cloud Run Authorize & charge

    CC Notifier Cloud Run Notify user Shipper Cloud Functions Prepare & ship items MAX: 5 times BACKOFF Payment Processor Cloud Run Authorize & charge CC Notifier Cloud Run Notify user Shipper Cloud Functions Prepare & ship items Pager Cloud Run Escalate to support SUCCESS ERROR Configurable retries Configurable exception handling
  27. Conditionals and 3rd party calls Notifier Cloud Run Notify user

    Shipper Cloud Functions Prepare & ship items Pager Cloud Run Escalate to support SUCCESS ERROR Out of Stock? No Request from the supplier Yes Read inventory Inventory DB Update inventory Inventory DB Supplier API
  28. Other useful features Subworkflows to encapsulate common reusable flows Connectors

    to connect to other Google Cloud services & APIs More iterations, callbacks (preview)
  29. Deploy, execute, manage workflows # Deploy a workflow gcloud workflows

    deploy my-workflow --source=workflow.yaml # Execute a workflow gcloud workflows execute my-workflow # See the result gcloud workflows executions describe <your-execution-id> --workflow my-workflow
  30. Image Processing Pipeline - Eventarc and Workflows End Users Images

    Input Cloud Storage Images Output Cloud Storage Filter Cloud Run Cloud Storage Trigger Eventarc Image Processing Workflows Watermarker Cloud Functions Resizer Cloud Functions Labeler Cloud Functions github.com/GoogleCloudPlatform/eventarc-samples/tree/main/processing-pipelines/image-workflows