Upgrade to Pro — share decks privately, control downloads, hide ads and more …

From complexity to observability using OpenTele...

From complexity to observability using OpenTelemetry

CloudConf, Turin, May 16th, 2023

Understanding the behavior of modern applications can be challenging because they are built using many components that interact with each other. Starting with an application deployed using containers and serverless functions, we use OpenTelemetry to instrument and collect telemetry data such as traces and metrics. Then, we visualize telemetry data to gain insight into the running system, detect anomalies, troubleshoot issues, and perform root-cause analysis. Through this end-to-end example, you will learn how to build observable applications.

Danilo Poccia

May 16, 2023
Tweet

More Decks by Danilo Poccia

Other Decks in Programming

Transcript

  1. © 2023, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. From complexity to observability using OpenTelemetry Chief Evangelist (EMEA), AWS Danilo Poccia (he/him) @danilop
  2. © 2023, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Monolithic Application Services Microservices
  3. © 2023, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. “Complexity arises when the dependencies among the elements become important.” Scott E. Page, John H. Miller Complex Adaptive Systems
  4. © 2023, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. How Amazon SQS works Front End Back End Metadata Amazon DynamoDB Load Manager
  5. © 2023, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. “A complex system that works is invariably found to have evolved from a simple system that worked.” Gall’s Law
  6. © 2023, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. “A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system.”
  7. © 2023, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. “Amazon S3 is intentionally built with a minimal feature set. The focus is on simplicity and robustness.” – Amazon S3 Press Release, March 14, 2006
  8. © 2023, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Amazon S3 8 → more than 200 microservices Mai-Lan Tomsen Bukovec AWS Vice President (Storage, Streaming, Messaging, and Monitoring/Observability)
  9. © 2023, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Coupling: Integration’s magic word Coupling is a measure of independent variability between connected systems Decoupling has a cost, both at design and run-time Coupling isn’t binary Coupling isn’t one-dimensional A B Source: EnterpriseIntegrationPatterns.com
  10. © 2023, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. The appropriate level of coupling depends on the level of control you have over the endpoints. Gregor Hohpe Enterprise Integration Patterns
  11. © 2023, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Serverless Serverful
  12. © 2023, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Observability & Instrumentation • “Observability lets us understand a system from the outside, by letting us ask questions about that system without knowing its inner workings.” • “In order to be able to ask those questions of a system, the application must be properly instrumented. That is, the application code must emit signals such as traces, metrics, and logs.” • “An application is properly instrumented when developers don’t need to add more instrumentation to troubleshoot an issue, because they have all of the information they need.” https://opentelemetry.io/docs/concepts/observability-primer/#what-is-observability
  13. © 2023, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Don’t build a network of connected “black boxes” Building observable applications is a developer responsibility
  14. © 2023, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. OpenTelemetry An open-source framework for instrumenting, generating, collecting, and exporting telemetry data
  15. © 2023, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. What is OpenTelemetry (OTel)? • CNCF popular project • Instrument once • Collect and aggregate • Send to multiple destinations • Cloud-centered framework • Supports hundreds of destinations and 11 programing languages
  16. © 2023, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. OpenTelemetry data sources and protocol • Traces – What happens when a request is made by user or an application • Metrics – A measurement about a service, captured at runtime • Logs – A timestamped text record with metadata • Baggage – Contextual information that’s passed between spans • OpenTelemetry Protocol (OTLP) – Telemetry data delivery protocol
  17. © 2023, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Reference architecture Code OTel Instrumentation Destination 1 Destination 2 Destination N … Receiver Exporter Processor OTel collector
  18. © 2023, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Working with the open-source community
  19. © 2023, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. AWS Distro for OpenTelemetry (ADOT) A secure, production-ready open-source distribution supported by AWS Certified by AWS for security and predictability Upstream-first distro of the popular CNCF project Backed by AWS support One-click deploy and configure from AWS container and AWS Lambda consoles Exporters for AWS monitoring solutions including – Amazon CloudWatch, AWS X-Ray, Amazon OpenSearch Service, Prometheus, SigV4Auth, and partner solutions
  20. © 2023, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. ADOT Public Roadmap
  21. © 2023, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Tracing with AWS auto instrumentation agent ADD https://github.com/aws-observability/ aws-otel-java-instrumentation/releases/latest/download/ aws-opentelemetry-agent.jar /app/aws-opentelemetry-agent.jar ENV JAVA_TOOL_OPTIONS="-javaagent:/app/aws-opentelemetry-agent.jar" ENV OTEL_RESOURCE_ATTRIBUTES="service.name=MyApp" ENV OTEL_IMR_EXPORT_INTERVAL="10000” ENV OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:4317" S A M P L E D O C K E R F I L E
  22. © 2023, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Custom tracing attributes using annotations import io.opentelemetry.extension.annotations.WithSpan; import io.opentelemetry.extension.annotations.SpanAttribute; public class MyObservedClass { @WithSpan public void myObservedMethod(@SpanAttribute("firstParam") String firstParam, @SpanAttribute("secondParam") long secondParam) { <...> } } S A M P L E J A V A C O D E
  23. © 2023, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Instrument once, send to multiple destinations P A R T N E R S
  24. © 2023, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Demo
  25. © 2023, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. “Guess the Number” app • Nice way to test a new programming language or new environments • Random number between 1 and 100 • Try your guess • Too high, too low, correct • What happens if I want to run a scalable, multitenant version?
  26. © 2023, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. “Guess the Number” app architecture Internet Browser Database Compute Winner Message Message Queue User
  27. © 2023, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. “Guess the Number” app architecture Internet Browser User ??? Amazon DynamoDB Winner Message Amazon SQS
  28. © 2023, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. “Guess the Number” app architecture Internet Browser User ??? Amazon DynamoDB Winner Message AWS App Runner Amazon Elastic Container Service (Amazon ECS) Amazon Elastic Kubernetes Service (Amazon EKS) AWS Lambda Amazon SQS
  29. © 2023, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. “Guess the Number” app architecture Internet Browser User ??? Amazon DynamoDB Winner Message AWS App Runner Amazon Elastic Container Service (Amazon ECS) Amazon Elastic Kubernetes Service (Amazon EKS) AWS Lambda Amazon SQS
  30. © 2023, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. “Guess the Number” compute AWS App Runner Amazon Elastic Container Service (Amazon ECS) Amazon Elastic Kubernetes Service (Amazon EKS) AWS Lambda Single web container Managed OTel agent Tracing (X-Ray) Any container OTel sidecar Simplified console experience Tracing Metrics Any container OTel sidecar EKS add-on support for ADOT Operator Tracing Metrics Serverless function X-Ray active tracing Lambda layer with OTel agent Tracing Metrics
  31. © 2023, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Code & Workshop Code used in the demo https://github.com/danilop/CloudConf2023-observability-demo AWS Observability Workshop https://catalog.workshops.aws/observability
  32. © 2023, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Takeaways Complexity Observability OpenTelemetry AWS Distro for OpenTelemetry (ADOT) Instrument once → Collect & analyze everywhere
  33. © 2023, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Thank you! © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. @danilop Give me your feedback!