From complexity to observability using OpenTelemetry

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
From complexity to observability
using OpenTelemetry
Chief Evangelist (EMEA), AWS
Danilo Poccia (he/him)
@danilop

View Slide

CloudConf 2013

View Slide

Monolithic
Application
Services Microservices

View Slide

“Complexity arises when
the dependencies among
the elements become
important.”
Scott E. Page, John H. Miller
Complex Adaptive Systems

View Slide

How Amazon SQS works
Front End
Back End
Metadata
Amazon
DynamoDB
Load
Manager

View Slide

“A complex system that
works is invariably found
to have evolved from a
simple system that
worked.”
Gall’s Law

View Slide

“A complex system
designed from scratch
never works and cannot
be patched up to make it
work. You have to start
over with a working
simple system.”

View Slide

“Amazon S3 is intentionally
built with a minimal feature set.
The focus is on simplicity and
robustness.”
– Amazon S3 Press Release,
March 14, 2006

View Slide

Amazon S3
8 → more than 200
microservices
Mai-Lan Tomsen Bukovec
AWS Vice President (Storage, Streaming, Messaging,
and Monitoring/Observability)

View Slide

Coupling: Integration’s magic word
Coupling is a measure of independent
variability between connected systems
Decoupling has a cost, both at design and run-time
Coupling isn’t binary
Coupling isn’t one-dimensional
A B
Source: EnterpriseIntegrationPatterns.com

View Slide

The appropriate level of
coupling depends on the
level of control you have
over the endpoints.
Gregor Hohpe
Enterprise Integration Patterns

View Slide


View Slide

Serverless Serverful

View Slide

Observability & Instrumentation
• “Observability lets us understand a system from the outside, by letting us ask
questions about that system without knowing its inner workings.”
• “In order to be able to ask those questions of a system, the application must be
properly instrumented. That is, the application code must emit signals such as
traces, metrics, and logs.”
• “An application is properly instrumented when developers don’t need to add
more instrumentation to troubleshoot an issue, because they have all of the
information they need.”
https://opentelemetry.io/docs/concepts/observability-primer/#what-is-observability

View Slide

Don’t build a network of connected “black boxes”
Building observable applications
is a developer responsibility

View Slide

OpenTelemetry
An open-source framework for instrumenting, generating,
collecting, and exporting telemetry data

View Slide

What is OpenTelemetry (OTel)?
• CNCF popular project
• Instrument once
• Collect and aggregate
• Send to multiple destinations
• Cloud-centered framework
• Supports hundreds of destinations
and 11 programing languages

View Slide

OpenTelemetry data sources and protocol
• Traces – What happens when a request is made by user or an application
• Metrics – A measurement about a service, captured at runtime
• Logs – A timestamped text record with metadata
• Baggage – Contextual information that’s passed between spans
• OpenTelemetry Protocol (OTLP) – Telemetry data delivery protocol

View Slide

Reference architecture
Code
OTel
Instrumentation
Destination
1
Destination
2
Destination
N
…
Receiver Exporter
Processor
OTel collector

View Slide

Working with
the open-source
community

View Slide

AWS Distro for OpenTelemetry (ADOT)
A secure, production-ready open-source distribution supported by AWS
Certified by AWS for security and predictability
Upstream-first distro of the popular CNCF project
Backed by AWS support
One-click deploy and configure from AWS container and
AWS Lambda consoles
Exporters for AWS monitoring solutions including –
Amazon CloudWatch, AWS X-Ray, Amazon OpenSearch
Service, Prometheus, SigV4Auth, and partner solutions

View Slide

ADOT Public Roadmap

View Slide

Tracing with AWS auto instrumentation agent
ADD https://github.com/aws-observability/
aws-otel-java-instrumentation/releases/latest/download/
aws-opentelemetry-agent.jar /app/aws-opentelemetry-agent.jar
ENV JAVA_TOOL_OPTIONS="-javaagent:/app/aws-opentelemetry-agent.jar"
ENV OTEL_RESOURCE_ATTRIBUTES="service.name=MyApp"
ENV OTEL_IMR_EXPORT_INTERVAL="10000”
ENV OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:4317"
S A M P L E D O C K E R F I L E

View Slide

Custom tracing attributes using annotations
import io.opentelemetry.extension.annotations.WithSpan;
import io.opentelemetry.extension.annotations.SpanAttribute;
public class MyObservedClass {
@WithSpan
public void myObservedMethod(@SpanAttribute("firstParam") String firstParam,
@SpanAttribute("secondParam") long secondParam) {
<...>
}
}
S A M P L E J A V A C O D E

View Slide

Instrument once, send to multiple destinations
P A R T N E R S

View Slide

Demo

View Slide

“Guess the Number” app
• Nice way to test a new programming language or new environments
• Random number between 1 and 100
• Try your guess
• Too high, too low, correct
• What happens if I want to run a scalable, multitenant version?

View Slide

“Guess the Number” app architecture
Internet
Browser
Database
Compute
Winner
Message
Message
Queue
User

View Slide

Internet
Browser
User ??? Amazon DynamoDB
Winner
Message
Amazon SQS

View Slide

Internet
Browser
Winner
Message
AWS App Runner Amazon Elastic Container
Service (Amazon ECS)
Amazon Elastic Kubernetes
Service (Amazon EKS)
AWS Lambda
Amazon SQS

View Slide

View Slide

Internet
Browser
Winner
Message
AWS Lambda
Amazon SQS

View Slide

“Guess the Number” compute
AWS Lambda
Single web container
Managed OTel agent
Tracing (X-Ray)
Any container
OTel sidecar
Simplified console
experience
Tracing
Metrics
Any container
OTel sidecar
EKS add-on support for
ADOT Operator
Tracing
Metrics
Serverless function
X-Ray active tracing
Lambda layer
with OTel agent
Tracing
Metrics

View Slide

Code & Workshop
Code used in the demo
https://github.com/danilop/CloudConf2023-observability-demo
AWS Observability Workshop
https://catalog.workshops.aws/observability

View Slide

Takeaways
Complexity
Observability
OpenTelemetry
AWS Distro for OpenTelemetry (ADOT)
Instrument once → Collect & analyze everywhere

View Slide

Thank you!
@danilop Give me your feedback!

View Slide

From complexity to observability using OpenTelemetry

From complexity to observability using OpenTelemetry

Danilo Poccia

More Decks by Danilo Poccia

Other Decks in Programming

Featured

Transcript