Observability both on and for an internal platform

Platforms are seen as the cure for slow, risky, and unmanageable software engineering. But they, too, are software. This talk looks at how platforms not only provide users with access to observability, but also rely on it for their success.

Transcript

1. Observability both on and for an internal platform Abby Bangser (she/her) abby@syntasso.io
2. Observability on the platform is about providing support for application developers
3. Questions answered include: How can we manage data consistently and securely? Where does cost awareness and optimisation get addressed? What can we do to reduce vendor lock in? Can new projects get observability "for free"?
4. And you end up spending 90% of your time on the infrastructure…
5. And (maybe) 10% of time on app developer experience
6. So today may be about observability…
7. So today may be about observability… but you know that stuff.
8. So today may be about observability… but you know that stuff. So let's look at that DevEx. Let's talk platforms.
9. DevOps was a DevEx improvement.
10. DevOps was a DevEx improvement. And platforms is the anti-DevOps.
11. DevOps was a DevEx improvement. And platforms is the anti-DevOps. But not in the way you may think 😉
12. Reviewing DevOps likely feels obvious. But it is a part of this journey so humour me 🙏
13. Agile brought people together.
14. Agile brought people together. DevOps wanted to extend this. Operations Development
15. Silos are everywhere Finance Operations Development Testing Security
18. DevOps reduces friction between silos through automation, collaboration, fast feedback, and iterative improvement
19. The most popular way to remove friction was to remove the silos Operations Development
20. Introduction of full stack succeeded in increasing collaboration, fast feedback and iterative improvement.
21. But that isn't the full story. It is also where the DevEx cracks start to show.
22. Every team doesn't have the skills or interest to deliver every capability .
23. Every team doesn't have the skills or interest to deliver every capability. Even if they do, is this secure and cost effective for the company?
24. Complete autonomy can lead to debilitating responsibility .
25. Complete autonomy can lead to debilitating responsibility . Especially at scale.
26. Complete autonomy can lead to debilitating responsibility. Especially at scale. That doesn't discount DevOps as both effective and enjoyable !
27. Platform engineering enables DevOps principles at scale
28. "Application teams build, deploy, monitor, and are on call for the application components [...].
29. This is where the observability on the platform comes into play
30. Platforms needs to build instrumentation in
31. Platforms needs to build instrumentation in Allowing app developers to focus on enriching and using the data
32. Platforms shorten the application "full stack"
33. Allowing application developers to spend time with their code
34. https://cnoe.io/ https://thenewstack.io/crossplane-what-most-people-get-wrong-and-how-to-get-it-right/ And not just that… Platforms are their own shorter software stack. https://www.kratix.io/ https://platformengineering.org/platform-tooling
35. And those platform engineers need to spend time with their code too
36. "Application teams build, deploy, monitor, and are on call for the application components [...]. Platform teams build, deploy, monitor, and are on call for the platform components"
37. Think microservices meets vendor engineering
38. This is where the observability for the platform comes into play
39. Clear and trusted contracts differentiate platform engineering from historical "ops"
40. DevOps reduces friction between silos through automation, collaboration, fast feedback, and iterative improvement
41. DevOps reduces friction between silos through automation, collaboration, fast feedback, and iterative improvement Moar stacks (silos) has historically meant moar friction
42. Platform engineering enables DevOps principles at scale by transitioning understood offerings from collaboration mode to services .
43. Collaboration is one of three interaction modes
44. Collaboration: Optimised for empathy and exploration.
45. X-as-a-Service: Optimised for scale and autonomy
46. Interaction models are not permanent.
47. Interaction models are not permanent. And they not only evolve, they benefit from intentional transitions
48. This collaboration and the resulting services depend on a reinforcing cycle of observability
49. Anyone writing software needs observability.
50. Anyone writing software needs observability. Anyone using platforms needs visibility.
51. Anyone writing software needs observability. Anyone using platforms needs visibility. To provide useful visibility, you need to have observability.
52. S3 doesn't expose CPU, instead it expose retrieval time
53. Do your application developers really need ArgoCD UI?
54. Getting vendor observability right affects everything
55. Right-sized observability increases adoption through earned trust
56. Right-sized observability reduces costs through focusing support requests
57. Right-sized observability improves platforms through clearer user behaviour
58. • Platforms provide services, and observability is a key part of any service • Platforms need to provide observability offerings on the platform to: ◦ Encourage consumer ownership of their resources for application developers ◦ Support cost, security, and data management as an economy of scale • Platforms need observability as a characteristic for the platform to: ◦ Enable maintainability of the platform service for platform engineers ◦ Breed trust in the platform contract between platform and app developers • Platforms are not just software, they are vendor software ◦ Provide on-demand access, built in security and governance, and scale effectively through fleet management Takeaways
59. Thank you Abby Bangser (she/her) abby@syntasso.io