Platform Engineering at Mercari

Talk at Platform Engineering Meetup #2 https://platformengineering.connpass.com/event/280339/

Transcript

1. Platform Engineering at Mercari Platform Engineering Meetup #2
2. Taichi Nakashima @deeeet Director of Platform Engineering
3. Agenda
 • Overview of Mercari Developer Platform • Lessons learned
4. Overview of Mercari Developer Platform
5. Mission of Platform Engineering at Mercari
 Providing core infrastructure, toolings and workflows for Mercari Group developers so that they can deliver new value and experience to Mercari customers quickly and reliably.
6. Mission of Platform Engineering at Mercari
 Providing core infrastructure, toolings and workflows for Mercari Group developers so that they can deliver new value and experience to Mercari customers quickly and reliably.
7. Platform ？
8. Mission of Platform Engineering at Mercari
 Providing core infrastructure, toolings and workflows for Mercari Group developers so that they can deliver new value and experience to Mercari customers quickly and reliably.
9. Build Test Deploy Operate Infrastructure Network Data Interface Documentation On-support Production readiness check Service catalog Template project Common language package/library Protobuf management CI platform QA environment & automation Cloud administration Kubernetes administration Terraform & Kubernetes monorepo CDN administration Cloud network Service mesh Data pipeline and lake Data governance (lineages) CD platform Experimentation platform Supply Chain Security protection Delivery configuration management Observability platform Production access managements
10. Build Test Deploy Operate Platform DX CI/CD Infrastructure Network Platform Infra Network Interface Data Data Operations Experimentation
11. Lessons Learned
12. Lessons Learned
 • Business as First Priority • Platform as Product • Start from “Collaboration” • Keep Appropriate Distance from Stream-aligned Team • Platform Engineering = Migration Engineering
13. Lessons Learned
 • Business as First Priority • Platform as Product • Start from “Collaboration” • Keep Appropriate Distance from Stream-aligned Team • Platform Engineering = Migration Engineering
14. Business as First Priority

15. Lessons Learned
 • Business as First Priority • Platform as Product • Start from “Collaboration” • Keep Appropriate Distance from Stream-aligned Team • Platform Engineering = Migration Engineering
16. Platform as Product
17. Platform as Product
 "Platform as a Product" is a mindset where the internal developer platform is managed continuously like “Product”, by considering developers as “customer,” focusing on meeting its needs. Unlike a Project, which has a finite life cycle with a clear start and end based on achieving specific goals, this approach sees the platform as an ongoing concern that requires continuous refinement and improvement.
18. https://miro.com/app/board/uXjVPhewWSg=/
19. Discovery Exploring the issues developers are facing Evaluating the solution works or not. Deciding which issues to address in which order. Prioritization Validation
20. Lessons Learned
 • Business as First Priority • Platform as Product • Start from “Collaboration” • Keep Appropriate Distance from Stream-aligned Team • Platform Engineering = Migration Engineering
21. Start from “Collaboration”
22. https://teamtopologies.com/book
23. X-as-a-Service / Self-service Stream-Aligned Teams Platform Team
24. Collaboration X-as-a-Service / Self-service
25. Start from “Collaboration”
 • Easy Discovery & Prioritization: Platform team can actually facing the issue as Stream-aligned team and understand it clearly • Easy Validation: Platform Team can get feedback directly from the team and iterate quickly
26. Lessons Learned
 • Business as First Priority • Platform as Product • Start from “Collaboration” • Keep Appropriate Distance from Stream-aligned Team • Platform Engineering = Migration Engineering
27. Keep Appropriate Distance from Stream-aligned Teams
28. Keep Appropriate Distance from Stream-aligned Teams
29. Collaboration X-as-a-Service / Self-service
30. Force of “Collaboration” … When “Collaboration”, there is a natural force to be the same team forever e.g., Platform Team is taking all operation or On-calling 👉 Platform becomes specific to the team. It can not be scaled Collaboration Getting too close
31. Against the force of “Collaboration”
 • Have a deadline of the “Collaboration” • Keep future “X-as-a-Service” in mind • Have a strict line on what NOT to do
32. Getting too further away Force of X-as-a-Service … When “X-as-a-Service”, there is a natural force to getting further and further away from the stream aligned team. Platform Team only cares about its internal … Steam-aligned Team does not know what Platform Team is doing … 👉 Platform Team can not properly discover the issue to be solved or validate the solution … X-as-a-Service
33. Against the force of “X-as-a-Service”
 • Keep the visibility of Platform activity • Have direct feedback channels with Stream-aligned team • Use “Collaboration” for exploring and validation
34. Lessons Learned
 • Business as First Priority • Platform as Product • Start from “Collaboration” • Keep Appropriate Distance from Stream-aligned Team • Platform Engineering = Migration Engineering
35. Platform Engineering = Migration Engineering
36. Migration Project at Mercari
 • Replacing legacy infrastructure/system with new platform ◦ e.g., Moving monolith system from On-prem environment ◦ e.g., Moving SaaS based CI platform to self-hosted • Enabling new features ◦ e.g., Migrating to GKE workload Identity • Applying best practices to the service running on the platform ◦ e.g., Improving resource utilization (FinOps)
37. Time Number of internal tools With Migration Without Migration Reducing the maintenance costs and technical debt Increasing the leverage of tooling improvements to everyone Keeping the system complexity lows Lots of similar tool is built but not maintained…
38. “X-as-a-Service” Makes Migration Hard
 • X-as-a-Service/Self-service means delegating responsibility of workflows or configuration management to Stream-aligned Teams ◦ = Platform Teams can not modify them • Some can be only by Platform Team but the most of migration requires support from Stream-aligned Teams …
39. Tips: Successful Migration
 • Design with Migration in mind • Start from “Collaboration” • Reduce the cost of migration as much as possible • Communicate, communicate, communicate
40. Conclusion
 • Business as First Priority • Platform as Product • Start from “Collaboration” • Keep Appropriate Distance from Stream-aligned Team • Platform Engineering = Migration Engineering