new API object types • Akin to a database table’s schema - the data model • Designed with custom automation mechanisms in mind • https://kubernetes.io/docs/user-guide/thirdpartyresources/
Bad configurations can disrupt the service. • Tweaking the storage parameters takes time. • Network acls. Lessons learned: • Each team should have it’s own prometheus stack. • Divide and conquer.
to quickly provision a dedicated stack. ◦ Don’t represent any additional burden to the teams. ◦ Provide pre configured EC2 and k8s service discovery. ◦ Helm based deployment. • Ticketmaster exporter database ◦ Provides a well known port range for the exporters. ◦ Managed Network ACLs. ◦ Scrape jobs are generated from this list.
• Storage configuration is auto tuned. • Alertmanager HA by default. Looking forward: • Federation and sharding. • Grafana integration. Company adoption rate: • Everyone loves it!.