(Semi-)Continuous Delivery at New Relic

From the inaugural San Francisco Continuous Delivery Meetup (http://www.meetup.com/Continuous-Delivery-San-Francisco/events/71332792/)

1. (Semi-)Continuous Delivery at New Relic Ralph Bodenner Director of Engineering, Core App @ralphbod Friday, July 27, 2012
2. Who, What, Why Friday, July 27, 2012
3. Dr. Paul MacCready Friday, July 27, 2012
4. Gossamer Condor Friday, July 27, 2012
5. Marketing Slide Friday, July 27, 2012
6. Two Software Companies In One Customer’s environment 2 Web App Servers HTTPS (our agent in our customer’s app) Sustained insertion rate 100k per second 24 Core Intel Nehalem 48 GB RAM SAS attached RAID 5 No Virtualization 12 Core Intel Nehalem 48 GB RAM Friday, July 27, 2012
7. Continuous Delivery • Frequent automated deployments • Monitoring that provides immediate actionable feedback • Master branch is always ready to be delivered to customers • Continuous deployment reduces technical risk • Continuous delivery reduces business risk • Testing is not a phase • Done means delivering value to customers Friday, July 27, 2012
8. Continuous Delivery of On-Premise Friday, July 27, 2012
9. Regular Heartbeats • Assumption: customers want the most up-to-date agents = wrong • Assumption: customers want big changes = wrong • Assumption: customers want predicatability = yes! • Lesson #1: Tailor your continuous delivery to your customers expectations Friday, July 27, 2012
10. Your First Day at New Relic Friday, July 27, 2012
11. Weekly Releases Friday, July 27, 2012
12. Daylight Releases Friday, July 27, 2012
13. Restructured the Organization • Merged QA into Development • Not a popular decision for some • Eliminated “complete, waiting for QA” from vocabulary • Quality Engineers are involved from day one • Lesson #2: Quality is not a separate activity, so don’t have a separate QA department Friday, July 27, 2012
14. Why Were We Waiting? • Lesson #3: Continuous delivery requires changing your social processes (as well as more automation) Friday, July 27, 2012
15. Deploy Captain Friday, July 27, 2012
16. Alerts (but...) Friday, July 27, 2012
17. Drinking Our Own Champagne Friday, July 27, 2012
18. What More Can We Do? • Architecture changes • Distinguish errors • More feature flags • No-code deploys • Longer-running work Friday, July 27, 2012
19. Now with Services • Growing company leads to bigger teams leads to splitting teams • ...leads to services • We were too casual about the split and just “did our side” • Lesson #4: monitor API users like you do customers Friday, July 27, 2012
20. Now with Services • Growing company leads to bigger teams leads to splitting teams • ...leads to services • We were too casual about the split and just “did our side” • Lesson #4: monitor API users like you do customers “[when] doing a big SOA ... monitoring and QA are the same thing” - Steve Yegge, Google, ex-Amazon http://bit.ly/yegge_rant Friday, July 27, 2012
21. Another Reason • We do Continuous Delivery for a somewhat different reason than most companies • Lesson #5: Continuous Delivery isn’t just for the customers Friday, July 27, 2012
22. Gossamer Penguin Friday, July 27, 2012
23. (Semi-)Continuous Delivery at New Relic Ralph Bodenner ralph@newrelic.com @ralphbod Friday, July 27, 2012