Modelling and Managing Deployment Costs of Microservice-Based Cloud Applications

Talk given at UCC'16

1. software evolution & architecture lab University of Zurich, Switzerland Modelling and Managing Deployment Costs of Microservice-Based Cloud Applications Philipp Leitner, Jürgen Cito, Emanuel Stöckli @xLeitix
2. Microservices FTW! Customer-Facing Web Interface /get order history /save /generate recommend /get customers /login orders /get /submit banking_ interface /process_ payment /landing frontend /order products /get
3. Microservices FTW! Customer-Facing Web Interface /get order history /save /generate recommend /get customers /login orders /get /submit banking_ interface /process_ payment /landing frontend /order products /get
4. Microservices FTW! Customer-Facing Web Interface /get order history /save /generate recommend /get customers /login orders /get /submit banking_ interface /process_ payment /landing frontend /order products /get
5. Microservices FTW! Customer-Facing Web Interface /get order history /save /generate recommend /get customers /login orders /get /submit banking_ interface /process_ payment /landing frontend /order products /get
6. Microservices FTW! Customer-Facing Web Interface /get order history /save /generate recommend /get customers /login orders /get /submit banking_ interface /process_ payment /landing frontend /order products /get
7. A Not-so-Unrealistic Scenario Initial Version Updated Version frontend /landing recommend /generate products /get frontend /landing recommend /generate products /get orderhistory /get
8. A Not-so-Unrealistic Scenario Initial Version Updated Version frontend /landing recommend /generate products /get frontend /landing recommend /generate products /get orderhistory /get
9. A Not-so-Unrealistic Scenario Initial Version Updated Version frontend /landing recommend /generate products /get frontend /landing recommend /generate products /get orderhistory /get
10. A Not-so-Unrealistic Scenario Initial Version Updated Version frontend /landing recommend /generate products /get frontend /landing recommend /generate products /get orderhistory /get
11. Modelling Microservice Costs CostHat
12. Modelling Microservice Costs Basic intuitions: CostHat
13. Modelling Microservice Costs Basic intuitions: • Total costs of deployment = sum of all individual service costs CostHat
14. Modelling Microservice Costs Basic intuitions: • Total costs of deployment = sum of all individual service costs • For each service: CostHat
15. Modelling Microservice Costs Basic intuitions: • Total costs of deployment = sum of all individual service costs • For each service: • Assuming a defined fixed SLA … CostHat
16. Modelling Microservice Costs Basic intuitions: • Total costs of deployment = sum of all individual service costs • For each service: • Assuming a defined fixed SLA … • … costs are a function of load (plus other factors) CostHat
17. Modelling Microservice Costs Basic intuitions: • Total costs of deployment = sum of all individual service costs • For each service: • Assuming a defined fixed SLA … • … costs are a function of load (plus other factors) • “Load” can be internal or external CostHat
18. Modelling Microservice Costs Basic intuitions: • Total costs of deployment = sum of all individual service costs • For each service: • Assuming a defined fixed SLA … • … costs are a function of load (plus other factors) • “Load” can be internal or external • Load changes propagate through the system CostHat
19. “Costs are a function of load” Data Store Backing Instance Backing Instance Backing Instance ... /ep1 /ep2 ... Data Store /ep1 /ep2 ... /ep1 Lambda Function Instance-Backed Microservice Lambda-Backed Microservice (a) (b) API API /ep2 Lambda Function ...
20. “Costs are a function of load” Data Store Backing Instance Backing Instance Backing Instance ... /ep1 /ep2 ... Data Store /ep1 /ep2 ... /ep1 Lambda Function Instance-Backed Microservice Lambda-Backed Microservice (a) (b) API API /ep2 Lambda Function ...
21. “Costs are a function of load” Data Store Backing Instance Backing Instance Backing Instance ... /ep1 /ep2 ... Data Store /ep1 /ep2 ... /ep1 Lambda Function Instance-Backed Microservice Lambda-Backed Microservice (a) (b) API API /ep2 Lambda Function ... EC2, Heroku, Elastic Beanstalk, …
22. “Costs are a function of load” Data Store Backing Instance Backing Instance Backing Instance ... /ep1 /ep2 ... Data Store /ep1 /ep2 ... /ep1 Lambda Function Instance-Backed Microservice Lambda-Backed Microservice (a) (b) API API /ep2 Lambda Function ... EC2, Heroku, Elastic Beanstalk, … AWS Lambda, External Services, …
23. “Costs are a function of load” Data Store Backing Instance Backing Instance Backing Instance ... /ep1 /ep2 ... Data Store /ep1 /ep2 ... /ep1 Lambda Function Instance-Backed Microservice Lambda-Backed Microservice (a) (b) API API /ep2 Lambda Function ... 0 1 2 3 4 5 6 7 8 0 2 4 6 8 10 Requests [1000 per minute] Service Costs [100 $] 10 0 2 3 4 5 6 7 8 9 0 2 4 6 8 Requests [1000 per minute] Service Costs [100 $]
24. Service Call Graphs frontend /landing 1 customers /login orders /get 0.2 0.2 recommend /generate 1 orderhistory /get 1 products /get 3.1 10 10 customers /get
25. Service Call Graphs frontend /landing 1 customers /login orders /get 0.2 0.2 recommend /generate 1 orderhistory /get 1 products /get 3.1 10 10 customers /get External Call
26. Service Call Graphs frontend /landing 1 customers /login orders /get 0.2 0.2 recommend /generate 1 orderhistory /get 1 products /get 3.1 10 10 customers /get
27. Service Call Graphs frontend /landing 1 customers /login orders /get 0.2 0.2 recommend /generate 1 orderhistory /get 1 products /get 3.1 10 10 customers /get
28. Service Call Graphs frontend /landing 1 customers /login orders /get 0.2 0.2 recommend /generate 1 orderhistory /get 1 products /get 3.1 10 10 customers /get
29. Calculating Total Costs
30. Calculating Total Costs Calculate requests per endpoint
31. Calculating Total Costs Calculate total costs based on this load
32. Using CostHat Potential usages: • Developer tooling (IDE plugins) • What-If analysis
33. Using CostHat Potential usages: • Developer tooling (IDE plugins) • What-If analysis
34. Performance of Recommendations
35. Performance of Recommendations (core aspect of usefulness for developer tooling)
36. Performance of Recommendations (core aspect of usefulness for developer tooling)
37. Performance of Recommendations (core aspect of usefulness for developer tooling)
38. Case Study Model Building • 7 services, 12 endpoints • Combination of instance- and lambda-backed services • Current AWS prices in summer 2016 • Other model parameters: • Assumptions (based on previous benchmarking studies) Customer-Facing Web Interface /get order history /save /generate recommend /get customers /login orders /get /submit banking_ interface /process_ payment /landing frontend /order products /get (focus on what-if analysis)
39. Four Scenarios
40. Four Scenarios Scenario 1 - Improved Conv. Rate • More requests to login
41. Four Scenarios Scenario 1 - Improved Conv. Rate • More requests to login
42. Four Scenarios Scenario 1 - Improved Conv. Rate • More requests to login Scenario 2 - More external load • The recommendation service gets used in other apps as well
43. Four Scenarios Scenario 1 - Improved Conv. Rate • More requests to login Scenario 2 - More external load • The recommendation service gets used in other apps as well
44. Four Scenarios Scenario 1 - Improved Conv. Rate • More requests to login Scenario 2 - More external load • The recommendation service gets used in other apps as well Scenario 3 - Instance Type Migr. • Changing the used backend instance for one service
45. Four Scenarios Scenario 1 - Improved Conv. Rate • More requests to login Scenario 2 - More external load • The recommendation service gets used in other apps as well Scenario 3 - Instance Type Migr. • Changing the used backend instance for one service
46. Four Scenarios Scenario 1 - Improved Conv. Rate • More requests to login Scenario 2 - More external load • The recommendation service gets used in other apps as well Scenario 3 - Instance Type Migr. • Changing the used backend instance for one service Scenario 4 - Code Change • Impact of new dependency to customer/get in the recommendation service
47. Four Scenarios Scenario 1 - Improved Conv. Rate • More requests to login Scenario 2 - More external load • The recommendation service gets used in other apps as well Scenario 3 - Instance Type Migr. • Changing the used backend instance for one service Scenario 4 - Code Change • Impact of new dependency to customer/get in the recommendation service
48. Related Research Jürgen Cito, Philipp Leitner, Harald C. Gall, Aryan Dadashi, Anne Keller, and Andreas Roth. 2015. Runtime metric meets developer: building better cloud applications using feedback. In 2015 ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software (Onward! 2015). ACM, New York, NY, USA, 14-27. DOI=http://dx.doi.org/10.1145/2814228.2814232 R. Singh, P. Shenoy, M. Natu, V. Sadaphal, and H. Vin. Predico: a System for What-If Analysis in Complex Data Center Ppplications. In Proceedings of the 12th International Middleware Conference, pages 120–139, 2011. D. Franceschelli, D. Ardagna, M. Ciavotta, E. Di Nitto. 2013. SPACE4CLOUD: a Tool for System Performance and Cost Evaluation of Cloud Systems. In Proceedings of the 2013 International Workshop on Multi-cloud Applications and Federated Clouds (New York, NY, USA, 2013), MultiCloud ’13, ACM, pp. 27–34.
49. Related Research Jürgen Cito, Philipp Leitner, Harald C. Gall, Aryan Dadashi, Anne Keller, and Andreas Roth. 2015. Runtime metric meets developer: building better cloud applications using feedback. In 2015 ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software (Onward! 2015). ACM, New York, NY, USA, 14-27. DOI=http://dx.doi.org/10.1145/2814228.2814232 Focus on performance rather than costs R. Singh, P. Shenoy, M. Natu, V. Sadaphal, and H. Vin. Predico: a System for What-If Analysis in Complex Data Center Ppplications. In Proceedings of the 12th International Middleware Conference, pages 120–139, 2011. D. Franceschelli, D. Ardagna, M. Ciavotta, E. Di Nitto. 2013. SPACE4CLOUD: a Tool for System Performance and Cost Evaluation of Cloud Systems. In Proceedings of the 2013 International Workshop on Multi-cloud Applications and Federated Clouds (New York, NY, USA, 2013), MultiCloud ’13, ACM, pp. 27–34.
50. Related Research Jürgen Cito, Philipp Leitner, Harald C. Gall, Aryan Dadashi, Anne Keller, and Andreas Roth. 2015. Runtime metric meets developer: building better cloud applications using feedback. In 2015 ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software (Onward! 2015). ACM, New York, NY, USA, 14-27. DOI=http://dx.doi.org/10.1145/2814228.2814232 Focus on performance rather than costs R. Singh, P. Shenoy, M. Natu, V. Sadaphal, and H. Vin. Predico: a System for What-If Analysis in Complex Data Center Ppplications. In Proceedings of the 12th International Middleware Conference, pages 120–139, 2011. D. Franceschelli, D. Ardagna, M. Ciavotta, E. Di Nitto. 2013. SPACE4CLOUD: a Tool for System Performance and Cost Evaluation of Cloud Systems. In Proceedings of the 2013 International Workshop on Multi-cloud Applications and Federated Clouds (New York, NY, USA, 2013), MultiCloud ’13, ACM, pp. 27–34. Palladio
51. Related Research Jürgen Cito, Philipp Leitner, Harald C. Gall, Aryan Dadashi, Anne Keller, and Andreas Roth. 2015. Runtime metric meets developer: building better cloud applications using feedback. In 2015 ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software (Onward! 2015). ACM, New York, NY, USA, 14-27. DOI=http://dx.doi.org/10.1145/2814228.2814232 Focus on performance rather than costs R. Singh, P. Shenoy, M. Natu, V. Sadaphal, and H. Vin. Predico: a System for What-If Analysis in Complex Data Center Ppplications. In Proceedings of the 12th International Middleware Conference, pages 120–139, 2011. What-If Analysis D. Franceschelli, D. Ardagna, M. Ciavotta, E. Di Nitto. 2013. SPACE4CLOUD: a Tool for System Performance and Cost Evaluation of Cloud Systems. In Proceedings of the 2013 International Workshop on Multi-cloud Applications and Federated Clouds (New York, NY, USA, 2013), MultiCloud ’13, ACM, pp. 27–34. Palladio
52. Conclusions • Costs of changes in Microservice apps are hard to predict • CostHat models deployment costs depending on load • Main concepts: • Service call graphs • Cost functions Future work: • Automated monitoring of model parameters • Larger-scale evaluation
53. Conclusions • Costs of changes in Microservice apps are hard to predict • CostHat models deployment costs depending on load • Main concepts: • Service call graphs • Cost functions Future work: • Automated monitoring of model parameters • Larger-scale evaluation