Lessons Learned from Software Modernization projects

Presentation of the AWS "Cloud for Breakfast" event series. https://www.innoq.com/de/news/2017/04/aws-microservices-breakfast-talks/

1. When theory meets real life Jerry Preissler Alexander Heusingfeld Lessons Learned in Software Modernization projects
2. Alexander Heusingfeld alexander.heusingfeld@innoq.com Senior Consultant @ innoQ @aheusingfeld Gerald Preissler Senior Consultant @ innoQ gerald.preissler@innoq.com @jerrypreissler
3. About innoQ > Software Architecture Consulting & Software Development > Around 120 Consultants/Engineers in Germany and Switzerland > Offices in Düsseldorf, Berlin, Frankfurt, Munich and Zürich > Customers: 90% of Customers in “Fortune 1000” Segment. Finance, Industry, Telecommunications, Public Sector
4. Some of our AWS projects... (partially anonymized due to NDAs)
5. Heraus Kulzer > Designed and developed a cloud based platform in AWS for order management of dental products of Kulzer, Mitsui Chemicals Group. (03/2015 - 12/2015) > innoQ team (4 engineers) developed complete software and defined cloud architecture in AWS > innoQ achieved complete automation of infrastructure provisioning (VPC, network, servers, services) and deployment with CloudFormation. > Start planned for North America, runs in AWS US zones. Currently in Validation phase (not yet in production). Other regions can follow with minimal efforts thanks to Cloud automation. > Technologies used: EC2, Beanstalk, S3, RDS (Postgres), CloudFormation, Spring Boot
6. Global Availability > eCommerce platform to deliver content to consumer devices in global markets using AWS technology (2015/2016) > innoQ team (5 engineers) defined and implemented Microservices architecture, cloud infrastructure and processes for provisioning and monitoring. > Implementation of an “Edge Layer” (similar to technology of Netflix) to enable regional caching of content using AWS services. > innoQ’s Cloud Architecture reduced latency to API endpoints and traffic. > Technologies: VPC, EC2, ECS, Docker, S3, CloudFormation, Beanstalk, AutoScaling, Route53 > Regions: Europe, North America, Pacific, Australia
7. Insurance Portal > Reimplementation and modernization of German customer’s portal of a large insurance company. Migrated platform from internal data center to AWS cloud (04/2015 - 02/2016). > innoQ team (7 engineers) was responsible for software and cloud architecture, developed web and integration software and designed scalable cloud infrastructure. > Use of Hardware Security Module (Cloud HSM) for encryption of traffic and data to comply with strict German data protection act, especially regarding health data. > Currently only German market, runs in AWS Frankfurt. > Technologies used: VPN/VPC-Peering, EC2, S3, CloudFormation, Elasticache, CloudHSM
8. Architecture Consulting...
9. “We want to have a microservice architecture!”
10. “We want to have a microservice architecture!” — Every Customer, since 2015
11. A monolith
12. When reviewing monolithic applications … © innoQ/Roman Stranghöner
13. …and taking a look into the black box… © innoQ/Roman Stranghöner
14. …you’ll likely find it consists of multiple business domains. © innoQ/Roman Stranghöner
15. Architectural Decisions
16. Architectural Decisions > Domain Architecture
17. Architectural Decisions > Macro Architecture > Domain Architecture
18. Architectural Decisions > Macro Architecture > Micro Architecture > Domain Architecture
19. Self-Contained System (SCS)
20. If you cut a monolithic system along its very domains …
21. … and wrap every domain in a separate, replaceable web application …
22. … then that application can be referred to as a self- contained system (SCS).
23. On its outside, an SCS is a decentralized unit that is communicating with other systems via RESTful HTTP or lightweight messaging.
24. Therefore self-contained systems can be individually developed for different platforms.
25. An SCS contains its own 
 user interface, specific 
 business logic and 
 separate data storage
26. Besides a web interface a self- contained system can provide an optional API.
27. The business logic part only solves problems that arise in its core domain. This logic is only shared with other systems over a well defined interface.
28. The business logic can consist of microservices to solve domain specific problems.
29. Every SCS brings its own data storage and with it redundant data depending on the context and domain.
30. These redundancies are tolerable as long as the sovereignty of data by its owning system is not undermined.
31. This enables polyglot persistence, which means a database can be chosen to solve a domain specific problem rather than to fulfill a technical urge. Neo4J CouchDB Oracle
32. Inside of a self-contained system a bunch of technical decisions can be made independently from other systems, such as programming language, frameworks, tooling or workflow.
33. The manageable domain specific scope enables the development, operation and maintenance of an SCS by a single team. Team 1 Team 2 Team 3
34. Self-contained Systems
 should be integrated over their web interfaces to minimize coupling to other systems. https://www.innoq.com/en/talks/2016/02/microxchg-2016-microservices-frontend-ui/
35. http://scs-architecture.org/ more information on self-contained systems (SCS) can be found at
36. Improvement Approaches http://aim42.org for Software Modernization
37. Close for change Change by extraction
38. Close for change Enable integrateability
 (auth/auth, navigation) Change by extraction
39. Close for change Enable integrateability
 (auth/auth, navigation) Create new system
 for new features Change by extraction
40. Close for change Enable integrateability
 (auth/auth, navigation) Create new system
 for new features Integrate and/or
 replace part Change by extraction
41. Close for change Enable integrateability
 (auth/auth, navigation) Create new system
 for new features Integrate and/or
 replace part Change by extraction
42. > Define external, “ideal” interface > Adapt to existing codebase > Transform system to match interfaces Outside-in interfaces
43. > Define external, “ideal” interface > Adapt to existing codebase > Transform system to match interfaces Outside-in interfaces
44. > Define external, “ideal” interface > Adapt to existing codebase > Transform system to match interfaces Outside-in interfaces
45. Steps for modularisation
46. Steps for modularisation • identify domains User Management Payment Product Management
47. Steps for modularisation • identify domains • group teams by domain User Management Payment Product Management
48. Steps for modularisation • identify domains • group teams by domain • agree on macro architecture User Management Payment Product Management
49. Steps for modularisation • identify domains • group teams by domain • agree on macro architecture • first delivery pipeline, then end-to-end features User Management Payment Product Management
50. Steps for modularisation • identify domains • group teams by domain • agree on macro architecture • first delivery pipeline, then end-to-end features • team decides migration approach case-by-case User Management Payment Product Management
51. Enable Experiments
52. innoQ Self-Service
53. Steps to publish an app to the web: innoQ Self-Service
54. Steps to publish an app to the web: 1. Implement app (adhere to template)
 2. Package app in DockerContainer
 3. Publish to ECS (Auth against Company-LDAP) innoQ Self-Service
55. Steps to publish an app to the web: 1. Implement app (adhere to template)
 2. Package app in DockerContainer
 3. Publish to ECS (Auth against Company-LDAP) Too complicated? Just use our self-build CLI to take care of steps 2 and 3. innoQ Self-Service
56. SCS in Real-Life: Insurance Portal
57. Sample Customer SCSs The basic composition of a web application using self-contained systems 
 is pretty straightforward
58. As always, when design meets real life, things get a little more complicated...
59. Ok, more than a little
60. Using AWS Services can actually safe some complexity on your side
61. Don't build your own PaaS DIY PaaS means you have to > self-host, self-configure, self-deploy, self-update and self-maintain needed infrastructure components > maintain knowhow of all the nuts and bolts Do the calculation: Is this your businesses' core?
62. Real-world requirements will introduce some complexity on their own
64. Testing
69. Operations
74. Latency
75. Latency • Mexiko: ~400ms
76. Latency • Mexiko: ~400ms • Australien: ~450ms
77. Latency • Mexiko: ~400ms • Australien: ~450ms + At peak times ~8% packet loss from Sydney to Frankfurt!
78. Latency • Mexiko: ~400ms • Australien: ~450ms + At peak times ~8% packet loss from Sydney to Frankfurt! Implement Stability Patterns!!!
79. Latency • Mexiko: ~400ms • Australien: ~450ms + At peak times ~8% packet loss from Sydney to Frankfurt! Note: Packet loss is much better today! Implement Stability Patterns!!!
81. Don't monitor hosts, monitor business services
82. conclusion
83. Summary
84. Summary > SCSs are a reasonable approach to Microservices
85. Summary > SCSs are a reasonable approach to Microservices > aim42 provides structure for software modernization
86. Summary > SCSs are a reasonable approach to Microservices > aim42 provides structure for software modernization > Not everyone who wants microservices is immediately capable to establish them
87. Summary > SCSs are a reasonable approach to Microservices > aim42 provides structure for software modernization > Not everyone who wants microservices is immediately capable to establish them > Don’t overwhelm people, change one thing at a time
88. Summary > SCSs are a reasonable approach to Microservices > aim42 provides structure for software modernization > Not everyone who wants microservices is immediately capable to establish them > Don’t overwhelm people, change one thing at a time > AWS can enable "small experiments" - we don't have excuses anymore
89. Thank you! Questions? Comments? Alexander Heusingfeld, @aheusingfeld alexander.heusingfeld@innoq.com Gerald Preissler, @jerrypreissler gerald.preissler@innoq.com innoQ Deutschland GmbH Krischerstr. 100 40789 Monheim am Rhein Germany Phone: +49 2173 3366-0 innoQ Schweiz GmbH Gewerbestr. 11 CH-6330 Cham Switzerland Phone: +41 41 743 0116 www.innoq.com Ohlauer Straße 43 10999 Berlin Germany Ludwigstraße 180 E D-63067 Offenbach Germany Kreuzstr. 16 D-80331 München Germany https://www.innoq.com/en/talks/
90. BACKUP SLIDES
91. innoQ Mission Statement
 
 innoQ helps its clients to use modern, leading-edge methods and technologies to implement pragmatic software solutions that match actual requirements.
92. Customers (excerpt) Allianz Alcatel - Lucent AXA Bank-Verlag Barmer, GEK, KKH, DAK Bertelsmann Norisbank CredaRate Credit Suisse Deutsche Bank Deutsche Post Deutsche Telekom Digital River DVB Bank Federal Reserve Bank Gothaer Systems Groupon Heraeus Kulzer HolidayCheck HP IBM LVM Microsoft Nokia Siemens Novartis O2 Otto RWE Systems T-Labs T-Systems Typesafe Software AG UBS XING Vodafone Zurich Financial Services
93. Key Focus Areas > Pragmatic Software Architecture > Intelligent creation and use of model information > Effective software development > Efficient operations concepts
94. Focus in engineering Focus of employees in Germany and Switzerland 5 backoffice employees 4 Managers C-Level 10 Consultants with special expertise in project management, requirements analysis and agile development 50 Consultants with main focus in software engineering and development 30 Consultants with main focus in system architecture, platform engineering and DevOps 6 Principals (sales & aqusition)
95. General services > Strategic technology consulting > Project management & requirements analysis > Targeted, focused support by experts > Contracting in customer projects
 (conceptualization, implementation) > Creating of PoCs and prototypes > Reviews, analyses, risk management, continuous quality assurance > Consulting, training, coaching > Delivery of complete solutions (T&M and fix-price)