Share
Where do we go from here? – Mastering the changed needs of architectural work
This presentation discusses how the needs and demands regarding architectural work massively changed over the course of the past 10-15 years and how we can respond to it.
It first sketches architectural work in the past based on the typical settings back then. It then briefly covers why the agile movement (as valuable as it was and still is from many other perspectives) impeded properly adapting architectural work to a changing environment.
After that, the presentation outlines a number of changes in technology, system properties and overarching trends that massively influence architecture and architectural work, resulting in a new image what we need to take into account when doing architectural work (spoiler alert: the new image is very different from the original image and a lot more complex).
The presentation finally offers a few recommendations in terms of guiding principles how to address architecture and architectural work today to live up to the changed needs and demands.
As always, the voice track is missing. Still, I hope the slides offer a few useful insights and ideas to ponder.
Side note: This presentation is the successor of one of my prior presentations called "Squaring the circle". While the first part of the presentation only slightly changed, I completely rewrote the recommendations part (of course, there were also a lot of updates regarding the overall voice track but you won't see that on the slides).
Uwe Friedrichsen
Transcript
64 slides
-
Where do we go from here? Mastering the changed needs
of architectural work Uwe Friedrichsen – codecentric AG – 2006-2024 -
Typical setting (1/2) • Many internally used systems • Relatively
few external systems • Mostly monolithic systems • Rich frontend apps or server-side rendered web pages • On-premises data centers -
Typical setting (2/2) • Perimeter-based security • Pre-definable load patterns
• “IT is a cost center” thinking • Effects of distribution handled by infrastructure -
Architectural work (1/3) • Design solution that satisfies project needs
• Project cost efficiency as dominant design driver • Focus on system structure • Layered architecture was predominant -
Architectural work (2/3) • Availability, security, scaling, … left to
Ops • Standard, often mandatory solutions offered by Ops • Often respective NFRs determined by solutions offered • Performance as 2nd class citizen • Usability and architecture completely unrelated • Integration as 2nd class citizen • Either solved ad hoc or via centralized integration solution -
Architectural work (3/3) • Speed of change usually constrained by
project lead times • Typically measured in months or years • Core area of tension • Solution maintainability and changeability • Project cost efficiency -
Focus on structure Core area of tension Changeability Maintainability Flexibility
Project efficiency Opposing goals Amplifying goals Usability Detached topic Performance Minor design consideration Availability Scalability Reliability Confidentiality Integrity Security “Ops problem” Dominant solution driver Portability Interoperability Minor design considerations -
Architectural work typically meant „Create building block design and decide
technology” * * within the predefined constraints -
The age of confusion • The IT world went “Agile”
• Developers were “The new kingmakers” * • Countermovement to the architecture-centric movement of the 1990s * https://www.oreilly.com/library/view/the-new-kingmakers/9781449368036/ -
“We do not need architects. The dev team is the
architect.” “We do not need architectural work. Architecture emerges.” -
Focus on structure Core area of tension Changeability Maintainability Flexibility
Project efficiency Opposing goals Amplifying goals Usability Detached topic Performance Minor design consideration Availability Scalability Reliability Confidentiality Integrity Security “Ops problem” Dominant solution driver Portability Interoperability Minor design considerations We were still here -
Novel drivers (1/3) • Cloud revolution * • Completely new
solution options • Pay per use • Elasticity • Moving data around has a price tag • Availability and reliability on non-HA hardware * https://www.ufried.com/blog/public_cloud_revolution_1/ -
Changeability Maintainability Usability Performance Availability Scalability Reliability Confidentiality Integrity Security
Portability Interoperability Flexibility Cloud revolution Project efficiency Opposing goals Amplifying goals Suitability New solution options Pay per use Interoperability Data flow Economic efficiency Data flow Elasticity Elasticity Scalability Resilience Availability Reliability Scale out on non-HA hardware Project efficiency -
Novel drivers (1/3) • Cloud revolution • Big data &
ML/AI • Huge amounts of data to store, move and process • Data flow becoming 1st class citizen of system design • Polyglot persistence • MLaaS/AIaaS -
Changeability Maintainability Usability Performance Confidentiality Integrity Security Portability Flexibility Big
data & ML/AI Project efficiency Suitability Data flow Resilience Project efficiency Economic efficiency Move, store & process data Mlaas/AIaas Economic efficiency Availability Scalability Reliability Interoperability Performance Process & move data Polyglot persistence Suitability Processing options Interoperability Move data Scalability Move, store & process data Store data Reliability Availability -
Novel drivers (1/3) • Cloud revolution • Big data &
ML/AI • Mobile devices • Completely new interaction patterns • Various novel device classes • Less reliable and often limited connectivity • Unpredictable load patterns -
Changeability Maintainability Usability Performance Confidentiality Integrity Security Portability Flexibility Mobile
devices Project efficiency Suitability Resilience Project efficiency Economic efficiency Availability Scalability Reliability Interoperability Performance Limited battery Scalability Unpredictable usage patterns Suitability New interaction patterns Usability Increased user expectations Adaptability Novel device classes Portability Novel attack vectors Security Confidentiality Integrity Availability Dependability Less reliable connectivity Versatility Manifold subclasses Interoperability Less reliable connectivity Changeability Increased user expectations -
Novel drivers (1/3) • Cloud revolution • Big data &
ML/AI • Mobile devices • IoT & Edge • Most of the effects of mobile devices “to the square” • Solutions enter safety-critical domains -
Dependability Versatility Changeability Maintainability Usability Performance Confidentiality Integrity Security Portability
Flexibility IoT & Edge Project efficiency Suitability Resilience Project efficiency Economic efficiency Availability Scalability Reliability Interoperability Performance Limited battery Suitability New interaction patterns Usability Novel device classes Portability Novel attack vectors Security Confidentiality Integrity Availability Dependability Less reliable connectivity Versatility Manifold subclasses Interoperability Less reliable connectivity Adaptability Safety Safety-critical applications -
Novel drivers (2/3) • Distributed systems • Highly connected, distributed
system landscape • Service-based applications, … • Intricate new failure types need to be handled • Effects of distribution penetrate to the application level • Near-realtime response expectations intensify effects -
Dependability Versatility Changeability Maintainability Usability Performance Confidentiality Integrity Security Portability
Flexibility Distributed systems Project efficiency Resilience Project efficiency Economic efficiency Availability Scalability Performance Latency Confidentiality Integrity Adaptability Safety Reliability Remote communication Security Safety Availability Dependability Distributed failure modes Reliability Resilience Interoperability Suitability Communication flow Usability Suitability Communication flow Interoperability Observability Distributed failure modes -
Novel drivers (2/3) • Distributed systems • Near-realtime • Batch
updates turn into online updates • Complex web of continuously communicating systems • Integration becomes first-class citizen • Near-realtime insights feedback loop • Intensifies effects of system distribution -
Dependability Versatility Changeability Maintainability Usability Performance Confidentiality Integrity Security Portability
Flexibility Near-realtime Project efficiency Suitability Resilience Project efficiency Economic efficiency Availability Scalability Interoperability Performance Latency Confidentiality Integrity Adaptability Safety Reliability Availability Dependability Distribution failure modes Reliability Resilience Interoperability Integration as 1st class citizen Observability -
Novel drivers (2/3) • Distributed systems • Near-realtime • 24x7
• Maintenance windows are gone • Overnight batch processing windows are gone • Upgrade/repair systems in-flight -
Dependability Versatility Changeability Maintainability Usability Performance Confidentiality Integrity Security Portability
Flexibility 24x7 Project efficiency Suitability Resilience Project efficiency Economic efficiency Availability Scalability Interoperability Confidentiality Integrity Adaptability Safety Reliability Availability Dependability In-flight system upgrade/repair Reliability Observability In-flight system upgrade/repair Observability Batch windows gone Performance -
Novel drivers (2/3) • Distributed systems • Near-realtime • 24x7
• Perimeter-less security • Trusted zones are gone: “The Internet is everywhere” • Remote work, BYOD, mobile devices, etc. intensify challenges • “Shift left security” mandatory -
Observability Integrity Integrity Dependability Versatility Changeability Maintainability Usability Performance Confidentiality
Security Portability Flexibility Perimeter-less security Project efficiency Suitability Resilience Project efficiency Economic efficiency Availability Scalability Interoperability Confidentiality Adaptability Safety Reliability Observability Observability Trusted zones gone Integrity Trusted zones gone Security Confidentiality -
Novel drivers (3/3) • Digital transformation • “IT and business
are the same side of the same coin; the other side are the market and your customers” • IT being integral part of all business offerings • IT being integral part of most customer interactions • IT enabling new, previously unthinkable business models • IT becoming indispensable part of business and private lives -
Integrity Integrity Dependability Versatility Changeability Maintainability Usability Performance Confidentiality Security
Portability Flexibility Digital transformation Suitability Resilience Project efficiency Economic efficiency Availability Scalability Interoperability Confidentiality Adaptability Safety Reliability Observability New business models Adaptability System lifecycle Products, not projects Economic efficiency Availability Dependability Integral part of business offerings Reliability Resilience Scalability Usability Suitability Integral part of customer interactions Integrity Security Confidentiality Safety Indispensable part of our lives Interoperability API-driven revenue streams Changeability New business models Flexibility Maintainability Human needs Socio-technical systems Affecting humans -
Novel drivers (3/3) • Digital transformation • Post-industrial markets •
Supply exceeds demand by far à customer-driven markets • Fast adoption to changing demands becomes vital • Lead and cycle times become essential • Continuous feedback at all levels needed • More complex multidisciplinary collaboration needed -
Human needs Socio-technical systems Integrity Integrity Dependability Versatility Changeability Maintainability
Usability Performance Confidentiality Security Portability Flexibility Post-industrial markets Suitability Resilience Economic efficiency Availability Scalability Interoperability Confidentiality Adaptability Safety Reliability Observability Fast adoption Adaptability Human needs Socio-technical systems Multidisciplinary collaboration Changeability Fast adoption Flexibility Maintainability Observability Observability Continuous feedback Usability Suitability Customer-driven market System lifecycle -
Novel drivers (3/3) • Digital transformation • Post-industrial markets •
Exploding complexity • Probably the 2nd biggest challenge of the upcoming years • IT drowns in complexity and every day it becomes worse • Due to shortsighted thinking • Due to important decisions not made -
Human needs Socio-technical systems Integrity Integrity Dependability Versatility Changeability Maintainability
Usability Performance Confidentiality Security Portability Flexibility Exploding complexity Suitability Resilience Economic efficiency Availability Scalability Interoperability Confidentiality Adaptability Safety Reliability Observability System lifecycle Simplicity Tackling complexity System empathy System viability and collaboration -
Novel drivers (3/3) • Digital transformation • Post-industrial markets •
Exploding complexity • Ecological sustainability • Probably the biggest challenge of the upcoming years • Making sustainability 1st class citizen of system design • Counterforce to many other forces -
Human needs Socio-technical systems Integrity Integrity Dependability Versatility Changeability Maintainability
Usability Performance Confidentiality Security Portability Flexibility Ecological sustainability Suitability Resilience Economic efficiency Availability Scalability Interoperability Confidentiality Adaptability Safety Reliability Observability System lifecycle Fostering sustainability Sustainability Simplicity System empathy -
Human needs Socio-technical systems Integrity Integrity Dependability Versatility Changeability Maintainability
Usability Performance Confidentiality Security Portability Flexibility Opposing goals Amplifying goals Suitability Resilience Economic efficiency Availability Scalability Interoperability Confidentiality Adaptability Safety Reliability Observability System lifecycle Simplicity System empathy Sustainability Project efficiency … and many more -
All quality goals need to be balanced according to the
given needs * * needs of different stakeholder groups, short-term and long-term needs, amplifying vs. opposing goals, … -
All influencing forces and the derived quality goals vastly influence
the resulting solution architecture -
Human needs Socio-technical systems Integrity Integrity Dependability Versatility Changeability Maintainability
Usability Performance Confidentiality Security Portability Flexibility Opposing goals Amplifying goals Suitability Resilience Economic efficiency Availability Scalability Interoperability Confidentiality Adaptability Safety Reliability Observability System lifecycle Simplicity System empathy Sustainability Project efficiency Focus on structure not sufficient anymore Data flow Communication flow Limited view on projects not sufficient anymore Economic perspective over lifecycle as 1st class citizen Prevalent complexity of IT system landscapes requires new design approaches Exploding number of device classes Integration as 1st class citizen Thinking in feedback loops Holistic view required. Focus on software alone not sufficient anymore Novel drivers of future system design. Counterforces to many other design forces Enterprise software affects safety-critical systems more and more Security as 1st class citizen in design (“Shift left security”) Huge new challenges due to highly interconnected distributed system landscapes (“microservices”) Unpredictable load patterns Massively increased user expectations influence system design Technology evolution provides new options and changes what a feasible solution is -
Lots of new forces. Lots of new tasks and challenges.
Partially completely novel forces and resulting tasks. -
Architecture is complex • The dev team as architect is
dead • The omniscient architect is dead, too • Architectural work has become a team sport … • … but needs a conductor -
Architecture unfolds • Emergent architecture is dead • BDUF is
dead, too • Start with JDUF and then evolve along the way … • … but make deliberate architectural decisions -
“A complex system designed from scratch never works and cannot
be made to work. You have to start over, beginning with a working simple system.” -- John Gall Source: John Gall, "Systemantics", 1975 -
Architectural work is systemic • Decisions have complex impact patterns
• Think global, act local • Foster feedback at all levels • Minimize overall complexity growth -
Do you really think hundreds of local optimizations completely ignoring
their context will result in a global optimum? Or do you rather think quite the opposite is going to happen? -
Architecture is socio-technical • Mind the impacted socio-technical systems •
Effectiveness of software design and development • Ease of operations • Effectiveness of problem detection and analysis • Efficiency of problem fixing or mitigation • Support of usage patterns, processes and organization • ... -
Architectural work is challenging • Level up architectural work •
Level up requirements engineering and communication • Master the foundations of good design • Help people understand software • Embrace distributed systems • Understand dependability, sustainability and simplicity -
“There is nothing a mere scientist can say that will
stand against the flood of a hundred million dollars. But there is one quality that cannot be purchased in this way - and that is reliability. The price of reliability is the pursuit of the utmost simplicity. It is a price which the very rich find most hard to pay.” -- Sir C. A. R. Hoare Source: “The emperor's old clothes“, Communications of the ACM, Vol. 24, No. 2, February 1981, pp. 75-83 -
Wrap-up • Architectural challenges have changed a lot • Complexity
went up significantly • New counterforces have emerged • Decision making has become a lot more demanding • Updated perception of architectural work required • Forget the traditional perception of architects and their work • New collaboration modes needed • Different skills have become vital