Upgrade to PRO for Only $50/Year—Limited-Time Offer! 🔥

The reusability fallacy

The reusability fallacy

This slide deck discusses reusability, a topic that is sort of a "holy grail" in software architecture. Unfortunately, reusability often is discussed in the wrong places and for the wrong reasons, especially if it is used to sell a new architectural paradigm ("via reuse, the initial investment will amortize very soon and it will pay for itself").

I start with the origins of the topic in the early 1970s (and the different conditions how software was developed back then). Then I discuss and debunk the most widespread fallacies that are typically used to press for reusability.

With that knowledge I present a few advices (not dogmas!) how to deal better with reusability - when not to use it, what to do instead of those places, and: When to use it!

Reusability has a huge value in software development and we need to make sure to harvest it. But pressing for it in the wrong places for the wrong reasons only causes problems and harm. We need to avoid that and the intention of the talk is to have better arguments if you encounter such a situation.

This is the unabridged slide deck containing all slides of the talk. For conferences, I usually leave out several slides to meet the session length constraints.

I also accompanied this slide deck with a blog series that can be found at https://ufried.com/blog/reusability_fallacy_1/. The series consists of 4 posts, covers the contents of this talk and provides a few more ideas and pointers.

Edit: Uploaded an updated version of the slide deck on Jun-30-2023

Uwe Friedrichsen

June 30, 2023
Tweet

More Decks by Uwe Friedrichsen

Other Decks in Technology

Transcript

  1. The promise (the dream) • Create software module once •

    Reuse as often as you like at no reproduction cost • “Invest once, harvest forever” • Big efficiency lever • Huge cost saver
  2. “Considerations and techniques are proposed that reduce the complexity of

    programs by dividing them into functional modules. This can make it possible to create complex systems from simple, independent, reusable modules. […] And, as the module library grows, increasingly sophisticated programs can be implemented using less and less new code.” [Ste 1974]
  3. The origins • Reusability discussion started ~1970 • Most companies

    and projects started at scratch • Barely more than 3GL compiler and linker available • No big standard libraries bundled with language • No big ecosystems available around language • Even most basic functionalities needed to be developed • Reusability could increase productivity significantly
  4. Reusability today (the fallacy) • Became an end in itself

    • “Reusability is needed. Period!” • Demanded by managers to improve cost efficiency • “I want reusability to improve the IT cost ratio” • Used by architects to sell new architectural paradigms • “Required investment will amortize soon due to reuse” à Never worked this way. Never will work this way.
  5. “Let us apply the reusability approaches from house building or

    car manufacturing” -- The unknown “industrialization of IT” advocate
  6. Standardization • Standardization helps building at reduced price • Same

    part can be reused in many products • Saves production costs over time • Design efforts only once • Production process design and set up only once • Required tooling created and installed only once • Cost savings are realized in production • Spreads design and set up costs over parts produced
  7. Our assembly lines are so efficient for more than 50

    years that most people forgot they even exist We call them “compiler” * * or CI/CD pipelines in the settings of today
  8. Software development is about designing the product The design is

    only complete after the last line of code is written * * Building the product from the design (source code) is merely pushing a button
  9. Our production process is already unbelievably cheap * and we

    can produce multiple copies from a design at zero cost We call it the “copy command” * compared to the production process of any physical goods
  10. New (industrially minted) idea ”The more reusable parts we use

    in our design process (a.k.a. software development), the more time, costs and effort we can save”
  11. “Software entities are more complex for their size than perhaps

    any other human construct, because no two parts are alike (at least above the statement level). If they are, we make the two similar parts into one, a subroutine, open or closed. In this respect software systems differ profoundly from computers, buildings, or automobiles, where repeated elements abound. [...] Likewise, a scaling-up of a software entity is not merely a repetition of the same elements in larger size; it is necessarily an increase in the number of different elements.” [Bro 1986]
  12. The tar pit by Frederick P. Brooks, Jr. (taken from

    the “The mythical man-month”) [Bro 1995]
  13. “One occasionally reads newspaper accounts of how two programmers in

    a remodeled garage have built an important program that surpasses the best efforts of large teams. And every programmer is prepared to believe such tales, for he knows that he could build any program much faster than the 1000 statements/year reported for industrial teams. Why then have not all industrial programming teams been replaced by dedicated garage duos? One must look at what is being produced.” [Bro 1995]
  14. 1x Program The original module, suitable for the context it

    was created for 3x Programming Product A generalized module, suitable for multiple contexts A module, ready to be used in an ecosystem of interacting modules A (re-)usable module, that provides a general solution for a problem 3x Programming System 9x Programming Systems Product
  15. 1x Program 9x Programming Systems Product Completeness of functionality •

    Handling of edge cases • Hardening implementation • Thorough functional testing • Design Documentation Completeness of accessibility • Precise interface definition • Clear behavioral contract • Thorough integration (or alike) testing • API Documentation
  16. Reusability rarely pays! Creating a reusable asset is a lot

    harder and costs a lot more than most people think
  17. Shirking the costs • Asset not implemented for being reused

    • Each “reusing” party needs to • Understand the functionality of the asset • Understand if it covers the given use case • Modify the asset or adopt the using parts • Test the resulting functionality • … • Increases costs and impedes reliability • Often leads to avoiding the asset and reimplementing it
  18. Conclusion Creating a reusable asset costs 9 times as much

    as creating a regular project asset. You only start saving money the 10th time you reuse the asset.
  19. The long version of the reasoning IT person (typically an

    architect) to business sponsor: “We definitely need to invest in implementing <cool new architectural paradigm>. I know this does not create any business value now, but it allows us to create reusable <architectural paradigm building blocks>! This way, we will have to pay a lot less for future projects, they will be a lot faster and we will be a lot more flexible. In short: the investment will pay for itself very soon.” Never worked this way, never will work this way!
  20. Everything we need for decoupling and reusability is already in

    place for more than 60 years We call it “modularization” * * being available ubiquitously in IT, new paradigms are not needed to foster reuse
  21. If you were not able to foster reuse with the

    means you have, what makes you think a new architectural paradigm will?
  22. Reusability suitability Poor Great Fine-grained Stable Simple Universal Technical Coarse-grained

    Volatile Complicated Specific Business Reusability promoters Reusability inhibitors Collection classes Technical framework Embedded HTTP server Currency Account & Transaction Ledger Order handling Available through programming language ecosystems, OSS solutions and alike Subject of “reusability via new paradigm” discussions Could be provided via internal company-specific libraries or alike
  23. The long version again … IT person (typically an architect)

    to business sponsor: “We definitely need to invest in implementing <cool new distributed architectural paradigm>. I know this does not create any business value now, but it allows us to create reusable <architectural paradigm building blocks>! This way, we will have to pay a lot less for future projects, they will be a lot faster and we will be a lot more flexible. In short: the investment will pay for itself very soon.” Now distributed hell breaks loose …
  24. If X then Y What we learned in our IT

    education If X then maybe Y This changes everything! What we need for distributed systems We are good at this (due to how our brains work) Inside process thinking Reasoning about deterministic behavior Designing a complicated system We are poor at that (due to how our brains work) Reasoning about non-deterministic behavior Across process thinking Designing a complex system
  25. Failures in distributed systems ... • Crash failure • Omission

    failure • Timing failure • Response failure • Byzantine failure
  26. ... lead to a variety of effects … • Lost

    messages • Incomplete messages • Duplicate messages • Distorted messages • Delayed messages • Out-of-order message arrival • Partial, out-of-sync local memory • ...
  27. ... turning seemingly simple issues into very hard ones Time

    & Ordering Leslie Lamport "Time, clocks, and the ordering of events in distributed systems" Consensus Leslie Lamport ”The part-time parliament” (Paxos) CAP Eric A. Brewer "Towards robust distributed systems" Faulty processes Leslie Lamport, Robert Shostak, Marshall Pease "The Byzantine generals problem" Consensus Michael J. Fischer, Nancy A. Lynch, Michael S. Paterson "Impossibility of distributed consensus with one faulty process” (FLP) Impossibility Nancy A. Lynch ”A hundred impossibility proofs for distributed computing"
  28. These effects are based on the non-determinism introduced by remote

    communication due to distributed failure modes
  29. To minimize the non-deterministic effects of distributed systems you need

    to minimize coupling between distributed parts
  30. The coupling dilemma • Reusability is tightest form of coupling

    possible • If a reused part fails, reusing parts fail, too • Availability requires avoiding cascading failures • If a part fails, the failure must not spread to other parts • Reusability means cascading failures by definition • Due to the tight coupling between reused and reusing part • Technical measures cannot avoid it • The tight coupling is at the functional level
  31. The value of reuse unfolds inside a process boundary Across

    process boundaries it should be avoided as it maximizes coupling
  32. “But every time I call a remote service, I reuse

    that service” -- The unknown software engineer
  33. Reusable asset Usable asset Asset Problem Solution Composition • Asset

    is part of the solution • Solution cannot work without the asset Collaboration • Asset is not part of the solution • Solution can work without the asset (if designed right) Asset Problem Solution
  34. <uses> Functionality vertical decomposition (layer design, composition) In practice, you

    typically use a combination of both approaches Core functional decomposition approaches horizontal decomposition (pillar design, segregation)
  35. Vertical decomposition • Based on functional decomposition • Typical drivers

    are reuse and avoidance of redundancy • Creates strong coupling (high functional dependence) • Often useful pattern inside a process boundary • Due to deterministic communication behavior • Problematic across process boundaries à Should be avoided in distributed design
  36. Horizontal decomposition • Based on functional segregation • Typical drivers

    are autonomy and independence • Creates low coupling (high functional independence) • Useful pattern across process boundaries • Can also be useful inside a process boundary • Less accidental "ripple" effects due to changes à Should be preferred in distributed design
  37. Watch out! • Vertical decomposition is our default design approach

    • We’ve learned it in our CS education (divide and conquer, ...) • It’s emphasized in our daily work (DRY, reusability, ...) • Even our IDEs support it (“Extract method”) • It's everywhere! It's predominant! • It takes energy not to apply vertical decomposition • Most people never learned horizontal decomposition
  38. While the distinction between “use” and “reuse” seems to be

    irrelevant at first sight, mastering it is essential in practice
  39. Conclusion Aim for usability across process boundaries (collaborating, functionally independent

    parts) * * and learn the associated design approaches (see, e.g. https://speakerdeck.com/ufried/getting-service-design-right)
  40. Reusability as business case • Neither works nor makes sense

    • Modularization is all you need, exists for more than 60 years • Usually leads to bad designs • Many tight couplings • Especially in conjunction with distributed paradigms • Bad for all people affected • Leads to cost savings pressure (the amortization promise) • Does not work as promised • Increased pressure – reinforces problems
  41. Advice #2 Be wary of reusability if the problem you

    try to solve does not exhibit good reusability suitability properties
  42. Advice #3 At a business case level, better argue with

    benefits like "increased business agility" or "better technology obsolescence management” * * especially if you want to introduce a a distributed modularization paradigm like microservices
  43. Corollary to advice #3 Do not try to introduce a

    distributed modularization paradigm like microservices if cost reduction is your sole driver
  44. Corollary to advices #2 and #4 Design advices are heuristics,

    not a dogma They are not a replacement for thinking
  45. Advice #5 Try to use existing reusable assets to increase

    your productivity Do not try to reinvent the wheel
  46. Advice #6 If you solve a problem that has good

    reusability suitability properties and is needed in many places, consider creating a reusable asset * * if you can afford to pay the extra costs
  47. Bonus advice Think about reusability in other areas of software

    engineering, e.g., on the level of ideas
  48. Problem 3 GL development base level Self-development base level New

    reusable assets created Reusability improves productivity and the scope of solvable problems over time Problem 3 GL development base level Self-development base level Recurring solution parts emerge Problem 3 GL development base level Reusable assets Self-developed solution part Self-development base level 3 GL development base level Self-development base level Less self- development needed Problem Self-developed solution part Reusable assets
  49. Wrap-up • Do not use reusability as sales argument for

    new paradigms • All you need for reusability is modularization • Avoid reusability across process boundaries • Learn the difference between “use” and “reuse” • Be willing to pay the price for creating reusable assets • But first consider reusability suitability • Use reusability as productivity driver • Reuse has a huge value used in the right places
  50. References [Bro 1986] Frederick P. Brooks, jr., “No silver bullet”,

    from H.-J. Kugler (ed.), "Information Processing 1986: World Congress Proceedings (IFIP congress series, vol 10)", 1986 [Bro 1995] Frederick P. Brooks, jr., “The tar pit”, from “The mythical man-month”, anniversary edition 1995 [Ste 1974] W. P. Stevens, G. J. Myers, and L. L. Constantine, “Structured design”, IBM Systems Journal, Vol 13, No. 2, 1974