Share
A Type is Worth a Thousand Tests
VIDEO: https://vimeo.com/191467217/3f45cbd32e (Sydney CocoaHeads, October 2016)
One of the biggest changes in the transition from Objective-C to Swift is the new type system. This change is at the heart of Swift’s language design, from optionals over value types to constrained generics, and to truly understand the language, a thorough understanding of its type system is indispensable.
In this talk, I will explain the advantages of Swift’s strong commitment to a sophisticated type system. I will illustrate the impact on language and program design through a series of examples, drawn from my experience of developing one of the first Swift-based applications published in the Mac App Store and illustrated by walking through key design issues in a simple iOS app. I will discuss why type-driven design benefits from functional programming principles, and also, how it allows us to write less tests without a loss of confidence in our code.
Presented at
- YOW! Connected 2016, Melbourne: http://connected.yowconference.com.au
- Sydney CocoaHeads, October 2016: http://www.sydneycocoaheads.com/2016/11/17/a-type-is-worth-a-thousand-tests/
A previous version was presented at Curry On 2016 in Rome: http://www.curry-on.org/2016/ (video of that somewhat shorter previous version of the talk: https://www.youtube.com/watch?v=q1Yi-WM7XqQ)
Manuel Chakravarty PRO
Transcript
103 slides
-
Manuel M T Chakravarty Applicative & UNSW Australia A Type
is Worth a Thousand Tests mchakravarty TacticalGrace justtesting.org haskellformac.com 1 40 minute talk time (+ 5 min discussion) [10min Swift & type systems; 15min value types; 15min protocols] » Let’s talk about Swift… -
Swift Let’s talk about… 2 * Apple’s new language to
replace Objective-C. * Who has used Swift? Who has used it in production? * By itself, Swift is an interesting & clever language design, but * considering its ecosystem, it is actually extraordinary. -
Swift Let’s talk about… Language Design 2 * Apple’s new
language to replace Objective-C. * Who has used Swift? Who has used it in production? * By itself, Swift is an interesting & clever language design, but * considering its ecosystem, it is actually extraordinary. -
Swift Let’s talk about… Language Design Ecosystem 2 * Apple’s
new language to replace Objective-C. * Who has used Swift? Who has used it in production? * By itself, Swift is an interesting & clever language design, but * considering its ecosystem, it is actually extraordinary. -
Swift Let’s talk about… Language Design Ecosystem Why is it
extraordinary ? 2 * Apple’s new language to replace Objective-C. * Who has used Swift? Who has used it in production? * By itself, Swift is an interesting & clever language design, but * considering its ecosystem, it is actually extraordinary. -
Unique Event 3 * Introduction of Swift is unique in
the history of PLs * Guided language migration of a huge developer community -
Java Objective-C C# 4 * Not the first of its
kind: (1) hybrid OO & functional * (2) seamless interoperability with platform’s principal language * The big difference: *platform dominance* ensured by platform vendor (Apple) -
Java Objective-C C# Scala Kotlin 4 * Not the first
of its kind: (1) hybrid OO & functional * (2) seamless interoperability with platform’s principal language * The big difference: *platform dominance* ensured by platform vendor (Apple) -
Java Objective-C C# Scala F# Kotlin 4 * Not the
first of its kind: (1) hybrid OO & functional * (2) seamless interoperability with platform’s principal language * The big difference: *platform dominance* ensured by platform vendor (Apple) -
Swift Java Objective-C C# Scala F# Kotlin 4 * Not
the first of its kind: (1) hybrid OO & functional * (2) seamless interoperability with platform’s principal language * The big difference: *platform dominance* ensured by platform vendor (Apple) -
Why do languages become popular? 5 * Language adoption is
driven by platforms (not by languages themselves) * Platforms can be a mix of ecosystem factors » Hence, Swift’s popularity is guaranteed… -
Why do languages become popular? Platforms 5 * Language adoption
is driven by platforms (not by languages themselves) * Platforms can be a mix of ecosystem factors » Hence, Swift’s popularity is guaranteed… -
Why do languages become popular? Platforms Unix C/C++ .NET C#
JVM Java Web JavaScript Cloud Python/Ruby Cocoa Objective-C 5 * Language adoption is driven by platforms (not by languages themselves) * Platforms can be a mix of ecosystem factors » Hence, Swift’s popularity is guaranteed… -
Why do languages become popular? Platforms Unix C/C++ .NET C#
JVM Java Web JavaScript Cloud Python/Ruby Cocoa Swift 5 * Language adoption is driven by platforms (not by languages themselves) * Platforms can be a mix of ecosystem factors » Hence, Swift’s popularity is guaranteed… -
Why will Swift be popular? 6 * Designated successor of
Objective-C: main language for macOS/iOS/watchOS/tvOS (13mio registered devs) * Apple is squarely pushing into the education space (”Everyone can code”) » Ok, it will be mainstream — why is that important? [After open-sourcing, Swift quickly became the most popular language project on GitHub] -
Why will Swift be popular? 6 * Designated successor of
Objective-C: main language for macOS/iOS/watchOS/tvOS (13mio registered devs) * Apple is squarely pushing into the education space (”Everyone can code”) » Ok, it will be mainstream — why is that important? [After open-sourcing, Swift quickly became the most popular language project on GitHub] -
Why will Swift be popular? Why is it extraordinary ?
6 * Designated successor of Objective-C: main language for macOS/iOS/watchOS/tvOS (13mio registered devs) * Apple is squarely pushing into the education space (”Everyone can code”) » Ok, it will be mainstream — why is that important? [After open-sourcing, Swift quickly became the most popular language project on GitHub] -
Functional Programming 7 * It’s the combination with the language
design: adoption of many functional features -
From Objective-C to Swift Higher-order functions & closures Value types
(products & sums) Advanced type system Explicit control of mutability Explicit optionals Generics & protocols with associated types Type inference REPL Pattern matching 8 * Most improvements facilitate functional programming including a sophisticated type system — a first in a mainstream language. » So, in summary… -
From Objective-C to Swift Higher-order functions & closures Value types
(products & sums) Advanced type system Explicit control of mutability Explicit optionals Generics & protocols with associated types Type inference REPL Pattern matching λ 8 * Most improvements facilitate functional programming including a sophisticated type system — a first in a mainstream language. » So, in summary… -
From Objective-C to Swift Higher-order functions & closures Value types
(products & sums) Advanced type system Explicit control of mutability Explicit optionals Generics & protocols with associated types Type inference REPL Pattern matching λτ 8 * Most improvements facilitate functional programming including a sophisticated type system — a first in a mainstream language. » So, in summary… -
“Swift encourages typed functional programming in a mainstream language.” 9
* A unique opportunity to improve programming practice on a large scale. » Seeing the WWDC 2014 reveal, I was immediately intrigued… -
Haskell for Mac Adopting Swift right out of the gate
10 * Swift introduction: WWDC’14 * Just started on Haskell for Mac (Haskell IDE & learning platform with playgrounds) for real * Adopted Swift from an early Swift beta release; submitted to App Store about 1 year later. » But let’s get back to… See also https://speakerdeck.com/mchakravarty/functional-programming-in-a-stateful- world -
“Swift encourages typed functional programming in a mainstream language.” 11
* Type system plays a special role in Swift’s design. * Value types, optional types, generics, associated types, type inference, algebraic types, protocol types, etc. » Why care about type systems? -
Catch bugs & prevent crashes 13 * Conventional wisdom: types
let the compiler catch bugs and prevent crashes. » Or, looking at it differently… -
“What if you could gain the same confidence in correctness
with fewer or simpler tests?” 15 * Less time writing, maintaining, and running tests, but still have confidence in code behaviour. » How does this work? -
Possible states of your application 16 * Error states =
once reached app misbehaves * Tested states = error states shown to be unreachable & valid states shown to be reachable * Type systems & FP statically reduce possible states. » That’s great, but does that mean that… -
Possible states of your application Error states 16 * Error
states = once reached app misbehaves * Tested states = error states shown to be unreachable & valid states shown to be reachable * Type systems & FP statically reduce possible states. » That’s great, but does that mean that… -
Possible states of your application Error states Tested states 16
* Error states = once reached app misbehaves * Tested states = error states shown to be unreachable & valid states shown to be reachable * Type systems & FP statically reduce possible states. » That’s great, but does that mean that… -
Possible states of your application Error states Tested states with
types 16 * Error states = once reached app misbehaves * Tested states = error states shown to be unreachable & valid states shown to be reachable * Type systems & FP statically reduce possible states. » That’s great, but does that mean that… -
Possible states of your application Error states Tested states eliminate
bugs with types 16 * Error states = once reached app misbehaves * Tested states = error states shown to be unreachable & valid states shown to be reachable * Type systems & FP statically reduce possible states. » That’s great, but does that mean that… -
Possible states of your application Error states Tested states eliminate
bugs eliminate tests with types 16 * Error states = once reached app misbehaves * Tested states = error states shown to be unreachable & valid states shown to be reachable * Type systems & FP statically reduce possible states. » That’s great, but does that mean that… -
Are types restraints? 17 * Does that mean that types
are merely a tool for policing your code? * And what if types don’t help finding bugs (i.e., reducing states in your code)? -
Types are a design tool! 18 * Type are a
language to talk about programs, * encoding static program properties (static = inferred/checked without running the code). * They are a guide, helping with code design & providing automated consistency checks. -
Type Systems Languages Program s “Make undesirable states unrepresentable.” 19
* For types to effectively constrain the state space, they need to inform code design. * For that to work well, language design matters. * Swift’s design recognises that. We need to as well (or types don’t help much). » This gets us back to Swift… -
“Swift encourages typed functional programming in a mainstream language.” 20
* But what does that mean in practice? * Two central & type-based features of Swift. * Importance: use in standard library (& new Foundation); evolution from Swift 1 to Swift 3 » Let’s look at these features in an iPhone app… -
“Swift encourages typed functional programming in a mainstream language.” Value
types Immutable model & UI state machines Protocols with associated types Generic change propagation 20 * But what does that mean in practice? * Two central & type-based features of Swift. * Importance: use in standard library (& new Foundation); evolution from Swift 1 to Swift 3 » Let’s look at these features in an iPhone app… -
Goals.app 21 * Simple app to track personal goals with
desired repetitions daily, weekly, or monthly. -
Goals.app 21 * Simple app to track personal goals with
desired repetitions daily, weekly, or monthly. -
Goals.app 21 * Simple app to track personal goals with
desired repetitions daily, weekly, or monthly. -
Goals.app 21 * Simple app to track personal goals with
desired repetitions daily, weekly, or monthly. -
Goals.app 21 * Simple app to track personal goals with
desired repetitions daily, weekly, or monthly. -
Value Types Localise & structure change 22 * What are
value types? * How do they localise and structure changing state? » First, what are value types? * See also WWDC 2016, Session 419 ”Protocol and Value Oriented Programming in UIKit Apps” -
Value types in Swift struct Point { var x: Double
var y: Double } struct enum Result<ResultType> { case result(value: ResultType) case error(err: NSError) } enum with associated values 23 * Two kinds of value types (products and sums) * Maybe generic (parameterised) * Switch with nested pattern matching to decompose -
Value types in Swift struct Point { var x: Double
var y: Double } struct enum Result<ResultType> { case result(value: ResultType) case error(err: NSError) } enum with associated values generic type parameter 23 * Two kinds of value types (products and sums) * Maybe generic (parameterised) * Switch with nested pattern matching to decompose -
Reference types versus value types var v: RefPnt class RefPnt
{ var x, y: Double … } v = RefPnt(x: 0, y: 0) var w: RefPnt 24 * Reference types: ’v’ changes when ’w’ changes * Same behaviour when ’v’ is passed to a parameter ’w’ of a function -
Reference types versus value types var v: RefPnt class RefPnt
{ var x, y: Double … } v = RefPnt(x: 0, y: 0) w = v x: 0, y: 0 var w: RefPnt 24 * Reference types: ’v’ changes when ’w’ changes * Same behaviour when ’v’ is passed to a parameter ’w’ of a function -
Reference types versus value types var v: RefPnt class RefPnt
{ var x, y: Double … } v = RefPnt(x: 0, y: 0) w = v w.x = 10 x: 0, y: 0 var w: RefPnt x: 10, y: 0 24 * Reference types: ’v’ changes when ’w’ changes * Same behaviour when ’v’ is passed to a parameter ’w’ of a function -
Reference types versus value types var v: Point struct Point
{ var x, y: Double } v = Point(x: 0, y: 0) var w: Point 25 * Value types: ’v’ remains unchanged by changes to ’w’. * Same behaviour when ’v’ is passed to a parameter ’w’ of a function -
Reference types versus value types var v: Point struct Point
{ var x, y: Double } v = Point(x: 0, y: 0) w = v x: 0, y: 0 var w: Point 25 * Value types: ’v’ remains unchanged by changes to ’w’. * Same behaviour when ’v’ is passed to a parameter ’w’ of a function -
Reference types versus value types var v: Point struct Point
{ var x, y: Double } v = Point(x: 0, y: 0) w = v w.x = 10 x: 0, y: 0 var w: Point x: 0, y: 0 x: 10, y: 0 25 * Value types: ’v’ remains unchanged by changes to ’w’. * Same behaviour when ’v’ is passed to a parameter ’w’ of a function -
var v: Point x: 0, y: 0 var w: Point
x: 0, y: 0 x: 10, y: 0 Localise change Facilitate local reasoning Value types 26 * Local reasoning and composition are core to functional programming. » Let’s look at this in your example app… -
Value Types Immutable Model 27 » We start by improving
the app architecture… -
MVC & MVVM Architectures Controller View Model 28 * Standard
Cocoa MVC or more fancy MVVM, write access to the model gets overshared. * Want to limit access and make change propagation explicit. » Let’s do that for Goals… -
MVC & MVVM Architectures View C Model View Model 28
* Standard Cocoa MVC or more fancy MVVM, write access to the model gets overshared. * Want to limit access and make change propagation explicit. » Let’s do that for Goals… -
MVC & MVVM Architectures View C Model View Model passed
around as reference type ⁇ Accidental changes ⁇ Changes in wrong order 28 * Standard Cocoa MVC or more fancy MVVM, write access to the model gets overshared. * Want to limit access and make change propagation explicit. » Let’s do that for Goals… -
Immutable Goals struct Goal { let uuid: UUID // fast
equality var colour: UIColor var title: String var interval: GoalInterval var frequency: Int } typealias GoalProgress = (goal: Goal, count: Int?) typealias Goals = [GoalProgress] // array 29 * Localise change and facilitate local reasoning * How do we propagate change in this set up? We’ll get to that in a moment. » This is a simple example. You may wonder, does this idea scale? -
Immutable Models Do Scale! Haskell for Mac Uses an immutable
model Adopts Cocoa Document Architecture “Functional Programming in a Stateful World” YOW! Lambda Jam 2015 (on speakerdeck.com) 30 * https://speakerdeck.com/mchakravarty/functional-programming-in-a-stateful-world » But what if I have additional constraints? -
But what if… 31 * What if you can’t choose
your model representation freely? * What if you use a database. * What if there are updates over the network or you use data syncing. * Haskell SpriteKit: https://github.com/mchakravarty/HaskellSpriteKit -
But what if… “Use an immutable wrapper API to wrap
mutable structures.” ✓ React (Native) Virtual DOM wraps DOM ✓ Haskell SpriteKit — Haskell library binding Algebraic data type wraps SpriteKit’s mutable object graph 31 * What if you can’t choose your model representation freely? * What if you use a database. * What if there are updates over the network or you use data syncing. * Haskell SpriteKit: https://github.com/mchakravarty/HaskellSpriteKit -
Replacing Tests ✓ Avoid accidental changes of model state Replaces
integration tests ✓ Avoid races on concurrent threads Often not properly tested 32 * Code that needs access to the current model state (such as views presenting model state) cannot accidentally change it. [Saves integration tests.] * Model state can be passed to concurrent threads without risk of races. [Typically not properly tested anyway.] » On to UI state machines… -
Is goal active? 34 * Modal mode changes of UI
screens * In each mode different sets of variables are valid (this is usually implicit & unchecked) * Goals.app: editing mode needs to maintain activity flags until changes committed -
Is goal active? typealias GoalProgress = (goal: Goal, count: Int?)
typealias Goals = [GoalProgress] Need to be careful not to lose progress info 34 * Modal mode changes of UI screens * In each mode different sets of variables are valid (this is usually implicit & unchecked) * Goals.app: editing mode needs to maintain activity flags until changes committed -
var goalsActivity: [Bool] Not valid outside of editing mode 35
* Maintain mode-specific state * State transitions need to be exhaustive, initialise variables & animations etc -
enum GoalsEditState { case displaying case editing(goalsActivity: [Bool]) } 36
* Enums encode alternative UI states * Associated values contains information specific to a particular UI state -
enum GoalsEditState { case displaying case editing(goalsActivity: [Bool]) } var
editState: GoalsEditState = .displaying … GoalsController: UITableViewController 37 * Exhaustiveness of switches ensures all transitions are covered. * Mode-specific state is initialised and to some degree finalised. -
enum GoalsEditState { case displaying case editing(goalsActivity: [Bool]) } var
editState: GoalsEditState = .displaying … switch editState { // change mode GoalsController: UITableViewController 37 * Exhaustiveness of switches ensures all transitions are covered. * Mode-specific state is initialised and to some degree finalised. -
enum GoalsEditState { case displaying case editing(goalsActivity: [Bool]) } var
editState: GoalsEditState = .displaying … switch editState { // change mode case .displaying: navigationItem.setLeftBarButton(nil, …) editState = .editing(goalsActivity: dataSource.goalsActivity) case .editing(let goalsActivity): GoalsController: UITableViewController alter UI state 37 * Exhaustiveness of switches ensures all transitions are covered. * Mode-specific state is initialised and to some degree finalised. -
enum GoalsEditState { case displaying case editing(goalsActivity: [Bool]) } var
editState: GoalsEditState = .displaying … switch editState { // change mode case .displaying: navigationItem.setLeftBarButton(nil, …) editState = .editing(goalsActivity: dataSource.goalsActivity) case .editing(let goalsActivity): navigationItem.setLeftBarButton(addButton, …) dataSource.commitGoalsActivity(goalsActivity) editState = .displaying } GoalsController: UITableViewController alter UI state 37 * Exhaustiveness of switches ensures all transitions are covered. * Mode-specific state is initialised and to some degree finalised. -
guard case .editing(var goalsActivity) = editState else { return }
let newIsActive = !goalsActivity[indexPath.item] cell.editingAccessoryType = newIsActive ? .checkmark : .none access to mode-specific state is guarded enum GoalsEditState { case displaying case editing(goalsActivity: [Bool]) } GoalsController 38 * Access to mode-specific state is guarded -
Replacing Tests ✓ Avoid inconsistent state changes Replaces UI tests
✓ Avoid non-exhaustive transitions Replaces exhaustiveness tests 39 * UI state remains consistent (the various variables making up the state machine state; states are explicit in the code & compiler can check exhaustiveness etc). [Tests checking state consistency and exhaustiveness of transitions.] -
Protocols with associated types Structured change propagation 40 * What
are protocols and what associated types? * How can we use them in a simple API for structured change propagation? » First, conventional change propagation… -
“Now that the model layer is immutable, how do we
change anything?” 41 » We don’t have to… -
42 Δt: 4s Mutable Variable 46 * The contents of
the variable changes… » …in contrast to the value of a time series… -
Δt 0 1 2 3 4 42 73 11 7
25 ✓ Flow of changes explicit in code ✓ Changes can be processed with combinators (map, fold, accumulate, and so on) 47 * The time series includes all values at once — it reifies time. * Can be regarded as a function over time, or an unbound stream of values. * The idea behind FRP. -
Changes API protocol Observable: class { associatedtype ObservedValue func observe<Context:
AnyObject>( withContext context: Context, observer: (Context, ObservedValue) -> ()) -> () } registered observer call back 48 * Simple interface for change propagation -
class Changing<Value>: Observable { typealias ObservedValue = Value func announce(change:
Value) { … } func observe … { … } } Ephemeral stream of changes protocol Observable: class { associatedtype ObservedValue func observe<Context: AnyObject>( withContext context: Context, observer: (Context, ObservedValue) -> ()) -> () } Add another change value into the stream 49 * Two forms of observables: (1) ephemeral and (2) accumulating * (1) Once registered an observer is notified of *subsequent* changes -
protocol Observable: class { associatedtype ObservedValue func observe<Context: AnyObject>( withContext
context: Context, observer: (Context, ObservedValue) -> ()) -> () } class Accumulating<Value, Accumulator>: Observable { typealias ObservedValue = Accumulator init <Observed: Observable where …> ( observing observed: Observed, startingFrom initial: Accumulator, accumulate: (Value, Accumulator) -> Accumulator) func observe … { … } } Accumulating value Accumulating change 50 * Two forms of observables: (1) ephemeral and (2) accumulating * (2) On registration, observer is immediately notified of current accumulator value -
extension Observable { } map apply function to each observed
value merge combine two streams of observations accumulate combine the values of a stream of observations using an accumulation function … 51 -
Change Propagation in Goals G0 3 G1 nil G2 1
Model Views 52 * GoalEdit and ProgressEdit *separates* changes to goals and progress counts, respectively. » Composition interfaces are checked… -
Change Propagation in Goals G0 3 G1 nil G2 1
observe Model Views observe 52 * GoalEdit and ProgressEdit *separates* changes to goals and progress counts, respectively. » Composition interfaces are checked… -
Change Propagation in Goals G0 3 G1 nil G2 1
observe Model Views merge observe 52 * GoalEdit and ProgressEdit *separates* changes to goals and progress counts, respectively. » Composition interfaces are checked… -
Change Propagation in Goals G0 3 G1 nil G2 1
observe Model Views ProgressEdits merge GoalEdits observe 52 * GoalEdit and ProgressEdit *separates* changes to goals and progress counts, respectively. » Composition interfaces are checked… -
Change Propagation in Goals G0 3 G1 nil G2 1
observe Model Views Edits accumulate ProgressEdits merge GoalEdits observe 52 * GoalEdit and ProgressEdit *separates* changes to goals and progress counts, respectively. » Composition interfaces are checked… -
Change Propagation in Goals G0 3 G1 nil G2 1
observe Model Views Accumulating<Edit, Goals> Edits accumulate ProgressEdits merge GoalEdits observe 52 * GoalEdit and ProgressEdit *separates* changes to goals and progress counts, respectively. » Composition interfaces are checked… -
typealias GoalEdits = Changing<GoalEdit> enum GoalEdit { case add(goal: Goal)
case delete(goal: Goal) case update(goal: Goal) case setActivity(activity: [Bool]) } extension GoalEdit { func transform(_ goals: Goals) -> Goals { switch self { case .add(let newGoal): … Accumulating<Edit, Goals> Edits ProgressEdits 54 * State transformer functions might be used as edits (in more complex scenarios) -
typealias GoalEdits = Changing<GoalEdit> enum GoalEdit { case add(goal: Goal)
case delete(goal: Goal) case update(goal: Goal) case setActivity(activity: [Bool]) } extension GoalEdit { func transform(_ goals: Goals) -> Goals { switch self { case .add(let newGoal): … Accumulating<Edit, Goals> Edits ProgressEdits 54 * State transformer functions might be used as edits (in more complex scenarios) -
typealias ProgressEdits = Changing<ProgressEdit> enum ProgressEdit { case bump(goal: Goal)
} extension ProgressEdit { func transform(_ goals: Goals) -> Goals { … } Accumulating<Edit, Goals> Edits GoalEdits 55 -
typealias ProgressEdits = Changing<ProgressEdit> enum ProgressEdit { case bump(goal: Goal)
} extension ProgressEdit { func transform(_ goals: Goals) -> Goals { … } } Accumulating<Edit, Goals> Edits GoalEdits 55 -
typealias ProgressEdits = Changing<ProgressEdit> enum ProgressEdit { case bump(goal: Goal)
} extension ProgressEdit { func transform(_ goals: Goals) -> Goals { … } } typealias Edits = Changing<Edit> enum Edit { case goalEdit(edit: GoalEdit) case progressEdit(edit: ProgressEdit) } Accumulating<Edit, Goals> GoalEdits 55 -
let model: Accumulating<Edit, Goals> = edits.accumulate(startingFrom: initialGoals) { edit, currentGoals
in return edit.transform(currentGoals) } Edits ProgressEdits GoalEdits 56 * Some similarity to Redux -
Replacing Tests ✓ Avoid changes from unexpected places Replaces integration
tests ✓ Avoid conflicting state changes Replaces integration tests 57 * Asserting properties of components during composition wrt to change propagation through the app * UI and concurrency-related problems are hard/complicated to test and often don’t get tested. -
mchakravarty TacticalGrace justtesting.org haskellformac.com Type systems are a design tool
Swift encourages typed functional programming Types replace entire categories of tests Check out the source code of Goals.app! https://github.com/mchakravarty/goalsapp types >< state 58 * Many of the replaced tests are often omitted as they are hard (UI testing, concurrency, …) -
Wikimedia https://www.flickr.com/photos/30998987@N03/5408763997 Image Attribution https://commons.wikimedia.org/wiki/File:Nile_blue_05.jpg https://dribbble.com/shots/1667698-Free-vector-Macbook-Ipad-and-Iphone https://www.flickr.com/photos/provisions/7986149891/ https://commons.wikimedia.org/wiki/File:Handcuffs01_2008-07-27.jpg https://commons.wikimedia.org/wiki/File:Set_square_Geodreieck.svg https://www.flickr.com/photos/digitalcurrency/2438119267
https://www.flickr.com/photos/pviojoenchile/2760021279/ https://openclipart.org/detail/463/heart 60