Slick in the field, learning to forget ORM

For many years developers on the JVM platform were used to ORM frameworks. Our code was filled with annotations (or worst XML configuration files). We learned how to map our classes to relational tables and deal with its peculiarities. We had big, sometimes huge graph of objects, mapped to relational tables. We brought them all to memory, changed them and let it be automagically persisted back at the end of our transactions.

In a Scala/Slick world we deal with data and persistence in a totally different way. Our model are immutable structures (case classes and tuples), nullable columns are mapped to Options (and that's cool), relations are expressed by id references instead of object associations (that's less cool). Instead of writing xml or filling our code with annotations, we write type-safe idiomatic Scala code. We compose queries in a functional style. We map and flatMap over it.

But is that good or bad? What brings Slick to the readability and maintainability of our code base? How do we test our business logic when a great deal of it is executed outside the JVM?

In this talk we'll share our experience in building real world applications with Slick. The lessons learned, the patterns and anti-patterns we went through in our journey to shift the way we were used to think about persistence.

This is not an introduction to Slick. This talk is aimed to developers already using Slick.

Transcript

1. Slick in the field learning to forget ORM
2. About Me Renato Guerra Cavalcanti (@renatocaval)
 Founder of BeScala
 renato@strongtyped.io
 www.strongtyped.io Tried psychologist…
 Than philosopher…
 Than developer
3. This talk is NOT about • Building complex queries in Slick • Solving left outer join, right outer join, etc. • Reducing noisy and verbosity (specially when joining)
4. About this talk • ORM - the memory / persistency challenge • Coding without associations • Building a thin layer on top of Slick • Organising your code base • Testing
5. ORM - the memory / persistency challenge • We have two representations of data that we must keep in sync • Object lifecycle - new / persisted / dirty • Versioning - optimistic locking • OneToMany / ManyToMany • Ordering for associations • Caching
6. ORM - the memory / persistency challenge • We have two representations of data that we must keep in sync • Object lifecycle - new / persisted / dirty • Versioning - optimistic locking • OneToMany / ManyToMany • Ordering for associations • Caching
7. ORM - the memory / persistency challenge • We have two representations of data that we must keep in sync • Object lifecycle - new / persisted / dirty • Versioning - optimistic locking • OneToMany / ManyToMany • Ordering for associations • Caching
8. ORM - the memory / persistency challenge • We have two representations of data that we must keep in sync • Object lifecycle - new / persisted / dirty • Versioning - optimistic locking • Ordering for associations • Ordering for associations • Caching
9. ORM - the memory / persistency challenge • We have two representations of data that we must keep in sync • Object lifecycle - new / persisted / dirty • Versioning - optimistic locking • Ordering for associations • OneToMany / ManyToMany • Caching
10. ORM - the memory / persistency challenge • We have two representations of data that we must keep in sync • Object lifecycle - new / persisted / dirty • Versioning - optimistic locking • Ordering for associations • OneToMany / ManyToMany • Caching
11. ORM - the memory / persistency challenge • We have two representations of data that we must keep in sync • Object lifecycle - new / persisted / dirty • Versioning - optimistic locking • Ordering for associations • OneToMany / ManyToMany • Caching
12. case class Artist(name:String, id:Option[Int] = None) ! case class Album(name:String, genre:String, artist:Artist, tracks:Seq[Track], id:Option[Int] = None) ! case class Track(name:String, id:Option[Int] = None) Coding without associations
13. case class Artist(name:String, id:Option[Int] = None) ! case class Album(name:String, genre:String, artistId:Int, id:Option[Int] = None) ! case class Track(name:String, albumId:Int, id:Option[Int] = None) case class Artist(name:String, id:Option[Int] = None) ! case class Album(name:String, genre:String, artist:Artist, tracks:Seq[Track], id:Option[Int] = None) ! case class Track(name:String, id:Option[Int] = None) Coding without associations
14. Trying to emulate it case class Album(name:String, genre:String, artistId:Int, id:Option[Int] = None) { ! // omitting implicit session def artist = Artists.findById(artistId) def tracks = id.map(i => Tracks.findByAlbumId(i)) .getOrElse(Seq()) } Coding without associations
15. case class Album(name:String, genre:String, artistId:Int, id:Option[Int] = None) { ! var artist : Artist = null var tracks : ListBuffer[Track] = ListBuffer() } Choose for mutability, argh! Trying to emulate it case class Album(name:String, genre:String, artistId:Int, id:Option[Int] = None) { ! // omitting implicit session def artist = Artists.findById(artistId) def tracks = id.map(i => Tracks.findByAlbumId(i)) .getOrElse(Seq()) } Coding without associations
16. Embrace your rows • case classes are isomorphic to tables, they are rows • No associations like in OO, you have keys • You write part of your business logic in terms of SQL. But in Scala and type-safe! • Extensions to provide CRUD operations and basic queries • So, I tried to build my own slick dao library. But…
17. • case classes are isomorphic to tables, they are rows • ActiveRecord • Extensions to provide CRUD operations and basic queries • So, I tried to build my own slick dao library. But… Embrace your rows
18. • case classes are isomorphic to tables, they are rows • ActiveRecord • Helpers to provide CRUD operations and basic queries • CRUD operations are tight related with object lifecycle
 (eventually versioning as well) Embrace your rows
19. • case classes are isomorphic to tables, they are rows • ActiveRecord • Helpers to provide CRUD operations and basic queries • CRUD operations are tight related to object lifecycle
 Embrace your rows
20. • case classes are isomorphic to tables, they are rows • ActiveRecord • Helpers to provide CRUD operations and basic queries • CRUD operations are tight related to object lifecycle • IDs are crucial for Entities
 Embrace your rows
21. Generic CRUD operations case class Foo(name:String, id:Option[Int] = None) ! class FooTable(tag:Tag) extends Table[Foo](tag, "FOOS") { def name = column[String]("NAME") def id = column[Int]("ID", O.PrimaryKey, O.AutoInc) def * = (name, id.?) <> (Foo.tupled, Foo.unapply) } val Foos = TableQuery[FooTable] ! val foo = Foo("foo") val id = Foos.returning(Foos.map(_.id)).insert(foo) foo.copy(id = Some(id))
22. case class Foo(name:String, id:Option[Int] = None) ! class FooTable(tag:Tag) extends Table[Foo](tag, "FOOS") { def name = column[String]("NAME") def id = column[Int]("ID", O.PrimaryKey, O.AutoInc) def * = (name, id.?) <> (Foo.tupled, Foo.unapply) } val Foos = TableQuery[FooTable] ! val foo = Foo("foo") val id = Foos.returning(Foos.map(_.id)).insert(foo) foo.copy(id = Some(id)) Generic CRUD operations
23. Generic DAO • A well known column that has the role of PK • Helpers to extract and add id to your models • That’s easy, let’s build a Generic DAO for Slick! • But…
24. Generic DAO • A well known column that has the role of PK • Helpers to extract and add the id to your models • That’s easy, let’s build a Generic DAO for Slick! • But…
25. Generic DAO • A well known column that has the role of PK • Helpers to extract and add the id to your models • That’s easy, let’s build a Generic DAO for Slick! • But…
26. Generic DAO • A well known column that has the role of PK • Helpers to extract and add the id to your models • That’s easy, let’s build a Generic DAO for Slick! • But…
27. –Christopher Vogt "When implementing an abstract DAO be prepared to face a lot of challenges and get to the edges of your Scala type system skills and learn quite a bit about type inference order, implicit parameters, etc.” Generic DAO
28. Stop talking! Show some code
29. Organising your code • Separate case classes from mapping (tables) • Query / DAOs separated from Table definitions • Use tuples for link tables, nothing else • Avoid returning tuples from joins. Map to dedicated case class
30. Organising your code • Separate case classes from mapping (tables) • Queries separated from Table definitions • Use tuples for link tables, nothing else • Avoid returning tuples from joins. Map to dedicated case class
31. Organising your code • Separate case classes from mapping (tables) • Queries separated from Table definitions • Use tuples for link tables, nothing else • Avoid returning tuples from joins. Map to dedicated case class
32. Organising your code • Separate case classes from mapping (tables) • Queries separated from Table definitions • Use tuples for link tables, nothing else • Avoid returning tuples from joins. Map to dedicated case class
33. back to code
34. Testing • Lots of logic executed on SQL • In-Memory DB for tests proved to be a bad choise
 (garbage collection of Alias class on H2) • Fixtures DSL
35. Testing • Lots of logic executed on SQL • In-Memory DB for tests proved to be a bad choice
 (garbage collection of Alias class on H2) • Fixtures DSL
36. Testing • Lots of logic executed on SQL • In-Memory DB for tests proved to be a bad choise
 (garbage collection of Alias class on H2) • Fixtures DSL
37. Questions http://bit.ly/active-slick @renatocaval
 renato@strongtyped.io