How Doctrine caching can skyrocket your application (PHPUK Conference)

When people talk about Doctrine (or any ORM for that matter), the performance issue always comes up fairly quickly. Besides the fact that Doctrine will help you develop faster, so a little overhead doesn't really matter, there are numerous options to increase the performance of the application.

By understanding how the system works in the first place, a lot of issues can be avoided right away.

When you have done everything to avoid these pitfalls, you can bring in the big guns: caching. Doctrine has different caching mechanism and since Doctrine 2.5 "Second Level Cache" was added to our toolbox. After this talk, you should know what the impact is of every cache and how to use it.

Transcript

1. HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION Jachim Coudenys Jachim Coudenys Jachim Coudenys Jachim Coudenys Jachim Coudenys Jachim Coudenys Jachim Coudenys Jachim Coudenys jachim.coudenys@combellgroup.com jachim.coudenys@combellgroup.com jachim.coudenys@combellgroup.com jachim.coudenys@combellgroup.com jachim.coudenys@combellgroup.com jachim.coudenys@combellgroup.com jachim.coudenys@combellgroup.com jachim.coudenys@combellgroup.com @coudenysj @coudenysj @coudenysj @coudenysj @coudenysj @coudenysj @coudenysj @coudenysj
2. Jachim Coudenys Jachim Coudenys
5. HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION
6. HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE HOW DOCTRINE CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN CACHING CAN SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR SKYROCKET YOUR APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION APPLICATION If you're using Doctrine ORM. If you're using Doctrine ORM.
7. WHAT IS DOCTRINE
8. OBJECT-RELATIONAL MAPPER ... is a programming technique for converting data between incompatible type systems using object-oriented programming languages. — https://en.wikipedia.org/wiki/Object-relational_mapping
9. TERMINOLOGY Entity Mapping Repository EntityManager
10. https://github.com/coudenysj/doctrine2-orm-tutorial https://github.com/coudenysj/doctrine2-orm-tutorial DOCTRINE2-ORM-TUTORIAL
11. SOME INTERNALS SOME INTERNALS SOME INTERNALS SOME INTERNALS SOME INTERNALS SOME INTERNALS SOME INTERNALS SOME INTERNALS
13. ENTITYMANAGER
15. UNIT OF WORK Maintains a list of objects affected by a business transaction and coordinates the writing out of changes and the resolution of concurrency problems. — https://martinfowler.com/eaaCatalog/unitOfWork.html
17. TRANSACTIONAL WRITE- BEHIND
19. IDENTITY MAP Ensures that each object gets loaded only once by keeping every loaded object in a map. Looks up objects using the map when referring to them. — https://martinfowler.com/eaaCatalog/identityMap.html
21. ENTITY STATES NEW MANAGED DETACHED REMOVED
23. HYDRATION
25. PROXIES (LAZY LOADING) An object that doesn't contain all of the data you need but knows how to get it. — https://martinfowler.com/eaaCatalog/lazyLoad.html
27. https://github.com/coudenysj/doctrine2-orm-tutorial https://github.com/coudenysj/doctrine2-orm-tutorial DOCTRINE2-ORM-TUTORIAL
28. TRACKING CHANGES TRACKING CHANGES TRACKING CHANGES TRACKING CHANGES TRACKING CHANGES TRACKING CHANGES TRACKING CHANGES TRACKING CHANGES
29. CREATING Create new object (NEW) Persist the object (MANAGED) Flush the EM UoW calculates changes => INSERT SQL Transaction
30. GETTING Request object Identity Map lookup Database query Object hydration Save hydrated object to IM (MANAGED)
31. UPDATING Get the object (MANAGED) Change a property Persist the object (no-op) Flush the EM UoW calculates changes => UPDATE SQL Transaction
32. DELETING Get the object (MANAGED) Ask for removal (REMOVED) Flush the EM UoW calculates changes => DELETE SQL Transaction
33. CHANGE TRACKING POLICIES
34. DEFERRED IMPLICIT Default Property-by-property comparison Persistence by reachability Slowest
35. DEFERRED EXPLICIT Property-by-property comparison EntityManager#persist(entity) No "dirty" checking
36. DEFERRED EXPLICIT http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/change-tracking-policies.html#deferred-explicit http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/change-tracking-policies.html#deferred-explicit /** * @Entity * @ChangeTrackingPolicy("DEFERRED_EXPLICIT") */ class User { // ... }
37. NOTIFY Notify interested listeners of changes NotifyPropertyChanged
38. NOTIFY http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/change-tracking-policies.html#notify http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/change-tracking-policies.html#notify use Doctrine\Common\NotifyPropertyChanged, Doctrine\Common\PropertyChangedListener; /** * @Entity * @ChangeTrackingPolicy("NOTIFY") */ class MyEntity implements NotifyPropertyChanged { private $listeners = array(); public function addPropertyChangedListener(PropertyChangedListener $listener) { $this->listeners[] = $listener; } }
39. NOTIFY http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/change-tracking-policies.html#notify http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/change-tracking-policies.html#notify class MyEntity implements NotifyPropertyChanged { protected function onPropertyChanged($propName, $oldValue, $newValue) { if ($this->listeners) { foreach ($this->listeners as $listener) { $listener->propertyChanged($this, $propName, $oldValue, $newValue); } } } public function setData($data) { if ($data != $this->data) { $this->onPropertyChanged('data', $this->data, $data); $this->data = $data; } } }
40. FURTHER IMPROVEMENTS
41. ALTERNATIVE QUERY RESULT FORMATS Read only data Scalar values Nested array graph
42. ALTERNATIVE QUERY RESULT FORMATS $dql = "SELECT b, e, r, p FROM Bug b JOIN b.engineer e ". "JOIN b.reporter r JOIN b.products p ORDER BY b.created DESC"; $query = $entityManager->createQuery($dql); $bugs = $query->getArrayResult(); foreach ($bugs as $bug) { echo $bug['description'] . " - " . $bug['created']->format('d.m.Y')."\n"; echo " Reported by: ".$bug['reporter']['name']."\n"; echo " Assigned to: ".$bug['engineer']['name']."\n"; foreach ($bug['products'] as $product) { echo " Platform: ".$product['name']."\n"; } echo "\n"; }
43. READ ONLY ENTITIES /** * @Entity(readOnly=true) * @Table(name="products") */ class Product { }
44. https://github.com/coudenysj/doctrine2-orm-tutorial https://github.com/coudenysj/doctrine2-orm-tutorial DOCTRINE2-ORM-TUTORIAL
45. EXTRA-LAZY COLLECTIONS LAZY (default) EAGER (join) EXTRA_LAZY
46. EXTRA-LAZY COLLECTIONS http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/tutorials/extra-lazy-associations.html#enabling-extra-lazy-associations http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/tutorials/extra-lazy-associations.html#enabling-extra-lazy-associations namespace Doctrine\Tests\Models\CMS; use Doctrine\ORM\Annotation as ORM; /** * @ORM\Entity */ class CmsGroup { /** * @ORM\ManyToMany( * targetEntity="CmsUser", mappedBy="groups", fetch="EXTRA_LAZY" * ) */ public $users; }
48. DOCTRINE\CACHE
49. DOCTRINE\COMMON\ CACHE\CACHE INTERFACE fetch($id) contains($id) save($id, $data, $lifeTime = false) delete($id)
50. CACHE DRIVERS APCu Memcached Xcache Redis File Array* ...
51. CACHING IN DOCTRINE
52. METADATA Removes parsing overhead (Annotations, YML, XML)
53. QUERY DQL => SQL
54. RESULT Store SQL data result to cache Still a need to hydrate objects Joins will be stored as-is
55. CUSTOM CACHING CUSTOM CACHING CUSTOM CACHING CUSTOM CACHING CUSTOM CACHING CUSTOM CACHING CUSTOM CACHING CUSTOM CACHING
56. HYDRATED RESULT
57. REPOSITORY DECORATOR
58. GENERALLY A BAD IDEA Serializing entities? Serializing collections? Detached entities https://github.com/doctrine/doctrine2/pull/172/file
60. SECOND LEVEL SECOND LEVEL SECOND LEVEL SECOND LEVEL SECOND LEVEL SECOND LEVEL SECOND LEVEL SECOND LEVEL CACHING CACHING CACHING CACHING CACHING CACHING CACHING CACHING
62. SECOND LEVEL CACHING No entity instances Only entity identifier and values Best suited for read-only data
63. TYPES Entity data: id + values Collection data: ownerId id + list of ids Query data: list of ids
64. ENTITY CACHE http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#entity-cache-definition http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#entity-cache-definition /** * @Entity * @Cache() */ class Country { // ... }
65. ASSOCIATION CACHE http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#association-cache-definition http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#association-cache-definition /** * @Entity * @Cache() */ class State { //... /** * @Cache() * @ManyToOne(targetEntity="Country") * @JoinColumn(name="country_id", referencedColumnName="id") */ protected $country; }
66. REGIONS
67. MODES READ_ONLY NONSTRICT_READ_WRITE READ_WRITE
68. QUERY CACHE
69. QUERY CACHE http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#using-the-query-cache http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#using-the-query-cache $result1 = $em->createQuery('SELECT c FROM Country c ORDER BY c.name') ->setCacheable(true) ->getResult();
70. QUERY CACHE: MODES Cache::MODE_GET Cache::MODE_PUT Cache::MODE_NORMAL (default) Cache::MODE_REFRESH
71. QUERY CACHE: DELETE / UPDATE QUERIES
72. QUERY CACHE: DELETE / UPDATE QUERIES http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#delete-update-queries http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#delete-update-queries // Execute and invalidate $this->em->createQuery( "UPDATE Entity\Country u SET u.name = 'unknown' WHERE u.id = 1" ) ->setHint(Query::HINT_CACHE_EVICT, true) ->execute();
73. QUERY CACHE: DELETE / UPDATE QUERIES http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#delete-update-queries http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#delete-update-queries // Execute $this->em->createQuery( "UPDATE Entity\Country u SET u.name = 'unknown' WHERE u.id = 1" ) ->execute(); // Invoke Cache API $em->getCache()->evictEntityRegion('Entity\Country');
74. QUERY CACHE: DELETE / UPDATE QUERIES http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#delete-update-queries http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#delete-update-queries // Execute $this->em->createQuery( "UPDATE Entity\Country u SET u.name = 'unknown' WHERE u.id = 1" ) ->execute(); // Invoke Cache API $em->getCache()->evictEntity('Entity\Country', 1);
75. QUERY CACHE: REPOSITORIES http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#using-the-repository-query-cache http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/second-level-cache.html#using-the-repository-query-cache // load from database and store cache query key hashing the query + parameters + last timestamp cache region.. $entities = $em->getRepository('Entity\Country')->findAll(); // load from query and entities from cache.. $entities = $em->getRepository('Entity\Country')->findAll(); // update the timestamp cache region for Country $em->persist(new Country('zombieland')); $em->flush(); $em->clear(); // Reload from database. // At this point the query cache key if not logger valid, the select goes straight $entities = $em->getRepository('Entity\Country')->findAll();
76. LIMITATIONS Single application Single Primary key column
77. https://github.com/coudenysj/doctrine2-orm-tutorial https://github.com/coudenysj/doctrine2-orm-tutorial DOCTRINE2-ORM-TUTORIAL
78. CONCLUSION Keep Identity Map / internals in mind Using the basic caching (query + mapping) + opcode caching is a must Cache heavy queries with result cache Give Second Level Caching a try (even if only for entities) Query Cache !== Second Level Query Cache
79. http://doctrine-project.org/ https://www.flickr.com/photos/spacex/40126461851 https://github.com/coudenysj?tab=repositories&q=doctrine @coudenysj jachim.coudenys@combellgroup.com THANK YOU https://joind.in/talk/4a903 https://joind.in/talk/4a903