Doctrine ORM: Deep Dive

Workshop slides to be used alongside the training repository, which can be found here: https://github.com/TheCadien/symfony-con-doctrine-deep-dive

Transcript

1. Oliver Kossin & Denis Brumann Doctrine ORM: Deep Dive SymfonyCon Amsterdam 2025
2. Oliver Kossin
 Senior Software Developer denkwerk Who are we? Denis Brumann
 Solution Architect Next Kraftwerke
3. Who are you? What is your PHP & Symfony experience? Which versions do you use in your day to day work? What is your Doctrine experience? How long have you been using it? Which database are you using it with primarily? Are you on the current major version? What features do you want to learn about?
4. Foundation
5. Doctrine Project More than an ORM PHP Drivers (mysqli, pdo, …) Doctrine DBAL Doctrine Libs (Common, Collection, …) Doctrine ORM Doctrine ODM Integrations (Doctrine Bundle)
6. Doctrine Libraries Low Level • Coding Standard • Common Extending DBAL • Data Fixtures • Migrations Abstractions • Persistence Shared functionality • Collections • Lexer • In fl ector • Instantiator Inactive • Annotations • Cache • Re fl ection
7. Doctrine DBAL Database Abstraction Layer “O ff ers an object-oriented API and a lot of additional horizontal features like database schema introspection and management” Current Major Version 4 With 3.0 the API changed a lot, e.g. stronger encapsulation of driver-speci fi c classes. Check the UPGRADE document in the repo https://github.com/doctrine/dbal/blob/3.10.2/UPGRADE.md#upgrade-to-30 Manage deprecations to prepare for upgrade
8. DBAL Architecture Layers Drivers abstract a PHP-speci fi c API Middleware sits between wrapper and driver Platforms abstracts generational changes, e.g. between MySQL and MariaDB Logging Schema Types
9. DBAL Architecture Platform con fi guration Drivers abstract a PHP-speci fi Middleware sits between wrapper and driver Platforms abstracts generational changes, e.g. between MySQL and MariaDB Logging Schema Types MariaDB mysql://app:!ChangeMe!@127.0.0.1:3306/app? serverVersion=10.11.2-MariaDB&charset=utf8mb4 MySQL mysql://app:!ChangeMe!@127.0.0.1:3306/app? serverVersion=8.0.32&charset=utf8mb4
10. DBAL Architecture Driver & Middleware Drivers abstract a PHP-speci fi c API Middleware sits between wrapper and driver Platforms abstracts generational changes, e.g. between MySQL and MariaDB Logging Schema Types Driver Connection Statement Result
11. Doctrine ORM Object Relational Mapper “O ff ers an object-oriented API and a lot of additional horizontal features like database schema introspection and management” Current Major Version 3 Works with both DBAL 3 & 4 Annotations are replaced with Attributes. Tools like Rector can help with the upgrade. Code generation is gone
12. ORM Architecture Layers EntityManager Entity Repository Query Unit Of Work Change Tracking Metadata Hydration Persister Mapping Cache Events
13. ORM Architecture Important Constraints Entity An entity class must not be fi nal nor read-only, although it can contain fi nal methods or read-only properties. Behavior in Entities is ok The Doctrine team discourages the use of anemic models, i.e. models with only getters/setters. • Entities can have invalid state, which might be accidentally submitted on fl ush • If persisted, corrupted data is loaded leading to hard to track bugs.
14. ORM Architecture UnitOfWork • InMemory-store of fetched or persist()-ed entities. • The thing being fl ush()-ed, i.e. persisting new entities, updating or deleting existing ones; making sure each is done in the right order • For more details on the concept,
 see: https://martinfowler.com/eaaCatalog/unitOfWork.html
15. Doctrine Upgrade Path “O ff ers an object-oriented API and a lot of additional horizontal features like database schema introspection and management” Current Major Version 3 Works with both DBAL 3 & 4 Annotations are replaced with Attributes. Tools like Rector can help with the upgrade. Code generation is gone
16. Exercise Setup the demo project
17. https://github.com/TheCadien/symfony-con-doctrine-deep-dive
18. Entity Mapping
19. Entity Mapping DBAL / Field Types String Mapping • AsciiStringType • StringType • TextType Date & Time Mapping • DateType • TimeType • DateTimeType • DateTimeTzType • VarDateTimeType + Immutable variations Removed in 4.0 • ArrayType • ObjectType Use JsonType for unstructured data Number Mapping • BigIntType • IntegerType • FloatType • NumberType Other Mapping • BlobType • BooleanType • EnumType • …
20. Entity Mapping DBAL / Field Types String • AsciiStringType • StringType • TextType Date • DateType • TimeType • DateTimeType • DateTimeTzType • VarDateTimeType + Immutable variations Removed • ArrayType • ObjectType Use JsonType for unstructured data Number Mapping • BigIntType • IntegerType • FloatType • NumberType Other • BinaryType • BlobType • BooleanType • EnumType Converts to PHP int when possible or keeps original type (usually string)
21. Entity Mapping DBAL / Field Types String • AsciiStringType • StringType • TextType Date • DateType • TimeType • DateTimeType • DateTimeTzType • VarDateTimeType + Immutable variations Removed • ArrayType • ObjectType Use JsonType for unstructured data Number Mapping • BigIntType • IntegerType • FloatType • NumberType Other • BinaryType • BlobType • BooleanType • EnumType Converts to BCMath Number-object
22. Entity Mapping DBAL / Field Types String • AsciiStringType • StringType • TextType Date & Time Mapping • DateType • TimeType • DateTimeType • DateTimeTzType • VarDateTimeType + Immutable variations Removed • ArrayType • ObjectType Use JsonType for unstructured data Number • BigIntType • IntegerType • FloatType • NumberType Other • BinaryType • BlobType • BooleanType • EnumType Uses slower DateTime::__construct() instead of ::createFromFormat() Usually only needed for certain Postgres con fi gurations
23. Entity Mapping ORM / Identi fi er Mapping <?php #[Entity] class Message { #[Id] #[Column(type: Types::INTEGER, options=[‘unsigned’: true])] #[GeneratedValue(strategy: ‘AUTO’)] private int|null $id = null; // … } Maps and converts 4-byte integer values. Unsigned integer values have a range of 0 to 4294967295 while signed integer values have a range of −2147483648 to 2147483647. Not all DB vendors support unsigned integers
24. Entity Mapping ORM / Text <?php #[Entity] class Message { #[Column(type: TYPES::string)] private string $subject; #[Column(type: TYPES::TEXT, nullable: true)] private string|null $text; // … } Maps and converts string data without a maximum length. If you don't know the maximum length of the data to be stored, you should consider using this type.
25. Entity Mapping ORM / Column type
 (default: “string”) The mapping type to use for the column. This can be inferred from the PHP type, but be careful a type ‘string|null’ does not also set nullable: true. name (default: derived from name of property) The name of the column in the database. length (default: 255) The length of the column in the database. Applies only if a string-valued column is used. unique (default: false) Whether the column is a unique key. nullable (default: false) Whether the column is nullable. insertable (default: true) Whether the column should be inserted. updatable (default: true) Whether the column should be updated. generated (default: null) Whether the generated strategy should be 'NEVER', 'INSERT' and ‘ALWAYS’. enumType (requires PHP 8.1 and doctrine/orm 2.11) The PHP enum class name to convert the database value into. precision (default: 0) The precision for a decimal (exact numeric) column (applies only for decimal column), which is the maximum number of digits that are stored for the values. scale (default: 0) The scale for a decimal (exact numeric) column (applies only for decimal column), which represents the number of digits to the right of the decimal point and must not be greater than precision. columnDe fi nition Allows to de fi ne a custom DDL snippet that is used to create the column. Warning: This normally confuses the SchemaTool to always detect the column as changed. options Key-value pairs of options that get passed to the underlying database platform when generating DDL statements.
26. Entity Mapping ORM / Association Mapping #[Entity] class Product { #[OneToMany(targetEntity: Feature::class, mappedBy: 'product')] private Collection $features; // ... public function __construct() { $this->features = new ArrayCollection(); } } #[Entity] class Feature { #[ManyToOne(targetEntity: Product::class, inversedBy: 'features')] #[JoinColumn(name: 'product_id', referencedColumnName: 'id')] private Product|null $product = null; // ... }
27. Entity Mapping ORM / Association Mapping
28. Entity Mapping ORM / Special case: Unidirectional OneToMany #[Entity] class User { #[JoinTable(name: 'users_phonenumbers')] #[JoinColumn(name: 'user_id', referencedColumnName: 'id')] #[InverseJoinColumn(name: 'phonenumber_id', referencedColumnName: 'id', unique: true)] #[ManyToMany(targetEntity: 'Phonenumber')] private Collection $phonenumbers; public function __construct() { $this->phonenumbers = new ArrayCollection(); } // ... } #[Entity] class Phonenumber { // ... }
29. Entity Mapping ORM / Lazy Loading #[Entity] class User { #[ManyToMany(targetEntity: ‘Phonenumber’, fetch: 'LAZY'|'EAGER'|'EXTRA_LAZY')] private Collection $phonenumbers; public function __construct() { $this->phonenumbers = new ArrayCollection(); } // ... }
30. Entity Mapping ORM / Proxy Con fi guration when@prod: doctrine: orm: auto_generate_proxy_classes: false enable_lazy_ghost_objects: true proxy_dir: '%kernel.build_dir%/doctrine/orm/Proxies'
31. Entity Mapping ORM / Cascade Options #[Entity] class User { #[ManyToMany(targetEntity: ‘Phonenumber’, cascade: ['persist', 'remove'])] private Collection $phonenumbers; public function __construct() { $this->phonenumbers = new ArrayCollection(); } // ... }
32. Entity Mapping ORM / Embeddable & Embed #[Entity] class User { #[Embedded(class: Address::class)] private Address $address; } #[Embeddable] class Address { #[Column(type: "string")] private string $street; #[Column(type: "string")] private string $postalCode; #[Column(type: "string")] private string $city; #[Column(type: "string")] private string $country; } columnPre fi x: null (default) Uses the property name, e.g. “address_street” columnPre fi x: “my_address” Uses the provided string as pre fi x, e.g. “my_address_street” columnPre fi x: false No pre fi x, so fi eld will be called “street”
33. Querying
34. Querying ORM / EntityRepository Explicit methods fi nd() fi ndAll() fi ndBy() fi ndOneBy() count() Implicit methods fi ndBy_FIELD_ fi ndOneBy_FIELD_ countBy_FIELD_
35. Querying ORM / QueryBuilder $qb = $this->entityManager->createQueryBuilder(); $qb ->select('u', 'a') ->from(CmsUser::class, 'u') ->innerJoin('u.articles', 'a', Join::ON, $qb->expr()->andX( $qb->expr()->eq('u.id', 'a.author_id'), $qb->expr()->isNotNull('u.name'), )); $this->entityManager->createQueryBuilder() ->from(CmsUser::class, 'u') ->delete(); $this->entityManager->createQueryBuilder() ->update(CmsUser::class, 'u') ->set(‘u.username', ':username');
36. Querying ORM / Transactions $em->getConnection()->beginTransaction(); try { // ... do some work $user = new User; $user->setName('George'); $em->persist($user); $em->flush(); $em->getConnection()->commit(); } catch (Exception $e) { $em->getConnection()->rollBack(); throw $e; } $conn->transactional(function($conn) { // ... do some work $user = new User; $user->setName('George'); }); Doctrine ORM already takes care of transactions. On fl ush() all inserts/updates/deletes are wrapped in a single transaction.
37. Querying ORM / Transactions
38. Querying ORM / DQL SELECT u FROM ForumUser u WHERE u.username = :name DELETE MyProject\Model\User u WHERE u.id = 4 UPDATE MyProject\Model\User u SET u.password = ‘new' WHERE u.id IN (1, 2, 3)
39. Querying Extensions
40. Querying ORM / Native SQL $sql = <<<SQL SELECT task FROM app_task_item AS task WHERE task.list_id = :list_id AND task.done = 1 SQL; $resultMapper = new ResultSetMappingBuilder($entityManager); $resultMapper->addRootEntityFromClassMetadata(TaskItem::class, ‘task’); // or: $entityRepository->createResultSetMappingBuilder(‘task’); return $entityManager->createNativeQuery($sql, $resultMapper) ->setParameter(‘list_id’, 1) ->getSingleScalarResult();
41. Hydration
42. Hydration ORM / Result Handling $dql = <<<DQL SELECT task FROM App\Entity\Task task WHERE task.list = :list_id AND task.done = true DQL; return $em->createQuery($dql) ->setParameter(‘list_id’, 1) ->getResult(); getResult List of result objects getSingleResult Result object or NoResultException or NonUniqueResultException getOneOrNullResult Result object or null or NonUniqueResultException getArrayResult Same as getResult, but returns the data as (associative) array getSingleColumnResult Result data from a single column (e.g. all id’s) as one-dimensional array getScalarResult List of scalar results getSingleScalarResult Single scalar result
43. Hydration ORM / Hydration types HYDRATE_OBJECT Hydrates result as the mapped object (entity) HYDRATE_ARRAY Hydrates result as raw data (associative array) HYDRATE_SCALAR Hydrates result as scalar (string, bool, int, fl oat, …) wrapped in array HYDRATE_SINGLE_SCALAR Hydrates result as single scalar (string, bool, int, fl oat, …) value HYDRATE_SIMPLEOBJECT Result data from a single column (e.g. all id’s) as one-dimensional array. Does not work with joined entities
44. Hydration ORM / Partial Results $dql = <<<DQL SELECT PARTIAL task.{id, name} FROM App\Entity\Task task WHERE task.list = :list_id AND task.done = true DQL; return $em->createQuery($dql) ->setParameter(‘list_id’, 1) ->getResult(); You can refresh the entity via EntityManager to fully load the data Excluded fi elds are not loaded and will not be lazy loaded
45. Hydration ORM / Custom hydration $dql = <<<DQL SELECT NEW App\Dto\CompletedListDto(list.name, COUNT(list.tasks)) FROM App\Entity\TaskList list WHERE list.id = :list_id DQL; return $em->createQuery($dql) ->setParameter(‘list_id’, 1) ->getResult();
46. Exercise Querying & custom hydration
47. Caching
48. Caching Doctrine Cache
49. Caching DoctrineBundle Con fi guration when@prod: doctrine: orm: query_cache_driver: type: pool pool: doctrine.system_cache_pool result_cache_driver: type: pool pool: doctrine.result_cache_pool framework: cache: pools: doctrine.result_cache_pool: adapter: cache.app doctrine.system_cache_pool: adapter: cache.system
50. Caching Types of Caches Query Cache Caches transformation from DQL to SQL. Highly recommended to have this enabled in production. Metadata Cache Caches class metadata, e.g. your entity mapping information. Highly recommended to have this enabled in production. Result Cache Caches query results. Should be used situationally to prevent working with stale information in your app. Also, querying the db sometimes can be faster than loading from cache. Always pro fi le to see if your optimization actually improves the runtime.
51. Events ORM / Enabling Result Caching $dql = <<<DQL SELECT task FROM App\Entity\Task task WHERE task.list = :list_id AND task.done = true DQL; return $em->createQuery($dql) ->setParameter(‘list_id’, 1) ->enableResultCache() // You can also pass lifetime and cache id here ->setResultCacheLifetime(3600) ->setResultCacheId(‘custom_id’) // By default, Doctrine assigns ID automatically ->getResult();
52. Caching Second Level Cache
53. Caching 2nd Level Cache / Usage READ_ONLY (default) • Can do reads, inserts and deletes, cannot perform updates or employ any locks. • Useful for data that is read frequently but never updated. • Best performer. • It is simple. NONSTRICT_READ_WRITE • Read Write Cache doesn’t employ any locks but can do reads, inserts, updates and deletes. • Good if the application needs to update data rarely. READ_WRITE • Read Write cache employs locks before update/delete. • Use if data needs to be updated. • Slowest strategy. • To use it the cache region implementation must support locking.
54. Caching 2nd Level Cache / Mapping #[Entity] #[Cache(usage: ‘READ_ONLY’|'NONSTRICT_READ_WRITE'|'READ_WRITE', region: ‘my_region’)] class Cart { #[Cache(usage: ‘NONSTRICT_READ_WRITE')] #[ManyToOne(targetEntity: Voucher::class)] #[JoinColumn(name: 'voucher_id', referencedColumnName: ‘id')] protected Voucher|null $voucher = null; #[Cache(usage: ‘NONSTRICT_READ_WRITE')] #[OneToMany(targetEntity: LineItem::class, mappedBy: ‘cart')] protected Collection $items; //... }
55. Events
56. Events ORM / Event types postLoad PostLoadEventArgs After entity has been loaded or refreshed, but before associations are initialized
 (accessing nested objects is not safe!) prePersist
 postPersist PrePersistEventArgs
 PostPersistEventArgs Before initial persist operation is executed After new entity is persisted (generated primary key is available) preUpdate
 postUpdate PreUpdateEventArgs PostUpdateEventArgs Before performing update operation After performing update operation preRemove
 postRemove PreRemoveEventArgs PostRemoveEventArgs Before and after removing an entity preFlush
 onFlush postFlush PreFlushEventArgs OnFlushEventArgs PostFlushEventArgs When fl ush() is called, but before any changes are calculated After calculating changes, but before they are executed (not called, if there are no changes) After changes have been made (not called, if no changes are made or errors occurred) onClear OnClearEventArgs After all references to entities are removed loadClassMetadata LoadClassMetadataEventArgs After mapping has been loaded onClassMetadataNotFound OnClassMetatadataNotFoundEventArgs When class metadata loading failed
57. Events ORM / Lifecycle Callbacks #[Entity] #[HasLifecycleCallbacks] class Product { #[PostUpdate] public function doSomething(PostUpdateEventArgs $eventArgs) { //... } }
58. Events ORM / EntityListener #[Entity] #[EntityListeners([UserListener::class])] class User { // .... }
59. Events ORM / Con fi guring EventListeners in Symfony # app/config/services.yaml services: App\EventListener\DoctrinePostPersistListener: tags: - name: ‘doctrine.event_listener’ event: ‘postPersist’ priority: 500 connection: default
60. Exercise Ensure TaskList.lastUpdated is triggered when an item is created or updated
61. Locking
62. Locking Prevent inconsistencies from competing requests User A User B Change item name to A Change item name to B !
63. Locking ORM / Supported locking modes
64. Locking ORM / Optimistic vs. pessimistic locking Optimistic Locking Pessimistic Locking Scope Business Transactions spanning multiple requests involving multiple transactions Row-level lock acquired on database level based on ANSI-SQL. How? By adding a version fi eld to the entity, updates on “older” versions can be blocked • Pessimistic Write (Doctrine\DBAL\LockMode::PESSIMISTIC_WRITE), locks the underlying database rows for concurrent Read and Write Operations. • Pessimistic Read (Doctrine\DBAL\LockMode::PESSIMISTIC_READ), locks other concurrent requests that attempt to update or lock rows in write mode. On Failure? An Exception is thrown. Business process can decide whether to discard the change or reconcile the planned changes with the current version Exception when acquiring lock without transaction otherwise see your database for details on how l Implementation notes You are responsible for keeping the version between requests, e.g. store it in a session or hidden fi eld in a form. Requires to disable auto-commit and use explicit transaction boundaries around the lock use case.
65. Exercise Use optimistic locking to ensure users can not accidentally delete a list which was modi fi ed by a di ff erent user
66. Batch Processing
67. Batch Processing ORM / Best Practices
68. Batch Processing ORM / General Tips What? Why? Avoid using ORM and fall back to DBAL when possible Object Hydration usually is one of the bigger costs and mass im-/export seldom requires handling entity- objects and instead just push data. Disable SQL Logger For batch operations disabling the SQL logger can improve performance. This can leave a blind spot for debugging, but you might want to use your DB tools for that instead. Perform inserts in bulk Perform bulk inserts (e.g. fl ush every 100 entries). You can use pro fi lers like Black fi re or tideways to fi nd a sweet spot between memory usage and query throughput. When reading large datasets, iterate instead of fetching all data at once. Prevents excessive memory usage. Split process into multiple partitioned processes Requires some additional work and can become tricky for more complex scenarios, but can speed up import using divide & conquer paradigm
69. Batch Processing Partitioning For reference: https://speakerdeck.com/dbrumann/how-to-write-well-designed-imports-with-symfony
70. Migrations
71. Migrations
72. Migrations Migration-class <?php declare(strict_types=1); namespace DoctrineMigrations; use Doctrine\DBAL\Schema\Schema; use Doctrine\Migrations\AbstractMigration; /** * Auto-generated Migration: Please modify to your needs! */ final class Version20251125150507 extends AbstractMigration { public function getDescription(): string { return ''; } public function up(Schema $schema): void { // this up() migration is auto-generated, please modify it to your needs } public function down(Schema $schema): void { // this down() migration is auto-generated, please modify it to your needs } }
73. Migrations Migration-class /** * Indicates the transactional mode of this migration. * * If this function returns true (default) the migration will be executed * in one transaction, otherwise non-transactional state will be used to * execute each of the migration SQLs. * * Extending class should override this function to alter the return value. */ public function isTransactional(): bool public function preUp(Schema $schema): void public function postUp(Schema $schema): void public function preDown(Schema $schema): void public function postDown(Schema $schema): void
74. Migrations Migration-class public function warnIf(bool $condition, string $message = 'Unknown Reason'): void public function abortIf(bool $condition, string $message = 'Unknown Reason'): void public function skipIf(bool $condition, string $message = 'Unknown Reason'): void public function up(Schema $schema): void { $this->warnIf(true, ‘Something looks off here’); }
75. Migrations Mark migrations as irreversible public function down(Schema $schema): void { $this->throwIrreversibleException(); }
76. Migrations Rollup • Delete all your migrations in your local environment • Create a new “start” migration, e.g. by dumping the current schema • Release version with the reset migration • During deployment do not perform automatic migrations • Instead, perform the doctrine:migrations:rollup command Rollup does not remove old migrations or dump the current schema. It “just” takes care of cleaning up the versions table.
77. Testing
78. Exercise Introduce zenstruck/foundry and dama/doctrine-test-bundle