Mastering Postgres Replication Slots: Preventing WAL Bloat and Other Production Issues (JavaZone 2025)

With logical replication, Postgres provides a powerful interface for extracting data change events from the transaction log, as for instance utilized by Debezium. Replication slots are Postgres' mechanism for making sure that any segments of the write-ahead log (WAL) of the database are kept around until they have been processed by registered replication consumers.

But there be dragons: When not being careful, a replication slot may cause unduly large amounts of WAL segments to be retained by the database. Join us for this lightning talk to learn about best practices helping to prevent this and other issues, discussing aspects like heartbeats, replication slot failover, monitoring, the management of Postgres publications, and more. While this is primarily based on experience gained from using replication slots via Debezium, the principles are generally applicable and are worth considering also when using other change data capture (CDC) tools for Postgres based on logical replication.

Transcript

1. Gunnar Morling @gunnarmorling.dev Mastering Postgres Replication Slots: Preventing WAL Bloat and Other Production Issues Megan Coughlin https://flic.kr/p/iggCAx (CC BY-ND 2.0) 🦋
2. #PostgresReplicationSlots · @gunnarmorling.dev "Help, my database is running out of disk space!!1!"
3. #PostgresReplicationSlots · @gunnarmorling.dev • Technologist at Confluent • Former project lead of Debezium • kcctl 🧸, JfrUnit, ModiTect, MapStruct • One Billion Row Challenge 1⃣🐝🏎 • Java Champion Gunnar Morling
4. #PostgresReplicationSlots · @gunnarmorling.dev Logical Replication Fine-grained Data Replication Between Databases
5. #PostgresReplicationSlots · @gunnarmorling.dev Logical Replication Fine-grained Data Replication Between Systems
6. #PostgresReplicationSlots · @gunnarmorling.dev Replication Slots Keeping Track of Consumer Progress
7. #PostgresReplicationSlots · @gunnarmorling.dev Replication Slots Keeping Track of Consumer Progress
8. #PostgresReplicationSlots · @gunnarmorling.dev Replication Slots Keeping Track of Consumer Progress
9. #PostgresReplicationSlots · @gunnarmorling.dev Choosing a Logical Decoding Plug-In Use pgoutput • Efficient binary representation • Available OOTB on Postgres 10+ • Fine-grained control over contents via publications
10. #PostgresReplicationSlots · @gunnarmorling.dev Examining pgoutput Data Change Events Use pgoutput-cli • Can’t use SQL interface (pg_logical_slot_peek_changes() etc.)
11. #PostgresReplicationSlots · @gunnarmorling.dev Examining pgoutput Data Change Events
12. #PostgresReplicationSlots · @gunnarmorling.dev Define Maximum Retained WAL Size • Rather cut loose consumers than kill the DB • Since Postgres 13:
13. #PostgresReplicationSlots · @gunnarmorling.dev Enable Heartbeats The Problem: WAL Global, Slots Per Database
14. #PostgresReplicationSlots · @gunnarmorling.dev Enable Heartbeats The Solution: “Artificial” Traffic
15. #PostgresReplicationSlots · @gunnarmorling.dev Enable Heartbeats The Solution: “Artificial” Traffic • With Debezium:
16. #PostgresReplicationSlots · @gunnarmorling.dev Use Fine-Grained Publications
17. #PostgresReplicationSlots · @gunnarmorling.dev Use Fine-Grained Publications
18. #PostgresReplicationSlots · @gunnarmorling.dev Monitor, Monitor, Monitor!
19. #PostgresReplicationSlots · @gunnarmorling.dev Monitor, Monitor, Monitor!
20. #PostgresReplicationSlots · @gunnarmorling.dev Drop Unused Replication Slots
21. #PostgresReplicationSlots · @gunnarmorling.dev "Help, my database is running out of disk space!!1!"
22. #PostgresReplicationSlots · @gunnarmorling.dev "Help, my database is running out of disk space!!1!" ❌
23. #PostgresReplicationSlots · @gunnarmorling.dev Learning More Online https://www.morling.dev/blog/mastering-postgres-replication-slots/
24. #PostgresReplicationSlots · @gunnarmorling.dev Learning More… …at JavaZone 📝 “Ins and Outs of The Outbox Pattern” 🗓 Thursday 10:20 AM 📍 Room VII
25. #PostgresReplicationSlots · @gunnarmorling.dev gmorling@confluent.io @gunnarmorling @gunnarmorling.dev morling.dev 📧 Get In Touch