Distributed Scheduling com Spring Boot: os desafios e percalços de implementar um job em background

Cedo ou tarde um desenvolvedor(a) implementará seu primeiro job em background, o que geralmente é uma tarefa simples para maioria dos sistemas pode se tornar um pesadelo em sistemas que precisam lidar com grande volumetria de dados, alta performance e paralelismo. Cenários como esses escondem diversos problemas na qual a maioria dos desenvolvedores não está acostumado, como volume de dados, falhas na rede, reprocessamento indevido, sobrecarga no banco de dados, erros de estouro de memória até indisponibilidade de todo o sistema.

Lidar com boa parte desses problemas não requer tecnologias e serviços da moda (hype), mas sim fundamentos sólidos em sistemas distribuídos. Nessa talk, pretendo apresentar como um desenvolvedor(a) experiente implementa um job em background levando em conta os principais percalços e desenha uma solução para escala horizontal ao mesmo tempo em que tira proveito das tecnologias adotadas pelo time de desenvolvimento.

Se você acredita que um job em background é uma tarefa simples, então essa talk é para você!

(GRAVAÇÃO: https://youtu.be/I_kEO_HPfBU?list=PLHMMERsvy9EyWQPru4SrJAYHEGKfkjRgP&t=130)

1. DISTRIBUTED SCHEDULING | Spring Boot os desa fi os e percalços de implementar um job em background
3. Maaaaas implementar um job em background…
5. https://deniseyu.io/art/
6. Computação paralela e distribuída
7. Cadê meu cartão?
11. db servidor
12. db servidor
13. db servidor
14. db servidor
15. db servidor
16. db servidor
17. db servidor
18. db servidor
19. db servidor
20. db servidor
21. Como agendar tarefas com Spring Boot?
22. @Component public class OneJob { @Scheduled(fixedDelay = 60_000) public void runQuiteOften() { // lógica do job vai aqui } }
23. @Component public class OneJob { @Scheduled(fixedDelay = 60_000) public void runQuiteOften() { // lógica do job vai aqui } }
24. @Component public class OneJob { @Scheduled(fixedDelay = 60_000) public void runQuiteOften() { // lógica do job vai aqui } }
25. @Component public class OneJob { @Scheduled(fixedDelay = 60_000) public void runQuiteOften() { // lógica do job vai aqui } }
26. @Component public class OneJob { public void runQuiteOften() { // lógica do job vai aqui } }
27. @Component public class OneJob { @Scheduled public void runQuiteOften() { // lógica do job vai aqui } }
28. @Component public class OneJob { @Scheduled(fixedDelay = 60_000) public void runQuiteOften() { // lógica do job vai aqui } }
29. @Component public class OneJob { @Scheduled(fixedDelay = 60_000) public void runQuiteOften() { // lógica do job vai aqui } }
30. E nossa tarefa?
31. db servidor
32. db servidor
33. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
34. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
35. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
36. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
37. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
38. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
39. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
40. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
41. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
42. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
43. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
44. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
45. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
46. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
47. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
48. Solução Ingênua
49. Rafael Ponte @rponte
51. Fortaleza - Terra do Sol
53. DISTRIBUTED SCHEDULING | Spring Boot os desa fi os e percalços de implementar um job em background
54. Solução Ingênua
55. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
56. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
57. select p.* from proposal p where p.status = 'ELIGIBLE' order by p.created_at asc
58. E SE…
59. …volume de dados crescer?
62. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
63. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
64. @Repository public interface ProposalRepository extends JpaRepository<Proposal, UUID> { public List<Proposal> findAllByStatusOrderByCreatedAtAsc(ProposalStatus status); }
65. @Repository public interface ProposalRepository extends JpaRepository<Proposal, UUID> { public List<Proposal> findAllByStatusOrderByCreatedAtAsc(ProposalStatus status); }
66. @Repository public interface ProposalRepository extends JpaRepository<Proposal, UUID> { public List<Proposal> findAllByStatusOrderByCreatedAtAsc(ProposalStatus status); }
67. select p.* from proposal p where p.status = 'ELIGIBLE' order by p.created_at asc
68. List<Proposal> findAllByStatusOrderByCreatedAtAsc(...);
69. List<Proposal> findAllByStatusOrderByCreatedAtAsc(...);
70. List<Proposal> findTop50ByStatusOrderByCreatedAtAsc(...);
71. select p.* from proposal p where p.status = 'ELIGIBLE' order by p.created_at asc
72. select p.* from proposal p where p.status = 'ELIGIBLE' order by p.created_at asc limit 50
73. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
74. Mas…
75. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
76. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } o loop vai encerrar muito CEDO !
77. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } Só vai rodar no próximo agendamento!
78. while(true)
79. @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); proposals.forEach(proposal -> { // processa cada proposta }); }
80. @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { // processa cada proposta }); } }
81. @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { // processa cada proposta }); } }
82. @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { // processa cada proposta }); } }
83. @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { // processa cada proposta }); } }
84. @Component public class AttachCardsToProposalsJob { // dependências e constantes @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } }
85. Mas ao rodar…
87. Hein?
88. @Component public class AttachCardsToProposalsJob { // dependências e constantes @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } }
89. @Component public class AttachCardsToProposalsJob { // dependências e constantes @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } } - carregando apenas 50 objetos por vez
90. @Component public class AttachCardsToProposalsJob { // dependências e constantes @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } } - criando nova referência - carregando apenas 50 objetos por vez
91. @Component public class AttachCardsToProposalsJob { // dependências e constantes @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } } - criando nova referência - carregando apenas 50 objetos por vez - saindo do loop na hora certa
92. Por que?
93. @Component public class AttachCardsToProposalsJob { // dependências e constantes @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } }
94. @Repository public interface ProposalRepository extends JpaRepository<Proposal, UUID> { public List<Proposal> findAllByStatusOrderByCreatedAtAsc(ProposalStatus status); }
95. Revisitando JPA e Hibernate
96. Job
97. Repository Job
98. Repository EM Job
99. Repository EM Job
100. Repository EM Job
101. Repository EM Job
102. Repository EM Job
103. Repository EM Job
104. Repository EM Job
105. Long-running transaction
106. @Transactional Repository EM Job Long-running transaction
107. @Transactional Repository EM Job Long-running transaction
108. @Transactional Repository EM Job Long-running transaction
109. @Transactional Repository EM Job (First-Level Cache) Long-running transaction
110. @Transactional Repository EM Job (First-Level Cache) Long-running transaction
111. A sacada aqui é entender…
112. @Transactional Repository EM Job (First-Level Cache) Long-running transaction
113. @Transactional EM (First-Level Cache) Long-running transaction
114. @Transactional EM (First-Level Cache) Long-running transaction
115. EM (First-Level Cache) @Transactional Long-running transaction
116. EM (First-Level Cache) @Transactional Long-running transaction
117. Repository EM Job (First-Level Cache) @Transactional Long-running transaction
118. Repository EM Job (First-Level Cache) Long-running transaction
119. Ahhh!!
120. @Component public class AttachCardsToProposalsJob { // dependências e constantes @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } }
121. @Component public class AttachCardsToProposalsJob { // dependências e constantes @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } } inicia e termina uma transação
122. Mas…
123. @Component public class AttachCardsToProposalsJob { // dependências e constantes @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } }
124. @Component public class AttachCardsToProposalsJob { // dependências e constantes @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); // inicia e comita transação proposal.attachTo(newCard); proposalRepository.save(proposal); // inicia e comita transação }); } } } analogo a 
 auto_commit = true
125. @Component public class AttachCardsToProposalsJob { // dependências e constantes @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); // inicia e comita transação proposal.attachTo(newCard); proposalRepository.save(proposal); // inicia e comita transação }); } } } Abrimos mão da CONSISTÊNCIA dos dados!
126. @Component
127. Short-running transaction
128. Job Transaction Job
129. Job Transaction Job inicio do job fi m do job
130. Job Transaction Job
131. inicio da transação fi m da transação Job Job Transaction
132. Job Transaction Job Long-running transaction
133. Job Transaction Job
134. Job Transaction Job
135. Job Transaction Job inicio da transação fi m da transação
136. Job Transaction Job Short-running transaction
137. Job Transaction Job
138. Job Transaction Job
139. Job Transaction Job
140. Job Transaction Job
141. Job Transaction Job
142. Job Transaction Job Short-running transaction
143. Repository EM Job (First-Level Cache) Short-running transaction
144. Repository EM Job @Transactional Short-running transaction (First-Level Cache)
145. Repository EM Job @Transactional Short-running transaction (First-Level Cache)
146. Repository EM Job @Transactional Short-running transaction (First-Level Cache)
147. @Transactional Repository EM Job (First-Level Cache) Short-running transaction
148. Repository EM Job @Transactional Short-running transaction (First-Level Cache)
149. Controle transacional programático
150. @Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { // executa código em escopo transacional } } }
151. @Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { @Transactional.execute(transaction -> { // executa código em escopo transacional }); } } }
152. @Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { @Transactional.execute(transaction -> { // executa código em escopo transacional }); } } }
153. @Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { @Transactional.execute(transaction -> { // executa código em escopo transacional }); } } }
154. @Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { @Transactional.execute(transaction -> { // executa código em escopo transacional }); } } }
155. @Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { @Transactional.execute(transaction -> { // executa código em escopo transacional }); } } }
156. @Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { @Transactional.execute(transaction -> { // executa código em escopo transacional }); } } }
157. @Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { @Transactional.execute(transaction -> { // executa código em escopo transacional }); } } }
158. @Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { transactionManager.execute(transaction -> { // executa código em escopo transacional }); } } }
159. @Component public class AttachCardsToProposalsJob { private TransactionTemplate transactionManager; @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { transactionManager.execute(transaction -> { // executa código em escopo transacional }); } } }
160. @Component public class AttachCardsToProposalsJob { private TransactionTemplate transactionManager; @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { transactionManager.execute(transaction -> { // executa código em escopo transacional }); } } }
161. @Autowired private TransactionTemplate transactionManager; @Scheduled(fixedDelay = 60_000) public void execute() { boolean pending = true; while (pending) { pending = transactionManager.execute(transactionStatus -> { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }
162. @Autowired private TransactionTemplate transactionManager; @Scheduled(fixedDelay = 60_000) public void execute() { boolean pending = true; while (pending) { pending = transactionManager.execute(transactionStatus -> { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }
164. inicio do job fi m do job
165. E SE…
166. …rodarmos a aplicação em cluster?
171. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE 1004 ELIGIBLE
172. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE 1004 ELIGIBLE
173. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE 1004 ELIGIBLE
174. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE 1003 ELIGIBLE 1004 ELIGIBLE
175. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE 1003 ELIGIBLE 1004 ELIGIBLE
176. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE 1004 ELIGIBLE
177. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE 1004 ELIGIBLE
178. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE_WITH_ATTACHED_CARD 1004 ELIGIBLE
179. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE_WITH_ATTACHED_CARD 1004 ELIGIBLE
180. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE_WITH_ATTACHED_CARD 1004 ELIGIBLE_WITH_ATTACHED_CARD
181. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE_WITH_ATTACHED_CARD 1004 ELIGIBLE_WITH_ATTACHED_CARD
183. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE 1004 ELIGIBLE
184. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE 1004 ELIGIBLE
185. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE 1003 ELIGIBLE 1004 ELIGIBLE
186. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE 1003 ELIGIBLE 1004 ELIGIBLE 2x
187. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE 1004 ELIGIBLE 2x
188. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE 1004 ELIGIBLE 2x 2x
189. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE_WITH_ATTACHED_CARD 1004 ELIGIBLE 2x 2x
190. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE_WITH_ATTACHED_CARD 1004 ELIGIBLE 2x 2x 2x
191. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE_WITH_ATTACHED_CARD 1004 ELIGIBLE_WITH_ATTACHED_CARD 2x 2x 2x
192. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE_WITH_ATTACHED_CARD 1004 ELIGIBLE_WITH_ATTACHED_CARD 2x 2x 2x 2x
193. Oops!
194. Synchronized?
195. @Component public class AttachCardsToProposalsJob { @Scheduled(...) public synchronized void execute() { // executa lógica aqui } }
200. @Component
201. Distributed Lock?
202. @Scheduled(...) public void execute() { if (isLeader()) { // executa lógica aqui } }
203. @Scheduled(...) public void execute() { if (isLeader()) { // executa lógica aqui } }
215. Não!
216. Que tal…
219. @Repository public interface ProposalRepository extends JpaRepository<Proposal, UUID> { public List<Proposal> findTop50ByStatusOrderByCreatedAtAsc(ProposalStatus status); }
220. @Repository public interface ProposalRepository extends JpaRepository<Proposal, UUID> { @Lock(LockModeType.PESSIMISTIC_WRITE) public List<Proposal> findTop50ByStatusOrderByCreatedAtAsc(ProposalStatus status); }
221. select p.* from proposal p where p.status = 'ELIGIBLE' order by p.created_at asc limit 50 

222. select p.* from proposal p where p.status = 'ELIGIBLE' order by p.created_at asc limit 50 
 for update
223. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE ... ...
224. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE ... ...
225. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE ... ...
226. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE ... ...
227. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE 1003 ELIGIBLE ... ...
228. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE 1003 ELIGIBLE ... ...
229. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE 1003 ELIGIBLE ... ...
230. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE 1003 ELIGIBLE ... ...
231. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE 1003 ELIGIBLE ... ...
232. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE 1003 ELIGIBLE ... ...
233. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...
234. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...
235. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...
236. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...
237. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE_WITH_ATTACHED_CARD ... ...
238. SIM!
239. E SE…
240. …precisarmos processar em paralelo?
245. Para isso…
246. 9630 9631 QUEUE ( fi la) 9629
247. 9630 9631 QUEUE ( fi la) producer 9629 produz: 
 1000 itens/min
248. 9630 9631 QUEUE ( fi la) producer consumer 9629 produz: 
 1000 itens/min consume: 
 600 itens/min
249. 9630 9631 QUEUE ( fi la) producer consumer 9629 produz: 
 1000 itens/min consumer consume: 
 400 itens/min consume: 
 600 itens/min
250. 9630 9631 QUEUE ( fi la) producer consumer 9629 produz: 
 1000 itens/min consumer consume: 
 400 itens/min consume: 
 600 itens/min
252. Não!
253. Que tal…
255. @Repository public interface ProposalRepository extends JpaRepository<Proposal, UUID> { @QueryHints({ @QueryHint( name = "javax.persistence.lock.timeout", value = LockOptions.SKIP_LOCKED) // org.hibernate.LockOptions }) @Lock(LockModeType.PESSIMISTIC_WRITE) public List<Proposal> findTop50ByStatusOrderByCreatedAtAsc(ProposalStatus status); }
256. @Repository public interface ProposalRepository extends JpaRepository<Proposal, UUID> { @QueryHints({ @QueryHint( name = "javax.persistence.lock.timeout", value = LockOptions.SKIP_LOCKED) // org.hibernate.LockOptions }) @Lock(LockModeType.PESSIMISTIC_WRITE) public List<Proposal> findTop50ByStatusOrderByCreatedAtAsc(ProposalStatus status); }
257. select p.* from proposal p where p.status = 'ELIGIBLE' order by p.created_at asc limit 50 
 for update
258. select p.* from proposal p where p.status = 'ELIGIBLE' order by p.created_at asc limit 50 
 for update skip locked
259. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE ... ...
260. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE ... ...
261. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE ... ...
262. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE ... ...
263. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE ... ...
264. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE ... ...
265. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE ... ...
266. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...
267. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...
268. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...
269. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...
270. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...
271. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...
272. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...
273. ID STATUS 1001 ELIGIBLE_WITH_ATTACHED_CARD 1002 ELIGIBLE_WITH_ATTACHED_CARD 1003 ELIGIBLE ... ...
274. Ou seja…
275. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE 1004 ELIGIBLE
276. ID STATUS 1001 ELIGIBLE 1002 ELIGIBLE 1003 ELIGIBLE 1004 ELIGIBLE
277. SIM!
278. E SE…
279. …impactar no startup da aplicação?
280. @Scheduled( fixedDelay = 60_000 ) public void execute() { // executa lógica aqui }
281. initialDelay
282. @Scheduled( fixedDelay = 60_000, initialDelay = 120_000 ) public void execute() { // executa lógica aqui }
283. E SE…
284. …precisarmos de maior throughput?
285. Hibernate batch size
286. @Autowired private TransactionTemplate transactionManager; @Scheduled(fixedDelay = 60_000) public void execute() { boolean pending = true; while (pending) { pending = transactionManager.execute(transactionStatus -> { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }
287. @Autowired private TransactionTemplate transactionManager; @Scheduled(fixedDelay = 60_000) public void execute() { boolean pending = true; while (pending) { pending = transactionManager.execute(transactionStatus -> { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }
288. @Autowired private TransactionTemplate transactionManager; @Scheduled(fixedDelay = 60_000) public void execute() { boolean pending = true; while (pending) { pending = transactionManager.execute(transactionStatus -> { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }
289. @Autowired private TransactionTemplate transactionManager; @Scheduled(fixedDelay = 60_000) public void execute() { boolean pending = true; while (pending) { pending = transactionManager.execute(transactionStatus -> { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }
290. @Autowired private TransactionTemplate transactionManager; @Scheduled(fixedDelay = 60_000) public void execute() { boolean pending = true; while (pending) { pending = transactionManager.execute(transactionStatus -> { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } } Insere novo cartão Atualiza proposta com cartão
291. Para cada iteração do loop…
292. 2 roundtrips INSERT INTO card(cardNumber) VALUES ('5260-9991-9040-0001'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2021;
293. 2 roundtrips INSERT INTO card(cardNumber) VALUES ('5260-9991-9040-0001'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2021;
294. 2 roundtrips INSERT INTO card(cardNumber) VALUES ('5260-9991-9040-0001'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2021;
295. INSERT INTO card(cardNumber) VALUES ('5260-9991-9040-0001'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2021; INSERT INTO card(cardNumber) VALUES (' 5513-8141-9261-0002'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2022; 4 roundtrips
296. INSERT INTO card(cardNumber) VALUES ('5260-9991-9040-0001'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2021; INSERT INTO card(cardNumber) VALUES (' 5513-8141-9261-0002'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2022; INSERT INTO card(cardNumber) VALUES (' 5599-4154-6354-0003'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2023; 6 roundtrips
297. INSERT INTO card(cardNumber) VALUES ('5260-9991-9040-0001'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2021; INSERT INTO card(cardNumber) VALUES (' 5513-8141-9261-0002'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2022; INSERT INTO card(cardNumber) VALUES (' 5599-4154-6354-0003'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2023; INSERT INTO card(cardNumber) VALUES (' 5419-8753-1126-0004'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2024; INSERT INTO ... UPDATE proposal SET ... 100 roundtrips
298. Latência
299. spring: jpa: properties: hibernate: jdbc: batch_size: 50 order_inserts: true order_updates: true application.yml
300. spring: jpa: properties: hibernate: jdbc: batch_size: 50 order_inserts: true order_updates: true application.yml
301. 2 roundtrips INSERT INTO card(cardNumber) VALUES (?) Params: [ 
 ('5513-8141-9261-0001'), 
 ('5513-8141-9261-0002'), 
 (‘5513-8141-9261-0003'), (‘5513-8141-9261-0004'), ... 
 ] UPDATE proposal SET status = ? WHERE id = ?; Params: [ 
 (‘ELIGIBLE_WITH_ATTACHED_CARD’, 2021), 
 ('ELIGIBLE_WITH_ATTACHED_CARD', 2022), 
 ('ELIGIBLE_WITH_ATTACHED_CARD', 2023), ('ELIGIBLE_WITH_ATTACHED_CARD', 2024), ... 
 ]
302. 2 roundtrips INSERT INTO card(cardNumber) VALUES (?) Params: [ 
 ('5513-8141-9261-0001'), 
 ('5513-8141-9261-0002'), 
 (‘5513-8141-9261-0003'), (‘5513-8141-9261-0004'), ... 
 ] UPDATE proposal SET status = ? WHERE id = ?; Params: [ 
 (‘ELIGIBLE_WITH_ATTACHED_CARD’, 2021), 
 ('ELIGIBLE_WITH_ATTACHED_CARD', 2022), 
 ('ELIGIBLE_WITH_ATTACHED_CARD', 2023), ('ELIGIBLE_WITH_ATTACHED_CARD', 2024), ... 
 ]
303. INSERT INTO card(cardNumber) VALUES (?) Params: [ 
 ('5513-8141-9261-0001'), 
 ('5513-8141-9261-0002'), 
 (‘5513-8141-9261-0003'), (‘5513-8141-9261-0004'), ... 
 ] 2 roundtrips UPDATE proposal SET status = ? WHERE id = ?; Params: [ 
 (‘ELIGIBLE_WITH_ATTACHED_CARD’, 2021), 
 ('ELIGIBLE_WITH_ATTACHED_CARD', 2022), 
 ('ELIGIBLE_WITH_ATTACHED_CARD', 2023), ('ELIGIBLE_WITH_ATTACHED_CARD', 2024), ... 
 ]
304. E SE…
305. …ocorrer erro na integração?
306. @Scheduled(...) public void execute() { boolean pending = true; while (pending) { pending = transactionManager.execute(txStatus -> { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }
307. @Scheduled(...) public void execute() { boolean pending = true; while (pending) { pending = transactionManager.execute(txStatus -> { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }
308. @Scheduled(...) public void execute() { boolean pending = true; while (pending) { pending = transactionManager.execute(txStatus -> { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }
309. https://deniseyu.io/art/
310. CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId());
311. CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); espera in fi nita? timeout
312. CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); espera in fi nita? timeout erros intermitentes? retry
313. CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); espera in fi nita? timeout erros intermitentes? retry posso reenviar a requisição? idempotência
314. E SE… E SE… E SE… E SE… E SE… E SE… E SE… E SE… E SE… E SE… E SE… E SE E SE… SE… E SE… E SE… E SE… E SE… E SE… E SE… SE… E SE…
315. CONCLUINDO
316. @rponte Rafael Ponte