Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Distributed Scheduling with Spring Boot: the ch...

Distributed Scheduling with Spring Boot: the challenges and pitfalls of implementing a background job

(🎥 RECORDING: https://www.youtube.com/watch?v=ghpljMg8Ecc)

Sooner or later a developer will implement his/her first background job using Java and Spring Boot, and what usually is a simple task for the majority of systems might become a nightmare in scenarios that need to deal with high performance, parallelism, distributed systems and a large volume of data. Scenarios like those hide several issues which many developers are not used to, such as large volumes of data, network failures, data inconsistency, out-of-memory errors and even taking the whole system down.

Although it seems controversial, dealing with many of these problems does not require hype technologies or services, but solid distributed systems fundamentals. This talk will present how an experienced developer implements a background job with Java and Spring Boot taking into consideration the main challenges and pitfalls it brings along, and how he/she designs a solution for high-performance, resilience and horizontal scalability at the same time he/she takes advantage of many modules of Spring Boot, Hibernate and the relational database.

If you still believe that a background job is a simple task, so this talk is for you!

Rafael Ponte

May 30, 2024
Tweet

More Decks by Rafael Ponte

Other Decks in Programming

Transcript

  1. DISTRIBUTED SCHEDULING | Spring Boot the challenges and pitfalls of

    implementing a background job Rafael Ponte | @rponte
  2. @Component public class OneJob { @Scheduled(fixedDelay = 60_000) public void

    runQuiteOften() { // lógica do job vai aqui } }
  3. @Component public class OneJob { @Scheduled(fixedDelay = 60_000) public void

    runQuiteOften() { // lógica do job vai aqui } }
  4. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  5. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  6. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  7. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  8. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  9. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  10. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  11. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  12. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  13. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  14. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  15. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  16. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  17. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  18. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } Naive Solution
  19. @rponte Rafael Ponte Brazilian Software Engineer at aka Prince of

    the Ocean aka Maharaja of the Legacies 19+ years of experience working with Java and distributed systems Loves working with "boring techs"
  20. DISTRIBUTED SCHEDULING | Spring Boot the challenges and pitfalls of

    implementing a background job Rafael Ponte | @rponte
  21. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  22. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  23. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  24. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  25. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  26. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  27. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  28. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  29. @Scheduled(…) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderBy…; proposals.forEach(proposal

    -> { CardDataResponse cardData = cardsClient.findCardByProposalId(…); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); }
  30. @Scheduled(…) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderBy…; proposals.forEach(proposal

    -> { CardDataResponse cardData = cardsClient.findCardByProposalId(…); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); }
  31. @Scheduled(…) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderBy…; proposals.forEach(proposal

    -> { CardDataResponse cardData = cardsClient.findCardByProposalId(…); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } Inserts a new card Updates a proposal with its card
  32. -- Inserts a new credit card INSERT INTO card (id,

    cardNumber) VALUES (42, ’5260-9991-9040-0001’); -- Updates the proposal with its credit card UPDATE proposal SET card_id = 42, status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 7031;
  33. -- Inserts a new credit card INSERT INTO card (id,

    cardNumber) VALUES (42, ’5260-9991-9040-0001’); -- Updates the proposal with its credit card UPDATE proposal SET card_id = 42, status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 7031;
  34. -- Inserts a new credit card INSERT INTO card (id,

    cardNumber) VALUES (42, ’5260-9991-9040-0001’); -- Updates the proposal with its credit card UPDATE proposal SET card_id = 42, status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 7031;
  35. @Scheduled(…) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderBy…; proposals.forEach(proposal

    -> { CardDataResponse cardData = cardsClient.findCardByProposalId(…); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); }
  36. @Scheduled(…) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderBy…; proposals.forEach(proposal

    -> { CardDataResponse cardData = cardsClient.findCardByProposalId(…); Card newCard = cardData.toModel(); cardRepository.save(newCard); // begins and commits a transaction proposal.attachTo(newCard); proposalRepository.save(proposal); // begins and commits a transaction }); }
  37. @Scheduled(…) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderBy…; proposals.forEach(proposal

    -> { CardDataResponse cardData = cardsClient.findCardByProposalId(…); Card newCard = cardData.toModel(); cardRepository.save(newCard); // begins and commits a transaction proposal.attachTo(newCard); proposalRepository.save(proposal); // begins and commits a transaction }); } it’s similar to auto_commit = true😱
  38. @Scheduled(…) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderBy…; proposals.forEach(proposal

    -> { CardDataResponse cardData = cardsClient.findCardByProposalId(…); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); }
  39. @Scheduled(…) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderBy…; proposals.forEach(proposal

    -> { CardDataResponse cardData = cardsClient.findCardByProposalId(…); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); }
  40. @Scheduled(…) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderBy…; proposals.forEach(proposal

    -> { CardDataResponse cardData = cardsClient.findCardByProposalId(…); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); }
  41. @Scheduled(…) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderBy…; proposals.forEach(proposal

    -> { CardDataResponse cardData = cardsClient.findCardByProposalId(…); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); }
  42. @Scheduled(…) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderBy…; proposals.forEach(proposal

    -> { CardDataResponse cardData = cardsClient.findCardByProposalId(…); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); }
  43. ID STATUS CARD_ID 7028 NOT_ELIGIBLE 39 7029 NOT_ELIGIBLE 40 7030

    ELIGIBLE_WITH_ATTACHED_CARD 41 7031 ELIGIBLE <null> 7032 ELIGIBLE_WITH_ATTACHED_CARD 96 ... ... ... ID CARD_NUMBER 40 5260-9991-9040-0040 41 5260-9991-9040-0041 42 5260-9991-9040-00042 Proposal Card Database
  44. ID STATUS CARD_ID 7028 NOT_ELIGIBLE 39 7029 NOT_ELIGIBLE 40 7030

    ELIGIBLE_WITH_ATTACHED_CARD 41 7031 ELIGIBLE <null> 7032 ELIGIBLE_WITH_ATTACHED_CARD 96 ... ... ... ID CARD_NUMBER 40 5260-9991-9040-0040 41 5260-9991-9040-0041 42 5260-9991-9040-00042 Proposal Card Database
  45. ID STATUS CARD_ID 7028 NOT_ELIGIBLE 39 7029 NOT_ELIGIBLE 40 7030

    ELIGIBLE_WITH_ATTACHED_CARD 41 7031 ELIGIBLE <null> 7032 ELIGIBLE_WITH_ATTACHED_CARD 96 ... ... ... ID CARD_NUMBER 40 5260-9991-9040-0040 41 5260-9991-9040-0041 42 5260-9991-9040-00042 Proposal Card Database
  46. ID STATUS CARD_ID 7028 NOT_ELIGIBLE 39 7029 NOT_ELIGIBLE 40 7030

    ELIGIBLE_WITH_ATTACHED_CARD 41 7031 ELIGIBLE <null> 7032 ELIGIBLE_WITH_ATTACHED_CARD 96 ... ... ... ID CARD_NUMBER 40 5260-9991-9040-0040 41 5260-9991-9040-0041 42 5260-9991-9040-00042 Proposal Card Database
  47. Runs every 1 minute ID COL1 COL2 1 2 3

    4 ID COL1 COL2 1 2 3 4 Data corruption error
  48. Runs every 1 minute Runs every 5 minutes ID COL1

    COL2 1 2 3 4 ID COL1 COL2 1 2 3 4 ID COL1 COL2 1 2 3 4 ID COL1 COL2 1 2 3 4 Data corruption error Fix corrupted data
  49. Runs every 1 minute Runs every 5 minute ID COL1

    COL2 1 2 3 4 ID COL1 COL2 1 2 3 4 ID COL1 COL2 1 2 3 4 ID COL1 COL2 1 2 3 4 Data corruption error Fix corrupted data No!
  50. @Scheduled(…) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderBy…; proposals.forEach(proposal

    -> { CardDataResponse cardData = cardsClient.findCardByProposalId(…); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); }
  51. @Scheduled(…) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderBy…; proposals.forEach(proposal

    -> { CardDataResponse cardData = cardsClient.findCardByProposalId(…); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); }
  52. @Transactional @Scheduled(…) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderBy…;

    proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(…); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); }
  53. @Transactional @Scheduled(…) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderBy…;

    proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(…); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); }
  54. @Transactional @Scheduled(…) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderBy…;

    proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(…); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); }
  55. @Transactional @Scheduled(…) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderBy…;

    proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(…); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } You got Recoverability!
  56. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  57. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  58. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  59. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findAllByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  60. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  61. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } }
  62. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } this loop will end very EARLY!
  63. @Component public class AttachCardsToProposalsJob { private CardsClient cardsClient; private CardRepository

    cardRepository; private ProposalRepository proposalRepository; @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } It will only run on the NEXT execution!
  64. @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { List<Proposal> proposals

    = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); proposals.forEach(proposal -> { // process each proposal }); }
  65. @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { while (true)

    { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { // process each proposal }); } }
  66. @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { while (true)

    { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { // process each proposal }); } }
  67. @Transactional @Scheduled(fixedDelay = 60_000) public void execute() { while (true)

    { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { // process each proposal }); } }
  68. @Component public class AttachCardsToProposalsJob { // dependencies and constants @Transactional

    @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } }
  69. @Component public class AttachCardsToProposalsJob { // dependencies and constants @Transactional

    @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } } - fetching only 50 objects at a time
  70. @Component public class AttachCardsToProposalsJob { // dependencies and constants @Transactional

    @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } } - creating a new reference - fetching only 50 objects at a time
  71. @Component public class AttachCardsToProposalsJob { // dependencies and constants @Transactional

    @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } } - exiting the loop at the right time - creating a new reference - fetching only 50 objects at a time
  72. @Component public class AttachCardsToProposalsJob { // dependências e constantes @Transactional

    @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } }
  73. Job

  74. @Component public class AttachCardsToProposalsJob { // dependencies and constants @Transactional

    @Scheduled(fixedDelay = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } }
  75. @Component public class AttachCardsToProposalsJob { // dependencies and constants @Scheduled(fixedDelay

    = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } } begins and ends a transaction
  76. @Component public class AttachCardsToProposalsJob { // dependencies and constants @Scheduled(fixedDelay

    = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); } } }
  77. @Component public class AttachCardsToProposalsJob { // dependencies and constants @Scheduled(fixedDelay

    = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); // begins and commits a transaction proposal.attachTo(newCard); proposalRepository.save(proposal); // begins and commits a transaction }); } } } We gave up on data consistency! 🤡
  78. @Component public class AttachCardsToProposalsJob { // dependencies and constants @Scheduled(fixedDelay

    = 60_000) public void execute() { while (true) { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(ELIGIBLE); if (proposals.isEmpty()) { break; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); // begins and commits a transaction proposal.attachTo(newCard); proposalRepository.save(proposal); // begins and commits a transaction }); } } } We give up on data consistency! Nooo!
  79. @Component public class AttachCardsToProposalsJob { @Transactional @Scheduled(fixedDelay = 60_000) public

    void execute() { while (true) { // executes the logic within a transactional scope } } }
  80. @Component public class AttachCardsToProposalsJob { @Transactional @Scheduled(fixedDelay = 60_000) public

    void execute() { while (true) { // executes the logic within a transactional scope } } }
  81. @Component public class AttachCardsToProposalsJob { @Transactional @Scheduled(fixedDelay = 60_000) public

    void execute() { while (true) { @Transactional.execute(transaction -> { // executes the logic within a transactional scope });; } } }
  82. @Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void

    execute() { while (true) { @Transactional.execute(transaction -> { // executes the logic within a transactional scope }); } } }
  83. @Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void

    execute() { while (true) { @Transactional.execute(transaction -> { // executes the logic within a transactional scope }); } } }
  84. @Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void

    execute() { while (true) { @Transactional.execute(transaction -> { // executes the logic within a transactional scope }); } } }
  85. @Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void

    execute() { while (true) { @Transactional.execute(transaction -> { // executes the logic within a transactional scope }); } } }
  86. @Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void

    execute() { while (true) { @Transactional.execute(transaction -> { // executes the logic within a transactional scope }); } } }
  87. @Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void

    execute() { while (true) { @Transactional.execute(transaction -> { // executes the logic within a transactional scope }); } } } It does NOT execute! 😵💫
  88. @Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void

    execute() { while (true) { @Transactional.execute(transaction -> { // executes the logic within a transactional scope }); } } }
  89. @Component public class AttachCardsToProposalsJob { @Scheduled(fixedDelay = 60_000) public void

    execute() { while (true) { transactionManager.execute(transaction -> { // executes the logic within a transactional scope }); } } }
  90. @Component public class AttachCardsToProposalsJob { private TransactionTemplate transactionManager; @Scheduled(fixedDelay =

    60_000) public void execute() { while (true) { transactionManager.execute(transaction -> { // executes the logic within a transactional scope }); } } }
  91. @Autowired private TransactionTemplate transactionManager; @Scheduled(fixedDelay = 60_000) public void execute()

    { boolean pending = true; while (pending) { pending = transactionManager.execute(transactionStatus -> { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }
  92. No!

  93. select p.* from proposal p where p.status = 'ELIGIBLE' order

    by p.created_at asc limit 50 for update
  94. No!

  95. @Repository public interface ProposalRepository extends JpaRepository<Proposal, UUID> { @QueryHints({ @QueryHint(

    name = "javax.persistence.lock.timeout", value = LockOptions.SKIP_LOCKED) // org.hibernate.LockOptions }) @Lock(LockModeType.PESSIMISTIC_WRITE) public List<Proposal> findTop50ByStatusOrderByCreatedAtAsc(ProposalStatus status); }
  96. @Repository public interface ProposalRepository extends JpaRepository<Proposal, UUID> { @QueryHints({ @QueryHint(

    name = "javax.persistence.lock.timeout", value = LockOptions.SKIP_LOCKED) // org.hibernate.LockOptions }) @Lock(LockModeType.PESSIMISTIC_WRITE) public List<Proposal> findTop50ByStatusOrderByCreatedAtAsc(ProposalStatus status); }
  97. @Repository public interface ProposalRepository extends JpaRepository<Proposal, UUID> { @QueryHints({ @QueryHint(

    name = "javax.persistence.lock.timeout", value = LockOptions.SKIP_LOCKED) // org.hibernate.LockOptions }) @Lock(LockModeType.PESSIMISTIC_WRITE) public List<Proposal> findTop50ByStatusOrderByCreatedAtAsc(ProposalStatus status); }
  98. select p.* from proposal p where p.status = 'ELIGIBLE' order

    by p.created_at asc limit 50 for update
  99. select p.* from proposal p where p.status = 'ELIGIBLE' order

    by p.created_at asc limit 50 for update skip locked
  100. @Autowired private TransactionTemplate transactionManager; @Scheduled(fixedDelay = 60_000) public void execute()

    { boolean pending = true; while (pending) { pending = transactionManager.execute(transactionStatus -> { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }
  101. @Autowired private TransactionTemplate transactionManager; @Scheduled(fixedDelay = 60_000) public void execute()

    { boolean pending = true; while (pending) { pending = transactionManager.execute(transactionStatus -> { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }
  102. @Autowired private TransactionTemplate transactionManager; @Scheduled(fixedDelay = 60_000) public void execute()

    { boolean pending = true; while (pending) { pending = transactionManager.execute(transactionStatus -> { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }
  103. @Autowired private TransactionTemplate transactionManager; @Scheduled(fixedDelay = 60_000) public void execute()

    { boolean pending = true; while (pending) { pending = transactionManager.execute(transactionStatus -> { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }
  104. @Autowired private TransactionTemplate transactionManager; @Scheduled(fixedDelay = 60_000) public void execute()

    { boolean pending = true; while (pending) { pending = transactionManager.execute(transactionStatus -> { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }
  105. @Autowired private TransactionTemplate transactionManager; @Scheduled(fixedDelay = 60_000) public void execute()

    { boolean pending = true; while (pending) { pending = transactionManager.execute(transactionStatus -> { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } } Inserts a new card Updates a proposal with its card
  106. INSERT INTO card(cardNumber) VALUES ('5260-9991-9040-0001'); UPDATE proposal SET status =

    'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2021; INSERT INTO card(cardNumber) VALUES (' 5513-8141-9261-0002'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2022; 4 roundtrips
  107. INSERT INTO card(cardNumber) VALUES ('5260-9991-9040-0001'); UPDATE proposal SET status =

    'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2021; INSERT INTO card(cardNumber) VALUES (' 5513-8141-9261-0002'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2022; INSERT INTO card(cardNumber) VALUES (' 5599-4154-6354-0003'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2023; 6 roundtrips
  108. 100 roundtrips INSERT INTO card(cardNumber) VALUES ('5260-9991-9040-0001'); UPDATE proposal SET

    status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2021; INSERT INTO card(cardNumber) VALUES (' 5513-8141-9261-0002'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2022; INSERT INTO card(cardNumber) VALUES (' 5599-4154-6354-0003'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2023; INSERT INTO card(cardNumber) VALUES (' 5419-8753-1126-0004'); UPDATE proposal SET status = 'ELIGIBLE_WITH_ATTACHED_CARD' WHERE id = 2024; INSERT INTO ... UPDATE proposal SET ...
  109. 2 roundtrips INSERT INTO card(cardNumber) VALUES (?) Params: [ ('5513-8141-9261-0001'),

    ('5513-8141-9261-0002'), (‘5513-8141-9261-0003'), (‘5513-8141-9261-0004'), ... ] UPDATE proposal SET status = ? WHERE id = ?; Params: [ (‘ELIGIBLE_WITH_ATTACHED_CARD’, 2021), ('ELIGIBLE_WITH_ATTACHED_CARD', 2022), ('ELIGIBLE_WITH_ATTACHED_CARD', 2023), ('ELIGIBLE_WITH_ATTACHED_CARD', 2024), ... ]
  110. 2 roundtrips INSERT INTO card(cardNumber) VALUES (?) Params: [ ('5513-8141-9261-0001'),

    ('5513-8141-9261-0002'), (‘5513-8141-9261-0003'), (‘5513-8141-9261-0004'), ... ] UPDATE proposal SET status = ? WHERE id = ?; Params: [ (‘ELIGIBLE_WITH_ATTACHED_CARD’, 2021), ('ELIGIBLE_WITH_ATTACHED_CARD', 2022), ('ELIGIBLE_WITH_ATTACHED_CARD', 2023), ('ELIGIBLE_WITH_ATTACHED_CARD', 2024), ... ]
  111. INSERT INTO card(cardNumber) VALUES (?) Params: [ ('5513-8141-9261-0001'), ('5513-8141-9261-0002'), (‘5513-8141-9261-0003'),

    (‘5513-8141-9261-0004'), ... ] 2 roundtrips UPDATE proposal SET status = ? WHERE id = ?; Params: [ (‘ELIGIBLE_WITH_ATTACHED_CARD’, 2021), ('ELIGIBLE_WITH_ATTACHED_CARD', 2022), ('ELIGIBLE_WITH_ATTACHED_CARD', 2023), ('ELIGIBLE_WITH_ATTACHED_CARD', 2024), ... ]
  112. INSERT INTO card(cardNumber) VALUES (?) Params: [ ('5513-8141-9261-0001'), ('5513-8141-9261-0002'), (‘5513-8141-9261-0003'),

    (‘5513-8141-9261-0004'), ... ] 2 roundtrips UPDATE proposal SET status = ? WHERE id = ?; Params: [ (‘ELIGIBLE_WITH_ATTACHED_CARD’, 2021), ('ELIGIBLE_WITH_ATTACHED_CARD', 2022), ('ELIGIBLE_WITH_ATTACHED_CARD', 2023), ('ELIGIBLE_WITH_ATTACHED_CARD', 2024), ... ] You got optimal batch processing!
  113. @Scheduled(...) public void execute() { boolean pending = true; while

    (pending) { pending = transactionManager.execute(txStatus -> { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }
  114. @Scheduled(...) public void execute() { boolean pending = true; while

    (pending) { pending = transactionManager.execute(txStatus -> { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }
  115. @Scheduled(...) public void execute() { boolean pending = true; while

    (pending) { pending = transactionManager.execute(txStatus -> { List<Proposal> proposals = proposalRepository.findTop50ByStatusOrderByCreatedAtAsc(…); if (proposals.isEmpty()) { return false; } proposals.forEach(proposal -> { CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); Card newCard = cardData.toModel(); cardRepository.save(newCard); proposal.attachTo(newCard); proposalRepository.save(proposal); }); return true; }); } }
  116. CardDataResponse cardData = cardsClient.findCardByProposalId(proposal.getId()); can I send the request again?

    idempotency transient errors? retry+backo f waits in fi nitely? timeout
  117. WHAT WHAT WHAT WHAT WHAT WHAT WHAT WHAT WHAT IF…

    WHAT IF… WHAT IF… WHAT WHAT IF… HAT IF… WHAT IF… WHAT WHAT IF… WHAT IF… WHAT IF… WHAT IF… AT IF… WHAT IF… F…WHAT
  118. …remains available in the face of failures …can handle huge

    volumes of data …can recover from crashes
  119. …designed and built to be scalable …remains available in the

    face of failures …can handle huge volumes of data …can recover from crashes
  120. …designed and built to be scalable …remains available in the

    face of failures …can handle huge volumes of data …optimized for high-throughput …can recover from crashes