to ASCII [Said] Management of K8s jobs for the Ignite cluster. [Chaker] Massive ingestion strategies [Said] Kogito and BPMN for business workflow and orchestration [Chaker] Other technologies used on the project [Antoine] UIs for managing ingestion jobs in K8s and Kogito [Chaker] SCIAM - 2025
by customer request Some facts have been simplified for the sake of clarity We won’t talk about accounting business logic We are not Mainframe experts SCIAM - 2025
to move its accountancy system to from the mainframe to microservices. We’re talking about 100 subsidiaries, 1 billion accounts, and 200 million movements per day… SCIAM - 2025
to move its accountancy system to from the mainframe to microservices. We’re talking about 100 subsidiaries, 1 billion accounts, and 200 million movements per day… The goal was to be able to process all this data as quickly as possible and to be able to scale the system. SCIAM - 2025
to move its accountancy system to from the mainframe to microservices. We’re talking about 100 subsidiaries, 1 billion accounts, and 200 million movements per day… The goal was to be able to process all this data as quickly as possible and to be able to scale the system. This talk will give you some insight on how we did it. SCIAM - 2025
the mainframe application that handles all the accounting of the group It is a monolithic COBOL application that has been in production for nearly 40 years SCIAM - 2025
the mainframe application that handles all the accounting of the group It is a monolithic COBOL application that has been in production for nearly 40 years It receives a huge amount of data from all the subsidiaries SCIAM - 2025
the mainframe application that handles all the accounting of the group It is a monolithic COBOL application that has been in production for nearly 40 years It receives a huge amount of data from all the subsidiaries A batch process runs every night to process all the data and generate the accounting entries SCIAM - 2025
of 2020 The goal was to rewrite the WAP application in Java and to move it to a microservices architecture The project is expected to be completed in 2030/2031 SCIAM - 2025
of 2020 The goal was to rewrite the WAP application in Java and to move it to a microservices architecture The project is expected to be completed in 2030/2031 the first phase of the project (biggest) will be in service in autumn SCIAM - 2025
of 2020 The goal was to rewrite the WAP application in Java and to move it to a microservices architecture The project is expected to be completed in 2030/2031 the first phase of the project (biggest) will be in service in autumn The team is composed of: 12 Java Devs, 5 teradata devs, 4 front devs, 10 Business Analysts, 1 tech lead, 1 PMO and 2 Managers SCIAM - 2025
that is used for large-scale computing purposes that require greater availability and security than a smaller-scale machine can offer. Mainframes are used primarily by large organizations for critical applications, bulk data processing, and transaction processing. SCIAM - 2025
that is used for large-scale computing purposes that require greater availability and security than a smaller-scale machine can offer. Mainframes are used primarily by large organizations for critical applications, bulk data processing, and transaction processing. Main language is COBOL (Common Business Oriented Language) and OS is Z/OS SCIAM - 2025
built-in redundancy and failover capabilities. Availability: They can run for long periods of time without downtime. Security: They have robust security features, including encryption, access controls, and auditing capabilities. SCIAM - 2025
built-in redundancy and failover capabilities. Availability: They can run for long periods of time without downtime. Security: They have robust security features, including encryption, access controls, and auditing capabilities. Scalability: They can handle large volumes of transactions and data processing, making them suitable for high-demand applications. SCIAM - 2025
and operate. Talent: shortage of skilled mainframe operators and developers (COBOL). Vendor lock-in: IBM has a monopoly on the mainframe market. SCIAM - 2025
and operate. Talent: shortage of skilled mainframe operators and developers (COBOL). Vendor lock-in: IBM has a monopoly on the mainframe market. Lack of flexibility: They are often seen as rigid and inflexible, making it difficult to adapt to changing business needs. SCIAM - 2025
is a big deal! Value Creation: The new system must be able to create value for the business. A good use case: If the system you want to migrate does only data ingestion and simple consultation, it is not worth the effort. SCIAM - 2025
from COBOL to Java (or any other language). You don’t change the architecture, you just change the language, and you generate non-maintainable code. SCIAM - 2025
from COBOL to Java (or any other language). You don’t change the architecture, you just change the language, and you generate non-maintainable code. Monolith wrapping: Wrap the application in a concrete dome. Features are exposed as API without touching the content You don’t change the architecture, you just change the language, and you generate non-maintainable code. SCIAM - 2025
from COBOL to Java (or any other language). You don’t change the architecture, you just change the language, and you generate non-maintainable code. Monolith wrapping: Wrap the application in a concrete dome. Features are exposed as API without touching the content You don’t change the architecture, you just change the language, and you generate non-maintainable code. Code rewriting: Rewrite the code from scratch. You change the architecture and you generate maintainable code. SCIAM - 2025
a fast startup time and low memory footprint It has a huge ecosystem of libraries and extensions It can go further thanks to GraalVM native image SCIAM - 2025
Teradata is a massively parallel processing (MPP) SQL engine We needed to process a lot of files We needed a powerful SQL engine to generate huge GROUP BY and JOINs requests SCIAM - 2025
by IBM for mainframes and midrange systems 8-bit character encoding, different from ASCII Incompatible with ASCII — requires conversion for interoperability SCIAM - 2025
to define the structure of records in COBOL programs Ensures consistency in data structure definitions across COBOL programs Often used in conjunction with data conversion routines (e.g., EBCDIC to ASCII) 01 CUSTOMER-RECORD. 05 CUST-ID PIC 9(5). 05 CUST-NAME PIC A(30). 05 CUST-ADDRESS PIC A(50). 05 CUST-BALANCE PIC S9(9)V99 COMP-3. * Ex : ±999999999.99 SCIAM - 2025
allowed) Java™ Batch Launcher and Toolkit for z/OS® (JZOS) (No one wants to touch mainframes) Cobrix - COBOL Data Source for Apache Spark (We have to drag the complexity of Apache Spark) SCIAM - 2025
GitHub You can contribute to it, everything is welcome Use it and test it Report issues Bug fixes New features Documentation github.com/bmTas/JRecord SCIAM - 2025
Triggers pod creation based on the file size Apache Ignite: In-memory distributed storage for file lines Java Quarkus: Lightweight JVM application in each pod, fast startup COS/S3: Source of the files to be processed SCIAM - 2025
IO (BufferedReader) Read the content of the file and store into the heap Collision issue (Even with a good hash function) OutOfMemory and heap space issues Garbage collector tuning headache SCIAM - 2025
(more control of memory) Use RocksDB as a key-value store (Fast and Reliable) Use Memory Mapped Files (FFM API de JDK 23) Use multi threading for file processing (More CPU usage) SCIAM - 2025
storage It is a C++ library built on LevelDB It is optimized for fast storage (SSD and Flash) It is designed for fast read and write operations It is used by many big companies (Facebook, LinkedIn, Yahoo, etc.) SCIAM - 2025
cookingPizza(int numberOfPizzas, int time) { //Make pizza dough var dough = beaterService.makePizzaDough(numberOfPizzas); //Roll out the dough and top var rollOutTop = rollOutService(dough); //Bake the pizza var pizza = ovenService.bake(rollOutTop,time); // Cut pizza pizza = ovenService.cut(pizza); return pizza; } } SCIAM - 2025
Not Optimized for cloud and architecture microservices. Long startup time Large compute consumption Poor performance Affect developers' productivity SCIAM - 2025
evolve High availability Schedules Human operation friendly Fault-Tolerance(Retry) Atomicity Data Consistency (Saga) Long Running Friendly SCIAM - 2025
Development (BDD). As we have a lot of SQL queries to test, we needed a way to test them. It was a blessing for us to have a common language between devs and business. SCIAM - 2025
Development (BDD). As we have a lot of SQL queries to test, we needed a way to test them. It was a blessing for us to have a common language between devs and business. In the end, BA were writing the tests and devs only creating new vocabulary. SCIAM - 2025
We have multiple schemas to manage on the projet It was a bit hard to have our DBA accept this tool Flyway was also considered but not chosen due to cost SCIAM - 2025
flow execution Access step-level logs Available actions: Restart (from beginning or specific step) Cancel Temporarily disable a flow type SCIAM - 2025
antoine_sd
bitkey
hideg
goataka
tomokusaba
_katsuma
meteatamel
kuro_kurorrr
manfredsteyer
rkaga
nrslib
headius
ike002jp
danielanewman
paigeccino
skipperchong
jmmastey
maltzj
hannesfritz
sergeychernyshev
caitiem20
pedronauck
tammyeverts
rasmusluckow
marktimemedia
![Agenda Introduction context and Architecture [Antoine] From EBCDIC](https://files.speakerdeck.com/presentations/4de5246bb55947ae86cadfed21f588c9/slide_4.jpg){kind=link}
