The Good, the Bad and the Ugly of Migrating Hundreds of Legacy Applications to Kubernetes

Running applications on Kubernetes can provide a lot of benefits: more dev speed, lower ops costs, and a higher elasticity & resiliency in production. Kubernetes is the place to be for cloud native apps. But what to do if you’ve no shiny new cloud native apps but a whole bunch of JEE legacy systems? No chance to leverage the advantages of Kubernetes? Yes you can! 

We’re facing the challenge of migrating hundreds of JEE legacy applications of a major German insurance company onto a Kubernetes cluster within one year. We're now close to the finish line and it worked pretty well so far.

The talk will be about the lessons we've learned - the best practices and pitfalls we've discovered along our way. We'll provide our answers to life, the universe and a cloud native journey like:
- What technical constraints of Kubernetes can be obstacles for applications and how to tackle these?
- How to architect a landscape of hundreds of containerized applications with their surrounding infrastructure like DBs MQs and IAM and heavy requirements on security?
- How to industrialize and govern the migration process?
- How to leverage the possibilities of a cloud native platform like Kubernetes without challenging the tight timeline?

Transcript

1. Migrating Hundreds of Legacy Applications to Josef Adersberger, CTO, QAware
 @adersberger
 }THE GOOD, THE BAD, THE UGLY proud CNCF member
2. HYPERSCALE RESILIENT OPEX SAVINGS SPEED
3. CIO Let’s bring 
 all our web 
 applications
 into the cloud! I’ve already 
 canceled our
 current data center 
 contract.
4. Chief Architect Great goal! But we’ll need 
 our time.
5. Excellent! You’ve got one year.
6. CIO My priorities: (1) Security level (2) Time (3) OpEx savings (4) Migration costs
8. 8 WE WERE BRAVE
9. AND WE HAD … IMPEDIMENTS
10. THE GOOD
11. 11 Visibility
12. Let’s architect the cloud!
13. PACKAGED AND DISTRIBUTED AS CONTAINERS BUILD AND COMPOSED AS MICROSERVICES DYNAMICALLY EXECUTED IN THE CLOUD CLOUD NATIVE APPLICATIONS 3 KEY PRINCIPLES
14. OMG - no greenfield
 approach!? Hundreds of legacy systems!?
15. Questionnaire: Typical Questions And Their Motivation 1. Technology stack (e.g. OS, appserver, jvm) 2. Required resources (memory, CPU cores) 3. Writes to storage (local/remote storage, write mode, data volume) 4. Special requirements (native libs, special hardware) 5. Inbound and outbound protocols (protocol stack, TLS, multicast, dynamic ports) 6. Ability to execute (regression/load tests, business owner, dev knowhow, release cycle, end of life) 7. Client authentication (e.g. SSO, login, certificates) What images to provide? How many applications will be hard/ inefficient to schedule (>3 GB RAM, > 2 cores)? What storage solutions to provide? What applications will be hard to impossible to be containerized? Are there any non cloud-friendly protocols? How risky is the migration? Is the migration maybe not needed? What IAM and security mechanisms have to be ported to the cloud?
16. 16 QAVALIDATOR SONARQUBE EAM TOOL QUESTIONNNAIRES JIRA XLS STATIC ANALYSIS IBM MIGRATION TOOL … MIGRATION TASKS BASIC TOUR-DE-MIGRATION SYSTEM PROPERTIES dashboard for information radiator freestyle analysis with Tableau MIGRATION DATABASE CLOUDALYZER
17. 17 Emergent design of software landscapes
18. OLD APPLICATIONS (~200) MORE MODERN APPLICATIONS (~200) ALL WEB APPLICATIONS (~400) Re-architect to run on k8s on AWS lift & shift VMs to AWS EC2
19. APPLICATION HTTPD WEB LAYER ALMIGHTY LEGACY FRAMEWORK J2EE 1.4 APPSERVER JVM 1.6 DB MQ HOST BATCH FS CLIENTS TLS 1.0+ mostly non-TLS;
 TCP-Binary, WS, REST, C:D, LDAP
 Corba, SMTP, FTP, NAS, … RACF ESB ONPREM DATA CENTER ONPREM DATA CENTER DB MQ HOST BATCH FS RACF ESB KUBERNETES / OPENSHIFT (CAAS) DOCKER JVM 8 ALMIGHTY FRAMEWORK NG INNER APPLICATIONS AWS WEB LAYER AWS CLIENTS TLS 1.2 all TLS 1.2 JEE 7 APPSERVER API GATEWAY OUTER APPLICATIONS all 2-way TLS 1.2
 & OIDC
 identity token
20. APPLICATION HTTPD WEB LAYER ALMIGHTY LEGACY FRAMEWORK J2EE 1.4 APPSERVER JVM 1.6 DB MQ HOST BATCH FS CLIENTS TLS 1.0+ mostly non-TLS;
 TCP-Binary, WS, REST, C:D, LDAP
 Corba, SMTP, FTP, NAS, … RACF ESB ONPREM DATA CENTER ONPREM DATA CENTER DB MQ HOST BATCH FS RACF ESB KUBERNETES / OPENSHIFT (CAAS) DOCKER JVM 8 ALMIGHTY FRAMEWORK NG INNER APPLICATIONS AWS WEB LAYER AWS CLIENTS TLS 1.2 all TLS 1.2 JEE 7 APPSERVER API GATEWAY OUTER APPLICATIONS all 2-way TLS 1.2
 & OIDC
 identity token
21. 2 Can we evolve existing enterprise applications into the cloud with reasonable effort? Containerization 12-Factor App Principles Microservices Cloud-native Apps Monolithic Deployment Traditional Infrastructure CLOUD FRIENDLY CLOUD NATIVE CLOUD ALIEN
22. 2 Can we evolve existing enterprise applications into the cloud with reasonable effort? Containerization 12-Factor App Principles Microservices Cloud-native Apps Monolithic Deployment Traditional Infrastructure Sweetspot
 time and value (security, opex) Put the monolith into a container … and enhance the
 application according the 12 factors CLOUD FRIENDLY
23. Edge Service to the Rescue
24. MONOLITH MONOLITH EDGE SERVICE BACKEND CLIENTS API GATEWAY BACKEND CLIENTS BEFORE AFTER
25. MONOLITH EDGE SERVICE CLIENTS TOKEN PROVIDER TLS 1.2 + diverse user contexts (SAML, Cookie, Header, Certificate, User/Pwd, …) Change user context to OIDC identity token IAM SYSTEMS API GATEWAY TLS 1.2 + identity token TLS 1.2 + identity token Redirect to (S)SO if user context not set / valid but required user context > 10
26. Edge Service <<pod>> Edge Service <<container>> Edge Service Chassis <<spring boot app>> Spring Framework Spring Boot Netflix Zuul / Spring Cloud Zuul Edge Service Filters Redis Edge Library Token Service Token Cache SPI Token SPI Token API Token Handling Filter <<pre>> Static Content Filter <<routing>> Security Filter <<pre>> Auth Plugins Token Provider Web Layer API Gateway
27. Sidecars to the Rescue
28. Container Patterns Applied • Log extraction / re-formatting (fluentd) • Scheduling (Quartz) Sidecar: Enhance container behaviour Ambassador: Proxy communication Adapter: Provide standardized interface • Configuration (ConfigMaps & Secrets to files) • TLS tunnel (Stunnel, ghosttunnel) • Circuit Breaking (linkerd) • Request monitoring (linkerd) Pod Application Container Pattern Container Other Container “Design patterns for container-based distributed systems”. Brendan Burns, David Oppenheimer. 2016
29. Kubernets 
 Constraints Initially we thought we’ll run into k8s restrictions on our infrastructure like: ‣ No support for multicast ‣ No RWX PVC available We did. But cutting these
 application requirements lead to a better architecture in each and every case.
30. THE BAD
31. STATE
32. Databases ONPREM DATA CENTER AWS DATABASES MONOLITH Databases stay in onprem data center • Advantages: • onprem/cloud version of the application can run in parallel • privacy • Disadvantages: • latency (no real problem) Activate TLS for all database connections
 (low effort on application-side but separate project on the database side)
33. Files File persistence is very restricted to keep persistent state completely out of AWS (privacy). ‣ No file writes allowed into container ‣ No RWX PVC available ‣ Files with application data written to PVC must be deleted after 15mins ‣ No NAS mounts from onprem data centers into containers allowed Migration tasks for affected applications ‣ Store files as BLOB in database or use FTPS
34. Session State 90% OF THE APPLICATIONS HAVE SESSION STATE 100% OF THE APPLICATIONS HAVE MORE THAN 1 INSTANCE Session Stickyness: not within the cloud! Session Persistence: ‣ Within existing application database: performance impact too high for most ‣ Redis: no transport encryption out-of-the-box and separate infrastructure required Session Synchronization: ‣ Application server: nope, no dynamic peer lookup within k8s ‣ In-memory data grid: Hazelcast. Nope. $$$ for TLS. ‣ In-memory data grid: Apache Ignite. Done. ‣ Embedded within application or standalone in a separate container. ‣ Little bit cumbersome but working k8s peer lookup ‣ Runs into JIT bug on IBM JVM (some methods have to be excluded from JIT)
35. DIAGNOS-
 ABILITY need a DevOps?
36. Metrics Events / Logs Traces Diagnosability
37. Metrics Events / Logs Traces Diagnosability
38. Metrics Events / Logs Traces • High effort to instrument for valuable insights •Scalability unclear for hundreds of applications •Applications have no time to run their own Prometheus instance •Scalability unclear for hundreds of applications (Jaeger & ZipKin) •Applications have no time to run their own instance •Scalability unclear (a lot of events lost) •Applications have no time to run their own EFK instance •Non-standardized log format requires custom log rewrite adapter but no fluentd DaemonSet Diagnosability
39. Metrics Events / Logs Traces
40. SECURITY need a SecDevOps?
41. We Came Far BACKEND MONOLITH EDGE SERVICE TLS 1.2 + user context API GATEWAY Two-way TLS 1.2 + identity token Two-way TLS 1.2 + identity token TOKEN CHECK TLS 1.2 + user context secrets injected at container startup security filters included CLIENTS CLIENTCERT ACL EGRESS RULES
42. The Bad: Certificate Management VAR 1: CLOUD NATIVE STYLE CERTIFICATE MANAGEMENT 
 (SPIFFE-BASED, AT SERVICE MESH OR APPLICATION LEVEL) VAR 2: REPLACE BY POLICIES (AT NETWORKING LEVEL) e.g. e.g. SPIRE
43. THE UGLY String hostRequest = new HostRequest().hostusPokus(message);
44. 44 CLOUD ENABLING 
 CLOUD ALIENS
45. TOXIC
 TECH
46. The Almighty Legacy Framework • “worry-free package framework” from the early 2000s with about 500kLOC and 0% test coverage • Migration tasks • from J2EE 1.4 to JEE 7 and Java 6 to 8 • add identity token check and token relay • modify session handling (synchronization) • modify logging (to STDOUT) • modify configuration (overwrite from ConfigMap) • enforce TLS 1.2 • place circuit breakers • predefined liveness and readiness probes • Strategies: • the hard way: migrate manually and increase coverage • decorate with ambassadors, sidekicks and adapters • do not migrate parts and replace that API within the applications APPLICATION ALMIGHTY LEGACY FRAMEWORK J2EE 1.4 APPSERVER JVM 1.6
47. 47 ~ 100 systems live on target by end of this year (after 8mo) ~ 200 systems live on target by end of first quarter 2018 other ~200 systems migrated by end of first quarter 2018 via virtual lift & shift. They will be migrated onto Kubernetes afterwards
48. That’s what we’ve learned from migrating hundreds
 of J2EE legacy apps
 onto Kubernetes?

49. No stupid “as-is into 
 containers” approach!
50. Getting as close as 
 possible to cloud friendly 
 application principles. Increasing the security 
 level by an order of 
 magnitude!
51. Q -> A proud member of the CNCF
52. TWITTER.COM/QAWARE - SLIDESHARE.NET/QAWARE Thank you! josef.adersberger@qaware.de @adersberger
53. BONUS SLIDES
54. 54 Industrialization
55. ARCHITECTURE TEAM (~2 FTE) SUPPORT TEAM (~10 FTE) DOZENS OF MIGRATION PROJECTS RUNNING IN PARALLEL (UP TO ~80) ‣ Training sessions ‣ Support sessions ‣ Guidance (cookbook, sample application) ‣ Development environment (SEU-as-Code) ‣ Standard images ‣ Automated Refactorings (RefactoBot) ‣ Pre-migrated frameworks ‣ Solutions: Edge service, ambassadors INDUSTRIALIZATION TEAM (~6 FTE) ‣ Application blueprint (target architecture, migration rules) ‣ Migration database
56. NO SPEAK 
 CLOUD
57. DMS System APPLICATION DMS Proprietary (binary)
 TCP-level on fixed ports
 Non-encrypted
 Multiple target servers DMS ADAPTER Hard-coded DNS names as reference to other servers in response APPLICATION DMS ADAPTER STUNNEL STUNNEL DMS ONPREM KUBERNETES ON AWS POD STUNNEL STUNNEL PODS SVC.HOST1 SVC.HOST2 SVC.HOST3 lbl:host1 lbl:host2 lbl:host3 ONPREM dnshostname1.com.myapp.svc.cluster.local dnshostname1.com
58. 8 PLAN BUILD RUN GRASP Continuous
 Delivery Continuous
 Feedback
59. IDE (ECLIPSE, INTELLIJ) ARTIFACTORY Kubernetes Cluster RUNNING TESTED APPLICATIONS INFORMATION RADIATOR GITHUB ENTERPRISE ACCOUNTABILITY PLUGIN JENKINS JARs
 Docker image Docker image deploy Trigger
 Code Code Status