Not Your Grandmother's On-Prem

The last two decades have been all about SaaS, with advantages that cannot be overstated. Except SaaS isn't always an option, nor is it always the right choice: businesses in tightly regulated industries, or where information security is paramount, for example, will not - often can not - consider any software that isn't under their control. For many software enterprises, this leads to the dreaded inevitability of on-premise deployment.

Fortunately, the situation today is dramatically different to a scant few years ago, let alone a decade or two: the same technologies that enable SaaS have also radically transformed on-prem deployment. Modern tools like Docker, Consul, ELK and Kubernetes - to name a few - can be leveraged to completely transform the experience for both customers and vendors. In this talk we'll contrast the challenges and advantages of SaaS and on-prem, see how things have evolved in recent history, and see how modern on-prem deployment can be, if not pleasurable, at least relatively painless.

Transcript

1. T o m e r G a b e l Kraków, 15-17 May 2019 // Not Your Grandmother’s On-Prem T o m e r G a b e l Porto, April 28th 2022 //
2. What’s all this, then? • “On-premise” • Used to be taken literally: “at the customer’s location” • This is no longer relevant • Most customers don’t own their own data center • Most customers don’t own their own hardware • Most customers don’t even lease long-term • Customers are likely running “on someone else’s prem” • A more useful interpretation: • “Running on hardware outside of my control”
3. Let’s get it out in the open… • On-prem is a swear-word these days • For good reasons! • Are they, though? • It’s 2022 • Much has changed • And I’ve recently wrapped a 6-month on-prem project • Let me tell you all about it!
4. Who or what am I? • Freelance software engineer • Been around the block • Up and down the stack • For a bunch of companies you’ve heard of • Old and grumpy
5. Yeah. So, on-prem
6. Why is it hard? SaaS On-Prem Release process Simple Complex Direct control Full, in-house Limited, involving 3rd parties Concurrently-deployed versions Small, finite set Potentially many Staff In-house and cooperative 3rd party
7. Why is it hard? Coordination 🤬
8. Why do this to yourself? • We all love SaaS • It’s the future, they said • They were right
9. Why do this to yourself? • We all love SaaS • But in some cases, it’s just not an option • Regulatory concerns (e.g. medical information) • Security considerations (e.g. critical infrastructure) • Physical constraints (e.g. no internet connectivity) • And sometimes it’s not practical • When you actually sell infrastructure • When your customers simply won’t buy SaaS
10. On-prem, Evolved
11. On the shoulders of giants • SaaS was a radical new paradigm 20 years ago • New tools evolved to support the ecosystem • Can we leverage these tools on-prem? Hypervisors Deploy heterogenous software in isolation Public cloud Programmable, elastic production hardware Containers Simple software packaging and distribution Container orchestrators Abstract execution environment
12. Yes we can.
13. First, a dose of reality • All on-prem projects aren’t created equal • What are your constraints? • Data persistence/retention Your network On-prem Your system Licensing Integrations Telemetry Persistent Storage?
14. First, a dose of reality • All on-prem projects aren’t created equal • What are your constraints? • Data persistence/retention • Cluster vs single host Your network On-prem Your system You? You? You? Licensing Integrations Telemetry
15. First, a dose of reality • All on-prem projects aren’t created equal • What are your constraints? • Data persistence/retention • Cluster vs single host • Cloud vs colo Your network VPC Your system Licensing Integrations Telemetry Cloud Provider Storage Identity Messaging DB
16. First, a dose of reality • All on-prem projects aren’t created equal • What are your constraints? • Data persistence/retention • Cluster vs single host • Cloud vs colo • Networking Your network On-prem Your system Licensing Integrations Telemetry Persistent Storage
17. Let’s talk tooling
18. Docker • It’s a godsend • Yeah, I know. Issues and nitpicks • BUT! • Consistent packaging • Consistent artifact hosting • Consistent operational model • Composes well • Reusable in all scenarios
19. docker-compose • Multiple components, single host • Lightweight orchestration • Dead-simple • Doesn’t get enough credit! • Composes really well with…
20. Virtual Machine Images • VMs are awesome: • Self-contained package • Consistent platform (-ish) • Consistent deployment model • Supported everywhere • Private/public cloud • Publish as VM image with marketplace offerings
21. Container orchestration • If you have to run on multiple hosts… • My condolences • It’s going to be hard • Kubernetes/Nomad can help • Especially if utilizing EKS/AKS/GKE • A huge subject requiring its own talk
22. Public cloud FTW • Do your customers run on AWS, Azure, GCP et al? • Awesome! You get loads built-in • Delivery mechanism (VMs) • Secure integration with customer resources (data stores etc.) • Reduced operational burden via available services • SLA and regulatory guarantees that are not your problem • Customer buy-in ahead of time!
23. Monitoring • SaaS observability tools • Great if you can use them! • Trusted 3rd party services • Secure and auditable access • Low operational burden • Predictable OpEx • A much easier sell!
24. Going oldschool • If you can’t ”call home”… • Log shipping is the only way to go • Experience makes for good logs • No silver bullet. Sorry! • Some mitigations • If on cloud, object store + temporary ACLs may help • Zoom/Meet/etc. “remote control” features are amazing
25. Case study
26. Product X • Unfortunately defunct • But that means I can talk about it! • A multi-component production system • Python 3.9 • Redis cache • PostreSQL • Front-end static bundle • External integration points
27. Constraints Customers running on public cloud (initially AWS, Azure) Completely isolated: no inbound or outbound traffic Deployment owned by a business analyst team • Very limited technical knowhow • Uncooperative ops department 01 02 03
28. Choices Single host (to start with) docker-compose • Lightweight and works • Easy to instruct on operation • Reusable for testing VM image published on marketplace Utilizes cloud RDBMS, blob store Packer targeting CentOS 7.9 01 02 03 04 05
29. What Worked Well • Trivial setup process • Initial VM deployment with a few clicks via marketplace • Fully integrated with customer’s identity/security • Trivial upgrades • Leveraging Docker tags • Little or no technical know-how required!
30. Challenges • Ops/SRE expertise • Many disparate tools (Packer, Docker, systemd…) • Multiple targets (AWS/Azure) • Lots of ”glue logic” with shell scripts • Support and diagnostics • No 3rd parties allowed • No direct access to server/database/logs
31. In Conclusion
32. Takeaways • The tools that enable SaaS are a boon for on-prem • Public clouds make for compelling on-prem targets • VMs and Docker make deployment almost painless • Supporting on-prem is surprisingly easy with: • 3rd party observability tools (logging, APM) • Modern communication tools (Zoom et al)
33. Not Your Grandmother’s On-Prem 1. On-prem is harder than SaaS 2. Modern tools make it quite painless 3. Be not afraid!
34. tomer@substrate.co.il @tomerg https://github.com/holograph Thank you for listening Questions?