Kamal 2 – Get Out of the Cloud

Slides about Kamal 2 features and some practical examples.

Showed live on the Tropical on Rails 2025, São Paulo, Brazil, April 4th, 2025.

Transcript

1. Kamal 2 – Get out of the cloud Igor Aleksandrov https://github.com/igor-alexandrov
2. Who am I? • On Rails since 2009 (RoR 2.2+) • CTO and co-founder of JetRockets • Docker Captain and Open-Source enthusiast • Kamal contributor since 0.14.0 release
3. I live in Georgia (the country) • Georgia has a wine tradition of more than 8000 years (Google it) • UNESCO honors the cultural significance of the Georgian alphabet • Come to visit Georgia! (მობრძანდით საქართველოში!)
4. Why does it matter to talk about the deployment? • Delivering changes to production is often a bottleneck of the development process • Heroku, Fly.io and Render give an excellent developer experience, but cost you premium • Hosted Kubernetes on AWS, GCP, Digital Ocean lock you to the vendor and cost you premium • More complex infrastructure solutions require more engineers in your team
5. Develop → Deliver
6. What is Kamal? • Essentially it is an orchestration tool around Docker • If you know Docker – no additional knowledge is needed • Written in Ruby on Go
7. 2915 LoC Kamal (MRSK) v0.14.0, counted by https://github.com/AlDanial/cloc 9004 including tests and comments
8. 5537 LoC Kamal v2.5.3, February 27, 2025 17456 including tests and comments
9. Is it a lot? • Sidekiq – 6030 LoC • Devise – 3445 LoC • AASM – 2139 LoC • Solid Queue – 1670 LoC Kamal has 5537 LoC in the v2.5.3 release.
10. The core idea of Kamal?
11. Capistrano for containers
12. Capistrano for containers It sounds obvious, but what does it mean? • It provides zero downtime deployments • It is imperative and simple • It is efficient
13. What are zero downtime deployments? Capistrano • Rely on the application server (e.g. Puma phased restart) Docker • Run Docker commands ◦ Start new container and wait until it is healthy ◦ Notify the proxy server about the new container ◦ Tell the proxy server to stop routing requests for the old container ◦ Stop old container
14. Being imperative Capistrano • Declare commands with DSL • Run each command and wait until it is finished Kubernetes & Docker Swarm • You are suggesting, not insisting on your actions
15. Being efficient • Utilize Docker’s layer caching • Not being as slow as AWS Beanstalk • Not add any additional complexity
16. What’s new in Kamal 2?
17. Traefik replaced by kamal-proxy
18. What were the reasons for this? Traefik looks like an almost perfect solution, doesn’t it? 1. Docker-ready 2. Declarative configuration ◦ Uses Docker labels to define routing 3. Automatic routing ◦ Traffic to the new container is routed automatically 4. Self-healing ◦ If the container restarts, Traefik dynamically updates routing
20. 1. Label-based configuration is challenging to maintain
23. 2. Kamal was used for something, it wasn’t designed for
25. 3. Traefik is declarative
26. - Okay… - But what does it mean?
29. We had no control over what was going on
30. 4. Docker container’s labels are immutable
31. Container deployment plan 1. Start the “new” container while the “old” is running. 2. Wait for the “new” to become healthy 3. Stop the “old” container. Easy-peasy?
33. Container deployment plan 1. Start the “new” container while the “old” is running. 2. Wait for the “new” to become healthy 3. Stop the “old” container.
34. It takes time for Traefik to update its configuration
35. Container deployment plan 1. Start the “new” container while the “old” is running. 2. Wait for the “new” to become healthy 3. “Tell” Traefik to stop routing requests to the “old” 4. Wait while Traefik will actually stop routing to the “old” container 5. Stop the “old” container. But Docker container’s labels are immutable and Traefik is declarative, we cannot “tell” him anything.
36. Workaround – plug each container with a “cord” to a socket
37. What is a cord? It is a modified health check command When starting a new container – tie the cord • --volume ~/.kamal/cords/onetribe-web-production-cf6ccd:/tmp/kamal-cord • --health-cmd "(curl -f http://localhost:3000/up || exit 1) && (stat /tmp/kamal-cord/cord > /dev/null || exit 1)" Before stopping the container – cut the cord • rm -r ~/.kamal/cords/onetribe-web-production-938b7e
38. Container deployment plan 1. Create a directory with a single file on the host 2. Start the “new” container, mounting the cord file into /tmp and including a check for the file in the docker health check 3. Wait for the “new” to become healthy 4. Delete the health check file (“cut the cord”) for the “old” container 5. Wait for it to become unhealthy and give Traefik a couple of seconds to notice 6. Stop the “old” container
39. kamal-proxy eliminates these problems • Configurable only with options • Has Let’s Encrypt built-in • Imperative `kamal-proxy deploy onetribe-pghero --target c67f2259dce6:8080 --host pghero.onetribe.team --tls` The command above will wait until the host is provisioned and the container is healthy. There also are `kamal-proxy remove`, `kamal-proxy list`, etc.
41. Longest discussion about Kamal 1.x
42. Kamal 2 adds SSL with 2 lines of code The only limitation – it requires deploying to a single web server, but you can deploy queue and accessories to others.
43. Thruster
44. One more Rails trifecta • Kamal • kamal-proxy (ex. Parachute) • Thruster (the most incomprehensible new thing)
45. Nginx was a common solution in the Rails stack
46. What were Nginx responsibilities? • Reverse proxy • Routing HTTP requests • Static files serving • Caching • Compression • Rate limiting (optional)
50. Kamal kamal-proxy • Reverse proxy • Routing HTTP requests Puma • Serving Static Files • Rate Limiting (optional) Who should be responsible for `caching` and `compression`?
52. Thruster • Go wrapper around Puma, other app servers are not (yet?) supported • Distributed as a gem (gem “thruster”) • Easy to run (“./bin/thrust ./bin/rails server”) • Handles: ◦ Serving static files with “X-Sendfile” support ◦ Caching of public assets ◦ Compression
53. Kamal 2 in production
54. Harden your instance! • `kamal setup` only installs Docker if it is missing • No fail2ban, no firewall, hardening is a must! • DO NOT EXPOSE the Docker daemon on port 2375 If you are using Ansible – https://github.com/dev-sec/ansible-collection-hardening
55. Learn Docker basics! To use Kamal you should be familiar with terminology: • Container • Image • Registry Have any questions? User Docker docs. https://docs.docker.com/get-started/docker-concepts/the-basics/what-is-a-conta iner/
56. Keep the Docker registry close • Kamal needs a Docker registry (at least for now) • Every deploy – the new image is built, pushed and pulled • ghcr.io, Docker Hub, ECR, DigitalOcean Container Registry, • You will pay for the traffic
57. Make Kamal a part of CI/CD process • Kamal can build images locally • I insist you to use Kamal as a CD workflow instead: ◦ Your architecture can be different from the architecture you deploy to ◦ Usually ARM64 locally vs x86-64 on the server (we all love Mac). ◦ Your internet connection may be slow and not stable (like in the hotel, here) • Kamal integrates well with any CI/CD platform: GH Actions, GitLab CI, BitBucket Pipeline If you are on GitHub Actions – https://github.com/igor-alexandrov/kamal-github-actions
58. Keep your workflows in a good condition • On GitHub – use composite actions • Don’t reinvent a bicycle, use existing actions e.g., ◦ https://github.com/webfactory/ssh-agent ◦ https://github.com/ruby/setup-ruby Use Kamal as an action too – https://github.com/igor-alexandrov/kamal-deploy
59. Concurrent deployments handling • The default behavior of GitHub Actions is to allow multiple jobs or workflow runs to run concurrently. • Your deployments should not run in parallel for the same environment
60. Concurrent deployments handling
61. Keep your secrets safe! • NEVER provide direct values to `.kamal/secrets` • Two options that are suitable for a production usage: ◦ Store secrets in GitHub Secrets and substitute them from environmental variables ◦ Store secrets in a password manager • I love the first option because of no additional services
62. Run migrations once! Rails' default Dockerfile sets an entry point to `/rails/bin/docker-entrypoint` by default.
63. Run migrations once! If the command is `./bin/rails server`, then `./bin/rails db:prepare` is executed, which also runs DB migrations. It knows nothing about the server on which it is executed.
64. But what if your setup has multiple web containers?
66. Run migrations once! Use `pre-deploy` hook instead. Kamal provides all the required variables and will ensure to run the command only on the primary server.
67. Logs! Logs! Logs! • Kamal logging uses Docker logging under the hood • Kamal limits maximum file size to 10 megabytes by default • Don’t forget to choose the right logging driver ◦ json-file (default) ◦ local ◦ awslogs ◦ gcplogs • You should have a log viewer (e.g. CloudWatch, Dozzle, Logdy) https://dozzle.dev OR https://logdy.dev
68. You can have multiple Kamal configs • The most common use case – microservices in a monorepo • Another use case – multi-tenant or multi-region deployments Kamal allows selecting the config file to use ./bin/kamal --config-file ./config/deploy.service1.yml
69. Kamal is not a silver bullet! • Don’t forget to configure a backup for your database • Don’t forget to set up monitoring • Don’t forget to update your Dockerfile dependencies • Keep your application up to date too • Know what you are doing and have fun!
70. Future Roadmap
71. “Kamal is in a great place now” – DHH
72. What can be improved Kamal • Get rid of remote image repository requirement kamal-proxy • Maintenance mode
73. Thank you! AMA. https://x.com/igor_alexandrov https://www.linkedin.com/in/igor-alexandrov