memory because: • Steep makes resident processes because it works as LSP server • Steep makes many processes ◦ For number of projects using Steep ◦ For number of CPUs because Steep launches workers for parallelization ◦ total_memory = projects.size * CPUs.size * memory_per_process
consume ~1.5GB memory in a middle size Rails application e.g. 8 core * 5 project * 1.5GB/proc = 60GB We need to decrease the memory usage in order for Steep to be widely used.
memory/objects during profiling • Pros: It's helpful to find a execution time bottleneck caused by memory allocation • Cons: It's not helpful to find the cause of the peak memory usage ◦ Too noisy ◦ Example: Steepのメモリ使用量を改善するつもりが、実行速度の改善をして いた - Money Forward Developers Blog https://moneyforward-dev.jp/entry/2024/07/29/improve-steep-performanc e
memory/objects when the profiling is finished • Pros: It's helpful to find a memory leak • Cons: It's not helpful to find the cause of the peak memory usage ◦ We need to stop profiling on the peak, but the peak is not obvious
is approximated as memory usage of long-lived objects • Majo collects allocation info only for long-lived objects ◦ It introduces object lifetime by how many times the object survived GC
◦ https://github.com/ruby/rbs/pull/1950 • Reduce Hash allocation during parsing ◦ 不要な処理が実行速度を速くする謎を追う - Money Forward Developers Blog https://moneyforward-dev.jp/entry/2024/09/26/removing-steps-make s-it-slower ◦ I will introduce this patch for the next Ruby version
3] Copy on Write Example (3) # Process A x = [1, 2, 3] if fork x.push(42) p x else p x end # Process A' x = [1, 2, 3] if fork x.push(42) p x else p x end
for the Master-Worker model • In the traditional Master-Worker model, workers are forked from the master process • In Fork-Worker, workers are forked from a worker process I borrowed this idea from puma and pitchfork (HTTP server for Ruby) https://github.com/puma/puma/blob/master/docs/fork_worker.md