Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Ruby 3x3 に向けたRubyのベンチマーク環境と成果 / Ruby Association Grant
Search
Takashi Kokubun
July 07, 2018
Programming
10
13k
Ruby 3x3 に向けたRubyのベンチマーク環境と成果 / Ruby Association Grant
Rubyアソシエーション開発助成成果報告会
https://rubyassociation.doorkeeper.jp/events/74355
Takashi Kokubun
July 07, 2018
Tweet
Share
More Decks by Takashi Kokubun
See All by Takashi Kokubun
YJIT Makes Rails 1.7x faster / RubyKaigi 2024
k0kubun
7
8.4k
Ruby JIT Hacking Guide / RubyKaigi 2023
k0kubun
2
9.2k
YJIT: Dive into Ruby's JIT compiler written in Rust / Rust.Tokyo 2022
k0kubun
1
1.9k
Towards Ruby 4 JIT / RubyKaigi 2022
k0kubun
3
10k
Optimizing Production Performance with MRI JIT / RubyConf 2021
k0kubun
1
380
Why Ruby's JIT was slow / RubyKaigi Takeout 2021
k0kubun
3
1.8k
数時間かかる週一リリースを毎日何度も爆速でできるようにするまで / CI/CD Conference 2021
k0kubun
21
14k
Ruby 3 JIT's roadmap / RubyConf China 2020
k0kubun
0
710
Ruby 3.0 JIT on Rails
k0kubun
9
8.9k
Other Decks in Programming
See All in Programming
Composing an API the *right* way (Droidcon Berlin 2024)
zsmb
1
450
feature環境をGitHub ActionsとCloudFormationでいい感じに管理する
nealle
2
310
Javaの現状2024夏 / Java current status 2024 summer
kishida
4
1.4k
CSC307 Lecture 06
javiergs
PRO
0
360
小さな開発会社を作った理由
polidog
0
1.9k
AWSでゲームサーバーを運用! Amazon GameLiftのお話
iriikeita
0
200
しくじり先生 Image Matching Challenge 2024 編
goosehaaan
0
810
Introduction to GitOps
hwchiu
0
110
さきがけから振り返るアーキテクチャ刷新 / Reflecting on the Architectural Renewal from the Vanguard
nrslib
2
770
CSC307 Lecture 13
javiergs
PRO
0
150
MIERUNE BBQにおけるユーザー中心設計()
mierune
PRO
1
110
APIのない大学ログインWebサービスをWKWebViewとJavaScriptでアプリ化した話
akidon0000
1
330
Featured
See All Featured
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
35
6.3k
Design by the Numbers
sachag
277
18k
10 Git Anti Patterns You Should be Aware of
lemiorhan
652
58k
Principles of Awesome APIs and How to Build Them.
keavy
124
16k
Rebuilding a faster, lazier Slack
samanthasiow
78
8.5k
How To Stay Up To Date on Web Technology
chriscoyier
784
250k
Fontdeck: Realign not Redesign
paulrobertlloyd
79
5.1k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
353
29k
A Tale of Four Properties
chriscoyier
155
22k
How GitHub (no longer) Works
holman
305
140k
Robots, Beer and Maslow
schacon
PRO
157
8.1k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
34
1.9k
Transcript
Ruby 3x3 ʹ͚ͨ RubyͷϕϯνϚʔΫڥͱՌ RubyΞιγΤʔγϣϯ։ൃॿ Ռใࠂձ @k0kubun
ࣗݾհ GitHub, Twitter: k0kubun Rubyίϛολ: JITίϯύΠϥͷ։ൃ, ERBͷϝϯςφϯε ࡞ͬͨ: Hamlit, MItamae,
md2key, xremap, LLRB
ࠓ͢͜ͱ 1. ຊϓϩδΣΫτԿΛղܾ͔ͨ͠ 2. ϕϯνϚʔΫϝτϦΫεͷϞσϦϯά 3. ϓϥάΠϯΞʔΩςΫνϟ 4. ϓϩδΣΫτऴྃޙͷ׆ಈ
1. ຊϓϩδΣΫτԿΛղܾ͔ͨ͠
͜Ε·ͰͷRubyͷϕϯνϚʔΫڥ
ϒϩοΫݺͼग़͠ͷΦʔόʔϔου ඪ४ͷbenchmarkϥΠϒϥϦbenchmark-ipsɺܭଌ୯Ґ͕ϒ ϩοΫʹͳ͍ͬͯΔ ϒϩοΫݺͼग़͠ී௨ͷϝιουݺͼग़͠૬ͷΦʔόʔ ϔου͕͋Δ ͦΕΑΓ࠷దԽ͞Ε໋ͨྩͷܭଌ?
ෳͷRuby࣮ؒͰͷൺֱͷ͠͞ ϒϩοΫͰಈతʹܭଌํ๏͕ࢦఆՄೳͩͱɺͦͷϒϩοΫΛ ผͷRubyΠϯλϓϦλʹͯ͠ܭଌ͢Δͷࠔ ͔͠͠Ruby 3x3ͷͨΊʹɺॲཧܥࣗମͷมߋͷޮՌΛධՁ ͍ͨ͠
ཚཱ͢ΔϕϯνϚʔΧʔ ඪ४ͷbenchmarkϥΠϒϥϦ benchmark-ips RubyϦϙδτϦͷ benchmark/driver.rb RubyBenchͷ benchmark/driver.rb ͷϑΥʔΫ VladʹΑΔMJIT-benchmarksͷ࣮ߦΤϯδϯ
ͰͲ͏͢Δ͔?
࠷͜ͷͷղܾʹ͍ۙπʔϧ ඪ४ͷbenchmarkϥΠϒϥϦ benchmark-ips RubyϦϙδτϦͷ benchmark/driver.rb RubyBenchͷ benchmark/driver.rb ͷϑΥʔΫ VladʹΑΔMJIT-benchmarksͷ࣮ߦΤϯδϯ
benchmark/driver.rb ͱ ෳRubyόΠφϦؒͰൺֱ͕Ͱ͖ɺ࣮ߦ࣌ؒͷଞʹϝϞϦ༻ྔΛܭଌͰ ͖Δ ϒϩοΫͰͳ͘whileϧʔϓΛ͏ͨΊɺܭଌͷΦʔόʔϔου͕গͳ͍ ͨͩ͠RubyϦϙδτϦͷεΫϦϓτʹಛԽͨ͠ॲཧ͕ೖ͍ͬͯΔ benchmark/driver.rb Λ͍ͯ͘͢͠օͰ͍͍͑ͷͰ? ͦΕ͕ benchmark_driver.gem
࠾͞ΕͨϓϩδΣΫτ “ΑΓ౷ҰతͰ؆୯ͳRubyͷϕϯνϚʔΫڥͷ࣮ݱ" benchmark_driver.gem ͷػೳ֦ॆ Ruby 3x3 ʹ͚ͨϕϯνϚʔΫηοτͱCIͷඋ
benchmark_driver.gem ͷಛ
1. ͋ΒΏΔϝτϦΫεΛ͋ΒΏΔํ๏Ͱग़ྗͰ͖Δ ޙड़͢ΔϓϥάΠϯػߏʹΑΓɺ༷ʑͳܭଌํ๏Λ༷ʑͳํ ๏ͰूܭՄೳ σϑΥϧτͰbenchmark-ips෩ͷܭଌɾग़ྗ ಉ͡ग़ྗͷ··ϝϞϦ༻ྔΛܭଌͨ͠ΓɺOptcarrotͷfps Λܭଌͨ͠ΓͰ͖Δ
2. ਫ਼ͷྑ͍ܭଌεΫϦϓτΛࣗಈͰੜ whileϧʔϓΛ͢ΔεΫϦϓτΛಈతʹੜ͠ɺ͔ͭwhileϧʔϓ ʹ͔͔Δ࣌ؒܭଌͯࠩ͠͠Ҿ͘ ͦͷͨΊɺܭଌରϒϩοΫͰͳ͘ඞͣจࣈྻͰ༩͑ΒΕΔ Rubyͷىಈ࣌ؒܭଌͯ͠͠·͏ΦϦδφϧͷbenchmark/ driver.rbͱҧ͍ɺੜͨ͠εΫϦϓτͰ࣌ؒͷܭଌΛ͢Δ ىಈ࣌ؒͷϒϨͷӨڹΛड͚ʹ͍͘
3. RubyʹՃ͑ɺYAMLͰ؆͔ܿͭॊೈʹܭଌ͕ߦͳ͑Δ ಈతͳϒϩοΫͰͳ͘จࣈྻͰ༩͑ΔҎ্ɺΘ͟Θ͟RubyͰϕ ϯνϚʔΫΛఆٛͤ͞Δඞཁ͕ͳ͍ YAMLͷγϯλοΫεͰϕϯνϚʔΫΛఆٛ͢Δͱ͘ॻ͚Δ .travis.yml .circleci/config.yml ͘Β͍ͷؾܰ͞Ͱஔ͍ͯΒ͍͍ͨ OptcarrotʹطʹYAML͕ஔ͞Ε͍ͯΔ
4. ෳͷRuby࣮ؒɺgemͷผόʔδϣϯؒͰൺֱͰ͖Δ RubyόΠφϦΛىಈͯ͠ܭଌ͢ΔͨΊɺෳͷRuby࣮ؒͰ ൺֱͰ͖ɺRuby 3x3ʹ࠷ద ҰRubyΛىಈͨ͠Βͦ͜Ͱಉ͡gemͷෳόʔδϣϯΛڞ ଘͤ͞Δ͜ͱෆՄೳ͕ͩɺbenchmark_driver.gemͰͦͷൺ ֱ͕Ͱ͖Δ
࣮ࡍͷ༻ྫ
Haml 4 vs Haml 5
ERB Ruby 2.4 vs Ruby 2.5
Optcarrot
VS JRuby
VS TruffleRuby
Docker Time.utc
Docker Time.utc w/ $TZ
Ruby 2.6 block.call
benchmark_driver.gemͷ։ൃ Ҏ֎ͷՌ
ϕϯνϚʔΫηοτͷ༻ҙ FluentdϕϯνϚʔΫͷࣗಈԽɾbenchmark_driverରԠ benchmark-driver/fluentd-benchmark MJIT-benchmarksͷbenchmark_driverରԠ benchmark-driver/mjit-benchmarks Watson1978͞ΜͷϕϯνϚʔΫηοτͷม benchmark-driver/ruby-method-benchmarks
benchmark-driver.github.io
benchmark-driver.github.io GitHub pages ϕϯνϚʔΫαʔόʔ cronͰruby/ruby ΛϙʔϦϯά + ϕϯνϚʔΫ middleman build
git push Travis ϕϯνϚʔΫ݁Ռͷ YAMLΛอଘ͢Δ GitHubϦϙδτϦ Πϯλʔωοτ ݁ՌͷYAMLΛ git push webhook ݟΕΔ आΓͯΔαʔόʔͷ༷Ͱ ֎෦͔ΒΞΫηεͰ͖ͳ͍
benchmark-driver.github.io benchmark-driver/ruby-builder, skybench, sky2-infra… RubyBenchͩͱϓϩδΣΫτظؒதʹ͍͡Γ͢ͷ͕͍͠ ͱߟ͑ͨͨΊɺࢼݧతʹผϓϩδΣΫτͱͯ࢝͠ಈ ࣮ࡍʹϓϩδΣΫτऴྃޙΛ͔͚ͨΒ͙͢৭ʑͳݖݶ ΛΒ͑ͨ ৄࡉޙͰ͠·͢
2. ϕϯνϚʔΫϝτϦΫεͷϞσϦϯά
มଟ͗͢ ܭଌରͷεΫϦϓτୡ ࣄલ࣮ߦεΫϦϓτͷ߹ͤ ܭଌ͢ΔϝτϦΫεɺ͋Δ͍ܭଌํ๏ Rubyͷόʔδϣϯ gemͷόʔδϣϯ
ෳϝτϦΫεͷಉ࣌ܭଌ ܭଌ͢ΔεΫϦϓτͷҰཡͱRubyͷόʔδϣϯҰཡ͚ͩͰදʹͳ͍ͬͯ ͯɺෳͷϝτϦΫεΛࠞͥΔͱਏ͍ Ͱuser, systemͷCPU࣌ؒͱ࣮࣌ؒΛಉ࣌ʹग़͢Έ͍ͨͳͷΓ͍ͨ (Ϟσϧ্ରԠ͚ͨ͠Ͳ࣮·ͩ͜Ε࣮ͯ͠ͳ͍) શͯͷϢʔεέʔεΛΧόʔͭͭ͠ഁ͍ͯ͠ͳ͍ϞσϧΛͲ͏ʹ͔ ࡞Εͳ͍͔?
v0.11~v0.12ݱࡏͷϞσϧ
v0.11~v0.12ݱࡏͷϞσϧ
v0.11~v0.12ݱࡏͷϞσϧ
ݕ౼ͨ͠ࣄ߲ HashͷΩʔඞͣΠϛϡʔλϒϧͳͷʹ͢Δ ͡Όͳ͍ͱOutputϓϥάΠϯ͕Hash͔ΒΛࢀর͢Δ࣌ʹࠔΔ Outputͷ͢͠͞ΑΓRunnerͷ࣮͕ഁ͠ͳ͍ͷΛ༏ઌͨ͠ߏ ಉ࣌ʹऔΒΕΔϝτϦΫε͕ͳΔ͍ۙ͘ॴʹདྷΔߏ Output࠷ѱ࠷ޙʹ·ͱΊͯͲ͏ʹ͔Ͱ͖ΔͨΊ
3. ϓϥάΠϯΞʔΩςΫνϟ
աڈͷϓϥάΠϯࣄྫ ϓϥάΠϯαϙʔτͷ͋Δͷ͜Ε·ͰͷϓϩδΣΫτ cookpad/barbeque itamae-kitchen/mitamae benchmark-driver/benchmark-driver
ڞ௨: ͔ࣗ͠ϓϥάΠϯॻ͍ͯͳ͍ ͔ࣗ͠ݾධՁͰbenchmark-driverͷΠϯλʔϑΣʔε͕Ұ൪ ͍ ͦͦ͋·ΓநԽʹ͍ͯͳ͍ཁ݅ͳؾ͍ͯ͠Δ ͳͷͰʹฉ͍͓͍͍ͯͯͩ͘͞
ϓϥάΠϯγεςϜͷ֓ཁ
2छྨͷϓϥάΠϯαϙʔτ RunnerϓϥάΠϯ: ܭଌํ๏ͱܭଌ͍ͨ͠ϝτϦΫεΛܾΊΔ OutputϓϥάΠϯ: ܭଌ݁ՌͷूܭΛߦͳ͏
RunnerϓϥάΠϯ Ruby API͔Β࣮ߦ͞ΕΔ͕ɺYAMLͷύʔε͜ͷRunnerϓ ϥάΠϯ͕ߦͳ͏ RunnerͷTypebenchmark-driverίϚϯυͰ༩͑ΒΕΔ͕ɺ YAMLͷதͰॻ͚ΔͷͰɺYAMLͰTypeΛݻఆͭͭ͠ಠ ࣗͷYAMLύʔαΛॻ͘ͱ͖์YAMLͷจ๏͕֦ுͰ͖ Δ
RunnerϓϥάΠϯ σϑΥϧτͷYAMLఆٛͷͷ: ips: ࣮ߦ࣌ؒ/࣮ߦճΛܭଌ͢Δ once: ςετ༻ʹ1ճ͚ͩΒͤΔ time: ࣮ߦ࣌ؒ memory: Max
Resident Set Size (ࠓLinuxͷΈαϙʔτ)
RunnerϓϥάΠϯ ಠࣗͷYAMLఆٛͷͷ: recorded: record outputΛ͏ͱੜ͞ΕΔYAMLͰɺಉ͡ܭ ଌ݁ՌΛԿग़ྗͰ͖Δ ruby_stdout: RubyίϚϯυͷstdoutΛύʔεͯ͠ϝτϦΫε ΛऔΔ
memory
memory
Optcarrot
Optcarrot
Runnerʹ͍ͭͯҙਤతʹαϙʔτΛΊͨػೳ Ruby͔Β࣮ߦ࣌ɺจࣈྻ͡Όͳͯ͘ϒϩοΫͰܭଌରΛ༩͑Δͷα ϙʔτ͍ͯͨ͠ Ҡߦύε + ෳόʔδϣϯ͕ࢦఆ͞Εͳ͚ΕevalͰΒͤͨํ͕͍ ࣮࠷ॳ͜ͷػೳ͕Runnerͱͯ͠ϓϥάΠϯαϙʔτ͞Ε͍ͯͨ(3छ ྨͷϓϥάΠϯ͕͋ͬͨ) υϥΠόͱಉ໊͡લۭؒͰΒͤΔ࣮ߦ͕ΫϦʔϯ͡Όͳ͘ɺಉ͡εΫ ϦϓτΛ࣮ߦ͢Δʹαϙʔτ͕ਏ͔ͬͨ͠ϝϦοτബ͍ͷͰΊͨ
v0.11~12ͷRunnerϓϥάΠϯͷॻ͖ํ YAMLΛநԽ͢ΔJobͱɺYAML͔ΒͦΕΛ࡞ΔJobParserΛఆٛ͢Δ #initializeʹɺCLIΦϓγϣϯΛ࣋ͭconfig, OutputΛநԽ͢Δ output, δϣϒͷϚτϦΫεΛ࡞Δcontexts͕༩͑ΒΕΔ #run ͖์࣮͍͍͕ͯ͠ɺoutputʹରͯ͠ϝιου܊Λܾ·ͬ ͨॱ൪ʹݺͳ͚ΕͳΒͳ͍ #with_warmup,
#with_benchmark, #with_job, #with_context, #report
OutputϓϥάΠϯ લड़ͨ͠ϞσϧΛड͚औͬͯूܭ͢Δׂ 1ͭܭଌ͢Δ͝ͱʹoutputʹͤΔΠϯλʔϑΣʔεͳͷͰɺ ݁Ռ͕ετϦʔϛϯάग़ྗͰ͖Δ શ෦ऴΘͬͨޙʹόϧΫͰग़ྗ͢ΔͨΊͷ؆୯ͳΠϯλʔ ϑΣʔε༻ҙ͍ͯ͠Δ
OutputϓϥάΠϯ ϏϧτΠϯͷϓϥάΠϯ compare: benchmark-ips෩ͷൺֱग़ྗ simple: benchmark/driver.rb ෩ग़ྗ markdown: GFMͷςʔϒϧग़ྗ record:
ܭଌ݁ՌΛอଘͯ͠ผͷoutputʹྲྀ͢༻
OutputϓϥάΠϯ ඇެࣜϓϥάΠϯ gruff: gruff.gemͰάϥϑ͕ग़ྗͰ͖Δ rubybench: RubyBenchʹ݁ՌΛૹ৴
Markdown
gruff
ϓϥάΠϯʹؔ͢Δҙࣄ߲ v0ͷؒԿϓϥάΠϯΠϯλʔϑΣʔε͕มΘ͍ͬͯͯɺv1ʹ͢Δ·Ͱ ֬ఆ͡Όͳ͍Ͱ͢ Runner Typeݻ༗ͷprelude͕άϩʔόϧͳcontexʹೖͬͯΔͷҾ͖ണ͕ ͢༧ఆ ϓϥάΠϯΛ࡞ͬͯͩ͘͞ΔํґଘΛ "benchmark_driver", "~> v0.X.Y"
ʹ ໘͍͚ͯͨͩ͠Δͱॿ͔Γ·͢ ಉ͡ v0.X ͷؒյ͞ͳ͍Α͏ʹ͍ͯ͠·͢
4. ϓϩδΣΫτऴྃޙͷ׆ಈ
RubyBench
RubyBenchͱ RubyͷύϑΥʔϚϯεΛࢹ͢ΔCI benchmark-driver.github.io ΑΓେલ͔Β͋Δ
RubyBenchͷΞʔΩςΫνϟ RubyBench ϫʔΧʔ task-spooler → docker run ruby … GitHub
fork for ruby/ruby RubyBench web Rails ΞϓϦέʔγϣϯ → Sidekiq GitHub ruby/ruby cronͰߋ৽ Webhook ssh tsp …
͜͜Λஔ͖͍͑ͨ RubyBench ϫʔΧʔ task-spooler → docker run ruby … GitHub
fork for ruby/ruby RubyBench web Rails ΞϓϦέʔγϣϯ → Sidekiq GitHub ruby/ruby cronͰߋ৽ Webhook ssh tsp …
RubyBenchͷ౷߹ JITͱͷൺֱάϥϑޙ͔ΒRubyBenchʹ࣮͞Εͨ όάͬͯͨͷͰͨ͠Γͨ͠ benchmark_driver͚ͷϕϯνϚʔΫηοτਫ਼ͷྑ͍ܭଌػೳ͕ࠓޙ׆༻͠ ͍͢Α͏ɺϕϯνϚʔΧʔͷbenchmark_driverͷஔ͖͑Λ͍ͬͯΔ طʹOptcarrotbenchmark_driverͰܭଌ͞Ε͍ͯΔ ॏෳ͕ͪ͠ͳ࣮͕ϓϥάΠϯͰநԽͰ͖͓ͯΓbenchmark_driverͷϓϥά Πϯػߏ͕׆༂͍ͯ͠Δ
RubyBench͚ʹɺϕϯνϚʔΫͷ֦ॆ͍͖ͬͯ·͢ VladͷMJIT͚ϕϯνϚʔΫूΛՃ༧ఆ Watson1978͞Μͷ֤ϝιου͚ϕϯνϚʔΫूΛมͨ͠ ͷΘ͍͔ͤͯͨͩ͘͞? https://github.com/benchmark-driver/ruby-method-benchmarks
·ͱΊ benchmark_driver ศརͰ͢ͷͰɺੋඇ͝ར༻͍ͩ͘͞ RubyͷߴԽRubyBenchʹطʹຊՌ͕׆༻͞Ε͍ͯ·͢
ँࣙ ຊϓϩδΣΫτɺాߞҰ༷Λ࢝Ίͱͨ͠RubyΞιγΤʔ γϣϯ༷ͷ͝ڠྗʹΑ࣮ͬͯݱ͢Δ͜ͱ͕Ͱ͖·ͨ͠ɻ͜ͷ ΛआΓͯޚྱਃ্͛͠·͢ɻ