Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
Rubyで音を視る
Search
Sponsored
·
SiteGround - Reliable hosting with speed, security, and support you can count on.
→
ydah
June 06, 2026
Technology
770
1
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Rubyで音を視る
松江Ruby会議12「Rubyで音を視る」の発表スライド
https://matsue.rubyist.net/matrk12/
#matrk12
ydah
June 06, 2026
Transcript
66 slides
2026/06/06 Ruby 12 Ruby seeing sound with ruby Yudai Takada,
ydah
None
None
None
None
None
Otsu City Traditional Performing Arts Hall Saturday, July 18, 2026
None
None
None
None
None
None
2026/06/06 Ruby 12 Ruby seeing sound with ruby Yudai Takada,
ydah
None
None
vizcore https://github.com/ydah/vizcore
Vizcore visualize( ) × core( ) Audio-reactive visuals Ruby DSL
Ruby scene renderer
2D/3D OpenGL
None
None
None
None
1
3
PCM samples Mic / File samples Array<Float> [ 0.0012, 0.0031,
-0.0028, . . . ]
None
Sample Window tick window 1024 60fps tick sample 圧
Samples RMS amplitude Sample RMS 夏 圧 0 [0.5, -0.5,
0.5, -0.5] (0.5 + -0.5 + 0.5 + -0.5) / 4 = 0 RMS = 1 N N ∑ i=1 x2 i
Samples FFT magnitudes / peak_frequency FFT 圧 WN = e−2πi/N
Ek = N/2−1 ∑ m=0 x2m Wkm N/2 Ok = N/2−1 ∑ m=0 x2m+1 Wkm N/2 Xk = Ek + Wk N Ok Xk+N/2 = Ek − Wk N Ok (k = 0,1,…, N 2 − 1)
magnitudes BandSplitter sub / low / mid / high FFT
Vizcore sub, low, mid, high sub: 20Hz . . 60Hz low: 60Hz . . 250Hz mid: 250Hz . . 4000Hz high: 4000Hz . . 20000Hz
Samples frame energy BeatDetector beat FFT beat beat frame 圧
E = 1 N N ∑ i=1 x2 i
beat beat_pulse beat boolean frame true 1 frame beat 圧
1.0 beat frame 0.86 長
beat history BPMEstimator bpm / beat_phase / bar_phase BPM beat
frame lag BPM beat beat beat = beat_pulse = bpm = beat_phase = bar_phase = bpm = 60.0 * frame_rate / best_lag
bands + onset drums con fi dence 圧 band onset
kick = sublow_strength * onset # sub/low ͷڧ͞ × onset snare = mid_strength * onset # mid ͷڧ͞ × onset hihat = high_strength * onset # high ͷڧ͞ × onset
magnitudes / spectrum spectral features FFT Spectral features FFT 圧
amplitude bands spectral features centroid: / rollo ff : / fl atness: / fl ux: /
raw features smoother audio features frame を EMA EMAt =
αxt + (1 − α)EMAt−1 amplitude: @sm.smooth(:amplitude, normalized[:amplitude]) bands: @sm.smooth_hash(normalized[:bands], namespace: :bands) fft: @sm.smooth_array(normalized[:fft], namespace: :fft)
amplitude bass / low high / treble beat beat_pulse bpm
/ phase spectral_* / / / / / を 長
None
audio features MappingResolver layer params map beat_pulse, to: :radius, gain:
160.0, min: 56, max: 164, attack: 1.0, release: 0.2
beat beat_pulse map beat_pulse, to: :radius, gain: 100, min: 50,
max: 150
beat map beat_pulse, to: :radius, gain: 100, min: 50, max:
150 100
beat map beat_pulse, to: :radius, gain: 100, min: 50, max:
150 50 150
beat map beat_pulse, to: :radius, gain: 100, min: 50, max:
150 radius
layer params audio_frame audio scene layers audio frame { “audio”:
{ "amplitude": 0.42, "beat": true, "beat_pulse": 1.0, "bands": { "low": 0.8 } }, "scene": { "name": "main", "layers": [ { "name": "rings", "params": { "radius": 140 } …
audio_frame WebSocket frontend WebGL renderer Frontend audio_frame Ruby Runtime Browser
Renderer (WebGL Runtime) Web Socket
6/4
BGM
None
None
50 6 4
2026/06/06 Ruby 12 Ruby seeing sound with ruby Yudai Takada,
ydah
圧 母 /
母
母 母 F1 / F2 spectral centroid low / mid
/ high ratio
F1/F2 母 F 圧 圧 : : : / :
:
母 0.0 1.0 1 F1 F2 母
None
/ / 圧 圧 burst fl atness ZCR nasal high
ratio
burst amplitude_delta = [ current[:amplitude] - previous[:amplitude], # લϑϨʔϜ͔ΒͷԻྔ૿Ճ 0.0
].max burst_strength = [ current[:onset], # Իͷ্ཱ͕ͪΓ current[:spectral_flux], # εϖΫτϧมԽྔ current.dig(:onsets, :mid), # தҬͷ্ཱ͕ͪΓ current.dig(:onsets, :high), # ߴҬͷ্ཱ͕ͪΓ amplitude_delta, # Իྔͷ૿Ճྔ recent_onset_max # ۙ૭ͷ࠷େ্ཱ͕ͪΓ ].max
spectral fl atness s_score = weighted_average([ high_ratio, # ߴҬͷڧ͞ flatness,
# ϊΠζͬΆ͞ zcr, # ܗͷࡉ͔͍༳Ε centroid # Իͷ໌Δ͞ ]) n_score = weighted_average([ nasal_score, # ඓԻͬΆ͞ 1.0 - high_ratio, # ߴҬ͕গͳ͍ low_mid_ratio, # ʙதҬͷڧ͞ 1.0 - zcr # ༳Ε͕গͳ͍ ])
attack feature attack = { strength: attack[:attack_strength], # ग़ͩ͠ͷڧ͞ centroid:
attack[:centroid_norm], # ग़ͩ͠ͷ໌Δ͞ high: attack[:high_ratio], # ߴҬͷڧ͞ low: attack[:low_ratio], # Ҭͷڧ͞ flatness: attack[:spectral_flatness],# ϊΠζͬΆ͞ zcr: attack[:zero_crossing_rate], # ࡉ͔͍༳Ε f1 : attack[:f1_frequency], # F1 ͬΆ͍ࢁ f2 : attack[:f2_frequency], # F2 ͬΆ͍ࢁ nasal: attack_nasal_score # ඓԻͬΆ͞ }
母 × kana_score = [ vowel_score * 0.55, # ԻΒ͠͞
consonant_score * 0.35, # ࢠԻΒ͠͞ burst_bonus * 0.10 # ग़ͩ͠ͷิਖ਼ ].sum candidate = { text: kana_table[consonant][vowel], # ͔ͳจࣈ vowel: vowel, # ਪఆԻ consonant: consonant, # ਪఆࢠԻ conf i dence: kana_score # ީิͷࣗ৴ }
None
vizcore Ruby ォ Ruby
$whoami ォ ydah ( ) SmartHR Ruby 西 Ruby Ruby
( 西 ) 西 圧 OSS 2
More Decks by ydah
See All by ydah
gem install とrequire のむこうがわ
ydah
0
260
わからない話を追いかけたら、プログラミング言語を作る側にいた
ydah
3
1.2k
Liberating Ruby's Parser from Lexer Hacks
ydah
2
4.5k
ルールルルルルRubyの中身の予備知識 ── RubyKaigiの前に予習しなイカ?
ydah
1
1.1k
計算機科学をRubyと歩む 〜DFA型正規表現エンジンをつくる~
ydah
3
1.1k
進捗
ydah
2
370
Joy with 3D Graphics Using Ruby
ydah
1
190
構文解析器入門
ydah
7
3.9k
Regional.rb and the Kyoto City
ydah
0
110
Other Decks in Technology
See All in Technology
実測データからサービスの成長余力を測る。負荷シナリオ・テストデータ・限界値の設計
kazatohiei
0
170
Terraform 0 to 100:100個以上のリポジトリを常に最新にし続けるための自動化
sue445
3
1.4k
AI-Ready なデータ基盤ってなに? & 新卒未経験が、いま考えていること
imuta
0
1.1k
専門家に使ってもらうAIプロダクトをつくる - 病院の医事課向けDPCコーディング支援システムの設計と評価
hagino3000
4
1k
市販IoT機器や自作IoT機器とHomeAssistantを駆使して家の電気代をできる限り見える化する方法の考案
harutiro
0
420
Amazon Bedrock AgentCore の基礎とセキュリティ・ガバナンス
ren8k
0
270
技術書典で本を売る流れ
yoshimura_datam
0
130
統制されたID・権限をどう利用するか : DPoP/FAPIによるAPIセキュリティ強化
oidfj
2
300
MCPでネットワーク障害調査を自動化する前に直すべき3つのこと
yama3133
1
660
コーディングできない僕にBobが舞い降りた
shumpei3
0
200
認知の質も文化醸成も。数年取り組んで見えてきた技術広報のKPI設計 / Tech PR KPIs: Lessons on Awareness Quality and Culture After Several Years
bitkey
PRO
0
210
Dark Web - Como o Cibercrime Opera na Internet
nicholsjasper
0
110
Featured
See All Featured
Build your cross-platform service in a week with App Engine
jlugia
234
19k
AI Search: Where Are We & What Can We Do About It?
aleyda
0
8.3k
SEO for Brand Visibility & Recognition
aleyda
0
4.9k
How People are Using Generative and Agentic AI to Supercharge Their Products, Projects, Services and Value Streams Today
helenjbeal
2
410
HDC tutorial
michielstock
2
1k
Test your architecture with Archunit
thirion
2
2.5k
Unsuck your backbone
ammeep
672
58k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
31
10k
Leading Effective Engineering Teams in the AI Era
addyosmani
9
2.9k
B2B Lead Gen: Tactics, Traps & Triumph
marketingsoph
1
310
DevOps and Value Stream Thinking: Enabling flow, efficiency and business value
helenjbeal
2
450
What does AI have to do with Human Rights?
axbom
PRO
1
2.5k