Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Go and on-demand video streaming / Goとオンデマンド動画配信
Search
Seiji Takahashi
December 13, 2017
Programming
4
2.9k
Go and on-demand video streaming / Goとオンデマンド動画配信
Lecture at university of Aizu / 2017/12/10
Seiji Takahashi
December 13, 2017
Tweet
Share
More Decks by Seiji Takahashi
See All by Seiji Takahashi
Go Backends for frontends with GraphQL and gRPC
timakin
6
4.1k
Design Pattern for Image and Text Composition in Go
timakin
5
6.7k
Golang API Testing the HARD way
timakin
13
6.7k
Head First Golang Image Package
timakin
2
10k
React Native Beyond Prototype
timakin
2
1.7k
Performance Optimization on Google AppEngine
timakin
5
6.4k
testcache.pdf
timakin
1
160
How Go cache
timakin
1
97
How Go cache tests
timakin
1
3.2k
Other Decks in Programming
See All in Programming
ИИ-Агенты в каждый дом – Алексей Порядин, PythoNN
sobolevn
0
140
Back to the Future: Let me tell you about the ACP protocol
terhechte
0
120
Platformに“ちょうどいい”責務ってどこ? 関心の熱さにあわせて考える、責務分担のプラクティス
estie
2
520
GraphQL×Railsアプリのデータベース負荷分散 - 月間3,000万人利用サービスを無停止で
koxya
1
1k
Reactをクライアントで使わない
yusukebe
7
6.3k
10年もののAPIサーバーにおけるCI/CDの改善の奮闘
mbook
0
650
株式会社 Sun terras カンパニーデック
sunterras
0
170
複雑化したリポジトリをなんとかした話 pipenvからuvによるモノレポ構成への移行
satoshi256kbyte
1
690
GitHub Actions × AWS OIDC連携の仕組みと経緯を理解する
ota1022
0
220
CSC509 Lecture 03
javiergs
PRO
0
320
Pythonスレッドとは結局何なのか? CPython実装から見るNoGIL時代の変化
curekoshimizu
4
1.1k
開発者への寄付をアプリ内課金として実装する時の気の使いどころ
ski
0
330
Featured
See All Featured
A Tale of Four Properties
chriscoyier
160
23k
Context Engineering - Making Every Token Count
addyosmani
3
160
Bash Introduction
62gerente
615
210k
Optimizing for Happiness
mojombo
379
70k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
52
5.6k
Unsuck your backbone
ammeep
671
58k
Java REST API Framework Comparison - PWX 2021
mraible
33
8.8k
Agile that works and the tools we love
rasmusluckow
330
21k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
18
1.2k
Navigating Team Friction
lara
189
15k
Why You Should Never Use an ORM
jnunemaker
PRO
59
9.5k
Large-scale JavaScript Application Architecture
addyosmani
513
110k
Transcript
Goͱ ΦϯσϚϯυಈը ৴ج൫ @__timakin__
• Seiji Takahashi • Github: timakin / Twitter: @__timakin__ •
גࣜձࣾσΟʔɾΤψɾΤʔ • Perl / JavaScript • גࣜձࣾτϥϯεϦϛοτ • C++ (Cocos-2dx) • גࣜձࣾGunosy ৽نࣄۀ։ൃࣨ • Go / Swift ࣗݾհ
גࣜձࣾGunosy ৽نࣄۀ։ൃࣨʹ͍ͭͯ
Gunosyͷ5ʙ10ޙͷΛ࡞Δ͘ɺ VRARɺԻUIͳͲࠓޙීٴ͢ΔՄೳੑͷ͋Δ ৽͍͠σόΠεٕज़Λݚڀ͠ɺ ৽نαʔϏεͷ্ཱͪ͛ͳͲΛߦ͏෦ॺ
Ϧαʔν • ւ֎ͷઌਐࣄྫͷ·ͱΊ • SDKͷެࣜDocsGithub repoͷ ίʔυΛړͬͯ࠷৽ٕज़ͷ νϡʔτϦɾԠ༻ྫΛ୳Δ • SlackʹRSS௨ɺϨϙʔτڞ༗
Δ͜ͱ
։ൃ • ͓ன͍ͭͰʹνϡʔτϦΞϧ ϋοΧιϯ • ্ཱ͕ͪΓظ ۃྗখنνʔϜͰ Δ͜ͱ
ྫ
ྫ
ษڧձ։࠵
ຊͷΞδΣϯμ
GoͱΦϯσϚϯυಈը৴ج൫ • ಈը৴ͱʁ • ৴·ͰͷྲྀΕ • GoΛͲ͏͏͔ • ·ͱΊ ΞδΣϯμ
ΞδΣϯμ GoͰ࡞ΔΦϯσϚϯυಈը৴ج൫ • ಈը৴ͱʁ • ৴·ͰͷྲྀΕ • GoΛͲ͏͏͔ • ·ͱΊ
େ͖͚ͯ͘ ̎ छྨ ಈը৴ͱʁ
ϦΞϧλΠϜ৴ • ͍ΘΏΔϥΠϒಈը • WebSocketͷ ίωΫγϣϯͳͲΛ ௨ͯ͡ϒϩʔυΩϟετ ಈը৴ͷछྨ
ΦϯσϚϯυ৴ • ͋Β͔͡Ί࡞͞Εͨ ಈըΛݟΔ • ಈը͕ඞཁʹͳͬͨ λΠϛϯάͰࣗ༝ʹ ࢹௌͰ͖Δ ಈը৴ͷछྨ
GoͰ࡞ΔΦϯσϚϯυಈը৴ج൫ • ಈը৴ͱʁ • ৴·ͰͷྲྀΕ • GoΛͲ͏͏͔ • ·ͱΊ ΞδΣϯμ
ಈը৴ʹඞཁͳ࡞ۀ Ξοϓϩʔυ ө૾ॲཧ ύοέʔδԽ ৴
ಈը৴ʹඞཁͳ࡞ۀ Ξοϓϩʔυ ө૾ॲཧ ύοέʔδԽ ৴ ετϨʔδ ৴ऀ AWSͷS3 Bucket GCPͷCloud
Storage ͳͲʹΞοϓϩʔυ
ಈը৴ʹඞཁͳ࡞ۀ Ξοϓϩʔυ ө૾ॲཧ ύοέʔδԽ ৴ Τϯίʔμʔ ετϨʔδ ࣈນૠೖೖྗө૾ͷ ΤϯίʔσΟϯάʹՃ͑ͯɺ αϜωΠϧͷΓऔΓͳͲ
ಈը৴ʹඞཁͳ࡞ۀ Ξοϓϩʔυ ө૾ॲཧ ύοέʔδԽ ৴ ετϨʔδ ಈըΛׂͯ͠ ޮతʹ৴Մೳͳ ϑΝΠϧΛੜ͢Δɺ ҉߸Խ͢Δ
ύοέʔδϯά"1*
ಈը৴ʹඞཁͳ࡞ۀ Ξοϓϩʔυ ө૾ॲཧ ύοέʔδԽ ৴ ৴"1* $%/ ετϨʔδ ड৴ऀ HTTPΫϥΠΞϯτʹରͯ͠
CDNΛ௨ͭͭ͠৴
શମ૾ ετϨʔδ ৴ऀ ө૾ॲཧɺ ύοέʔδϯά ৴"1* $%/ ड৴ऀ
GoͱΦϯσϚϯυಈը৴ج൫ • ಈը৴ͱʁ • ৴·ͰͷྲྀΕ • GoΛͲ͏͏͔ • ·ͱΊ ΞδΣϯμ
Ξοϓϩʔυ ө૾ॲཧ ύοέʔδԽ ৴ Go͕ؔΘΔͱ͜Ζ ʁ
Go͕ؔΘΔͱ͜Ζ ө૾ॲཧ • ఏڙը࣭ʹ߹ΘͤͨϑϨʔϜ/ϏοτϨʔτͷௐ • ϩεΛ͑ͨѹॖ ύοέʔδԽ • m3u8 +
TSϑΝΠϧͷੜ • ετϨʔδͷΞοϓϩʔυ ৴ • ಈըͷJSONΛฦ͢௨ৗͷAPIػೳ • αϜωΠϧؔ࿈ಈըੜ
Կߟ͑ͳ͍৴ mp4
Կߟ͑ͳ͍৴ mp4 Ϋιॏ͍
HLS৴ • HTTP Live Streaming • Apple͕iOS͚ʹ։ൃͨ͠ετϦʔϛϯάϓϩτίϧ • WindowsͳͲ࠶ੜڥ͕ݶఆ •
ϫʔΫϑϩʔ • ಈըίϯςϯπͷΞοϓϩʔυ • ηάϝϯτԽ • ಉ࣌ʹαϜωΠϧੜ৴࣭ผͷੜͳͲ • IndexFile(m3u8)Λ৴ͯ͠ɺΫϥΠΞϯτ ΠϯσοΫε͞ΕͨMPEG-2 TSϑΝΠϧΛಡΈʹߦ͘
HLSͷத segmented media files (MPEG-2 TS) Index file (m3u8)
HLS৴ͷྲྀΕ NV )JHI NV .JE NV -PX
؆୯ʹHLSΤϯίʔυ͢Δํ๏ ffmpeg Elastic Transcoder
؆୯ʹHLSΤϯίʔυ͢Δํ๏ ffmpeg Elastic Transcoder
ffmpeg • ಈըͱԻΛهɾมɾ࠶ੜ͢ΔͨΊͷϑϦʔιϑτ ΣΞ • ffmpegΛར༻ͨ͠Τϯίʔυɾσίʔυ͕Ͱ͖Δ ༻ιϑτΣΞஶ࡞ݖ৵ΛሃΉ…? https://blog.tai2.net/mpegla_and_ffmpeg.html
ffmpeg-based API • Docker multi-stage buildͰੜ͞ΕͨimageΛݩʹɺ execܦ༝ͰffmpegΛݺͿɻ • binding࣮͔ίϚϯυͷexecuteͰݺͿ΄͔ͳ͍ɻ •
ݸਓతʹGKEͰ্هimageΛݩʹͨ͠ίϯςφΛཱͯ ͯɺੜ݁ՌΛGoogle Cloud StorageʹΞοϓϩʔυ͢ ΔͳͲ͕ྑ͍ͱࢥͬͯΔɻ
ffmpeg-based API media file Kubernetes Engine Cloud Storage goroutine
ffmpeg encoding
ffmpeg encoding
Encoding command mp4ΛAACɺH.264ͰΤϯίʔυ͠ɺ 4ඵ͝ͱͷηάϝϯτʹׂͨ݁͠ՌΛอଘ͢Δίʔυɻ ಉҰσΟϨΫτϦʹm3u8ͱtsϑΝΠϧ͕ੜ͞ΕΔɻ
Exec Commands ffmpegͷbinding࣮(goav)Λ͏ͷ͋Γɻ ͨͩɺॊೈͳΦϓγϣϯઃఆͳͲΛؑΈΔͱffmpegΛ ͦͷ··࣮ߦͨ͠ํ͕ྑ͍ɻ
ffmpeg bindingΛΘͳ͍߹۪ʹݺͿ͔͠ͳ͍ Exec Commands
goroutineΛར༻ͯ͠ੜ͞Εͨ ϑΝΠϧΛฒྻΞοϓϩʔυ Parallel upload
ffmpeg-based APIͷ·ͱΊ • αʔόʔʹΠϯετʔϧ͞ΕͨffmpegΛར༻͢Δɻ • GKEͳͲΛ͍ͭͭɺDocker multi-stage buildͰੜ ͞ΕͨimageΛݩʹɺexecܦ༝ͰffmpegΛݺͿํ๏ ͋Γɻ
• binding࣮ΑΓίϚϯυͷexecuteΛ͓͢͢Ίɻ
؆୯ʹHLSΤϯίʔυ͢Δํ๏ ffmpeg Elastic Transcoder
Elastic Transcoder • AWS্ͷϚωʔδυಈըΤϯίʔμʔ • ίϯςϯπͷ࠶ੜ࣌ؒΛϕʔεʹ՝ۚ͞ΕΔ • ੜ݁ՌS3ΦϒδΣΫτͱͯ͠อଘ͞ΕΔ
Elastic Transcoder input bucket media file Lambda Transcoder output bucket
Transcoding Pipeline
Elastic Transcoder input bucket media file Lambda Transcoder output bucket
Transcoding Pipeline
None
• goͰlambdaؔΛॻ͚ΔϥΠϒϥϦ • όΠφϦͱͦΕΛݺͼग़͢index.jsΛੜͯ͠ɺ Node.jsϥϯλΠϜͰ࣮ߦ͢Δ go-apex
Elastic Transcoder input bucket media file Lambda Transcoder output bucket
Transcoding Pipeline
S3 Observation
Elastic Transcoder input bucket media file Lambda Transcoder output bucket
Transcoding Pipeline
Parameter Tuning
Elastic Transcoder ·ͱΊ • go-apexʹΑΔS3ࢹͱมॲཧ • ࠷ۙLambda͕σϑΥϧτͰGoରԠ͢Δతͳൃදͨ͠ ͠ݟͨ·ͬͨΒͦͬͪʹ͠·͠ΐ͏ɻ • ϚωʔδυͰ৭ʑͬͯ͘ΕΔɺׂසͳͲͷ
ύϥϝʔλʔνϡʔχϯάʹҙ͠·͠ΐ͏ɻ
ʕ ◔ϖ◔ʔ < Thank You!!