Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
JPEG streaming in Go
Search
Harukasan
PRO
March 14, 2017
Technology
2
2.6k
JPEG streaming in Go
pixiv Night #02 - 画像処理技術(go, blender, C++ライブラリ等)
2017-03-14
https://pixiv.connpass.com/event/50284/
Harukasan
PRO
March 14, 2017
Tweet
Share
More Decks by Harukasan
See All by Harukasan
pixivを支える技術 / 技育CAMPアカデミア
harukasan
PRO
3
440
20240401 新卒研修 - ピクシブにおける技術領域
harukasan
PRO
1
760
ピクシブのコンテンツ配信基盤技術 / pixiv TECH SALON
harukasan
PRO
5
5.5k
Goにおける画像ファイル処理 / golang.tokyo #19
harukasan
PRO
7
6.6k
WebRTC動画をトランスコードする / Transcoding video streams from WebRTC
harukasan
PRO
5
1.5k
ImageFluxを支えるリモート開発 / 20171202
harukasan
PRO
2
1.8k
YAPC::Fukuoka 前夜祭LT / Yet Another Pawoo Commit logs
harukasan
PRO
0
3k
YAPC::Fukuoka lunch session
harukasan
PRO
1
3k
マストドン会議: Pawoo / Mastodon Kaigi2
harukasan
PRO
2
470
Other Decks in Technology
See All in Technology
MLflowはどのようにLLMOpsの課題を解決するのか
taka_aki
0
180
OCI IAM Identity Domains Entra IDとの認証連携設定手順 / Identity Domain Federation settings with Entra ID
oracle4engineer
PRO
1
1.4k
困難を「一般解」で解く
fujiwara3
9
3.1k
Autonomous Database サービス・アップデート (FY25)
oracle4engineer
PRO
1
500
20250309 無冠のわたし これからどう先生きのこれる?
akiko_pusu
11
1.7k
早くて強い「リアルタイム解析基盤」から広げるマルチドメイン&プロダクト開発
plaidtech
PRO
1
180
事業を差別化する技術を生み出す技術
pyama86
4
1.2k
プルリクエストレビューを終わらせるためのチーム体制 / The Team for Completing Pull Request Reviews
nekonenene
4
2.1k
テクスチャ画像付きのメッシュモデルを3次元点群へ変換する
kentaitakura
1
120
LangGraph × Bedrock による複数の Agentic Workflow を利用した Supervisor 型のマルチエージェントの実現/langgraph-bedrock-supervisor-agent
ren8k
4
460
プロダクト開発者目線での Entra ID 活用
sansantech
PRO
0
290
DevinでAI AWSエンジニア製造計画 序章 〜CDKを添えて〜/devin-load-to-aws-engineer
tomoki10
0
280
Featured
See All Featured
Making the Leap to Tech Lead
cromwellryan
133
9.1k
The Cost Of JavaScript in 2023
addyosmani
47
7.5k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
21
2.5k
A better future with KSS
kneath
238
17k
Designing for Performance
lara
605
68k
Into the Great Unknown - MozCon
thekraken
35
1.7k
Stop Working from a Prison Cell
hatefulcrawdad
268
20k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
356
29k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
44
7.1k
Large-scale JavaScript Application Architecture
addyosmani
511
110k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
233
17k
Gamification - CAS2011
davidbonilla
80
5.2k
Transcript
JPEG streaming in Go !IBSVLBTBO4IVOTVLF.JDIJJ ͋Δ͍+1&(ͷϒϩοΫϊΠζͱͳʹ͔
Harukasan Shunsuke MICHII Lead engineer of pixiv Inc., Product manager
of ImageFlux, Infrastructure engineer, Kibana, Gopher
None
© pixiv Inc., SAKURA Internet Inc. ImageFlux service ͓٬༷ετϨʔδ ίϯςϯπΩϟογϡ
ը૾มॲཧ ΦϦδφϧը૾Λऔಘ มޙΩϟογϡΛ৴ ΤϯυϢʔβ ։ൃऀ ৴URLͷઃఆ ৴ݩΦϦδϯͷઃఆ ϨϙʔτͷӾཡ طଘετϨʔδΛͦͷ··ར༻Մೳ ը૾σʔλΛطଘͷετϨʔδαʔό͔ ΒҠಈ͢Δඞཁ͋Γ·ͤΜɻཧը໘͔ ΒΦϦδϯΛઃఆ͢Δ͚ͩͰ͙͢ʹը૾ม ͕ར༻ՄೳͰ͢ɻ URLΛม͑Δ͚ͩͰը૾ม ৴URLʹύϥϝʔλΛՃ͢Δ͚ͩͰ ҙͷը૾มΛߦ͑·͢ɻ εϚʔτϑΥϯɺPCͰผʑͷURLΛ͏͜ ͱͰɺ1ຕͷը૾͔Β༷ʑͳαΠζͷը૾ Λ৴͢Δ͜ͱ͕ՄೳͰ͢ɻ ը૾มॲཧ
© pixiv Inc., SAKURA Internet Inc. ImageFlux service ͓٬༷ετϨʔδ ίϯςϯπΩϟογϡ
ը૾มॲཧ ΦϦδφϧը૾Λऔಘ มޙΩϟογϡΛ৴ ΤϯυϢʔβ ։ൃऀ ৴URLͷઃఆ ৴ݩΦϦδϯͷઃఆ ϨϙʔτͷӾཡ طଘετϨʔδΛͦͷ··ར༻Մೳ ը૾σʔλΛطଘͷετϨʔδαʔό͔ ΒҠಈ͢Δඞཁ͋Γ·ͤΜɻཧը໘͔ ΒΦϦδϯΛઃఆ͢Δ͚ͩͰ͙͢ʹը૾ม ͕ར༻ՄೳͰ͢ɻ URLΛม͑Δ͚ͩͰը૾ม ৴URLʹύϥϝʔλΛՃ͢Δ͚ͩͰ ҙͷը૾มΛߦ͑·͢ɻ εϚʔτϑΥϯɺPCͰผʑͷURLΛ͏͜ ͱͰɺ1ຕͷը૾͔Β༷ʑͳαΠζͷը૾ Λ৴͢Δ͜ͱ͕ՄೳͰ͢ɻ ը૾มॲཧ
Decoding a JPEG image Resizing Encoding into a new JPEG
image Fetching from an origin Writing content into a client http.Transport libjpeg-turbo libswscale from ffmpeg http.ResponseWriter libjpeg-turbo
streaming in Go • io.Reader: reading content into the given
buffer • io.Writer: writing content from the given buffer typedef Reader interface { func Read(p []byte) (n int, err error) } typedef Writer interface { func Write(p []byte) (n int, err error) }
• Copying content from a reader into a writer io.Copy(dst,
src) 32KB buffer io.Reader io.Writer Read(buffer) Write(buffer) : 32KBͷόοϑΝ෦Ͱຖճmake
• Copying content from a reader into a writer •
To avoid allocating a buffer, use io.CopyBuffer: io.Copy(dst, src) 32KB buffer io.Reader io.Writer Read(buffer) Write(buffer) io.CopyBuffer(dst, src, buf) : 32KBͷόοϑΝ෦Ͱຖճmake
func ServeHTTP(w *http.ResponseWriter, r *http.Request) { req, _ := http.NewRequest("GET",
"http://www.example.com", nil) resp, _ := http.DefaultTransport.RoundTrip(req) io.CopyBuffer(w, resp.Body, buf) } HTTPίϐʔ͢Δ͚ͩͰετϦʔϛϯάͰ͖ΔʢΤϥʔॲཧ……ʣ
Decoding a JPEG image Resizing Encoding into a new JPEG
image Fetching from an origin Writing content into a client http.Transport libjpeg-turbo libswscale from ffmpeg http.ResponseWriter libjpeg-turbo
Streaming image scanlines • ScanlineReader interface typedef ScanlineReader interface {
func Config() *Config → plane sizes, pixel format,... func ReadScanLine(p [][]uint8) (n int, err error) → reading scanlines of planes into each buffer → PNG: JPEG: } R G B Y Cb Cr *blue difference chroma *red difference chroma
Decoding a JPEG image Resizing Encoding into a new JPEG
image Fetching from an origin Writing content into a client http.Transport libjpeg-turbo libswscale from ffmpeg http.ResponseWriter libjpeg-turbo
Decoding a JPEG image Resizing Encoding into a new JPEG
image Fetching from an origin Writing content into a client http.Transport Decoder.ReadScanLines Scaler.ReadScanLines http.ResponseWriter Encoder.ReadScanLines
func ServeHTTP(w *http.ResponseWriter, r *http.Request) { req, _ := http.NewRequest("GET",
"http://www.example.com", nil) resp, _ := http.DefaultTransport.RoundTrip(req) dec := jpeg.NewDecoder(resp.Body io.Reader) → ScanlineReader dec := scale.NewResize(dec ScanlineReader) → ScanlineReader enc := jpeg.NewEncoder() enc.Write(w io.Writer, dec ScanlineReader) }
streaming image scanlines • ScanlineReader interfaceΛ͏͜ͱͰը૾σʔλετϦʔϜͱ͠ ͯѻ͏͜ͱ͕Ͱ͖Δ • interfaceΛ͏͜ͱͰ֤ॲཧͷར༻ํ๏Λ౷ҰͰ͖Δ •
ඪ४ϥΠϒϥϦͷΠϯλʔϑΣʔεΑ͘ߟ͑ΒΕͯΔͷͰϚω͢Δ ͱྑ͍ײ͡ʹͳΔ
ͳͥετϦʔϛϯά͢Δඞཁ͕͋Δͷ͔ • ը૾σʔλΛ͍ͬΜʹॲཧ͢ΔͱશըૉͷϝϞϦ͕ඞཁ͕ͩ ετϦʔϜॲཧ͢ΕόοϑΝΛҰ֬อ͢Δ͚ͩͰࡁΉ • ֤৭ۭؒຖʹॲཧΛهड़Ͱ͖ɺແବͳ৭ม͕ൃੜ͠ͳ͍ JPEGಉ࢜ͷϦαΠζͷ߹Y'CbCr৭ۭؒͷ··ѻ͏͜ͱͰɺ RGBͷมʹΑΔϩε͕ൃੜ͠ͳ͍ • cgoͰϒϩοΫ͢Δ࣌ؒΛ͘͢Δ͜ͱ͕Ͱ͖Δ
Streaming JPEG scanlines typedef ScanlineReader interface { func Config() *Config
→ plane sizes, pixel format,... func ReadScanLine(p [][]uint8) (n int, err error) → JPEG: } Y Cb Cr *blue difference chroma *red difference chroma ͭ·Γɺ͜ͷΠϯλʔϑΣʔεͰJPEGΛσίʔυ͢ΕJPEGΛετϦʔϜॲཧͰ͖Δ
color conversion chroma subsampling huffman coding quantization DCT transformation ReadScanline
io.Reader RGB→Y'CbCrͷ৭ۭؒม ΫϩϚαϒαϯϓϦϯάॲཧ DCTม ྔࢠԽ ϋϑϚϯѹॖ
8x8ϐΫηϧ͝ͱʹDCTʢࢄίαΠϯมʣͰ ྲྀͱप͝ͱͷަྲྀʹม →ߴप΄Ͳߥ͘ͳΔΑ͏ʹྔࢠԽ͢Δ͜ͱͰใྔΛݮ →ϋϑϚϯූ߸Խͯ͠ѹॖ ΊͬͪΌࡶʹJPEGͷѹॖΛઆ໌͢Δͱ ߴप प
8x8ϐΫηϧ͝ͱʹDCTʢࢄίαΠϯมʣͰ ྲྀͱप͝ͱͷަྲྀʹม →ߴप΄Ͳߥ͘ͳΔΑ͏ʹྔࢠԽ͢Δ͜ͱͰใྔΛݮ →ϋϑϚϯූ߸Խͯ͠ѹॖ ΊͬͪΌࡶʹJPEGͷѹॖΛઆ໌͢Δͱ ߴप प
• JPEGͰ8x8ϐΫηϧʢ1ϒϩοΫʣ͝ͱʹDCTɺྔࢠԽΛߦ͏ • MCU: 8x8ըૉ͝ͱʹY'CbCrΛฒͨͷ • ͜ͷ8x8ըૉ͝ͱʹDCT/ྔࢠԽͰใྔΛམͱ͢ͷͰϒϩοΫڥք ʹϊΠζ͕ൃੜˠϒϩοΫϊΠζ MCU
color conversion chroma subsampling huffman coding quantization DCT transformation ReadScanline
io.Reader 8x8ըૉ͝ͱॲཧ͢Δ →ετϦʔϛϯάॲཧͰ͖Δʂ 1ըૉ͝ͱʹॲཧͰ͖Δ
/* ... static JDIMENSION read_scanlines(j_decompress_ptr dinfo, unsigned char *buf, int
stride, int height) { JSAMPROW *rows = alloca(sizeof(JSAMPROW) * height); int i; for (i = 0; i < height; i++) { rows[i] = &buf[i * stride]; } return jpeg_read_scanlines(dinfo, rows, height); } */ import "c" func (dec *Decoder) ReadScanline(p [][]uint8) (n int, err error) { n := C.read_scanline(dec.dinfo, p, dec.Stride, dec.Height); return n, nil } libjpegͷ߹jpeg_read_scanlinesΛݺͿ͚ͩ
Conclusion • GoͰඇৗʹ؆୯ʹετϦʔϜॲཧΛ࣮ݱͰ͖Δ • JPEGͷ߹8ߦͣͭσίʔυ/Τϯίʔυ͢Δ͜ͱ͕Ͱ͖Δ →ετϦʔϜͰ͖ΔΠϯλʔϑΣʔε͢ΕͪΐͬͱͣͭॲཧͰ͖Δ →libjpegετϦʔϜͰ͖ΔAPIΛ࣋ͬͯΔͷͰcgoͰݺͼग़͚ͩ͢ • ϒϩοΫϊΠζͱ8x8ըૉ͝ͱʹॲཧ͢ΔͷͰڥքʹग़ΔϊΠζ