Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
JPEG streaming in Go
Search
Harukasan
PRO
March 14, 2017
Technology
2
2.6k
JPEG streaming in Go
pixiv Night #02 - 画像処理技術(go, blender, C++ライブラリ等)
2017-03-14
https://pixiv.connpass.com/event/50284/
Harukasan
PRO
March 14, 2017
Tweet
Share
More Decks by Harukasan
See All by Harukasan
pixivを支える技術 / 技育CAMPアカデミア
harukasan
PRO
3
440
20240401 新卒研修 - ピクシブにおける技術領域
harukasan
PRO
1
760
ピクシブのコンテンツ配信基盤技術 / pixiv TECH SALON
harukasan
PRO
5
5.5k
Goにおける画像ファイル処理 / golang.tokyo #19
harukasan
PRO
7
6.6k
WebRTC動画をトランスコードする / Transcoding video streams from WebRTC
harukasan
PRO
5
1.5k
ImageFluxを支えるリモート開発 / 20171202
harukasan
PRO
2
1.8k
YAPC::Fukuoka 前夜祭LT / Yet Another Pawoo Commit logs
harukasan
PRO
0
3k
YAPC::Fukuoka lunch session
harukasan
PRO
1
3k
マストドン会議: Pawoo / Mastodon Kaigi2
harukasan
PRO
2
470
Other Decks in Technology
See All in Technology
どうすると生き残れないのか/how-not-to-survive
hanhan1978
13
10k
株式会社Awarefy(アウェアファイ)会社説明資料 / Awarefy-Company-Deck
awarefy
3
12k
開発組織を進化させる!AWSで実践するチームトポロジー
iwamot
2
650
【Forkwell】「正しく」失敗できるチームを作る──現場のリーダーのための恐怖と不安を乗り越える技術 - FL#83 / A team that can fail correctly by forkwell
i35_267
2
190
Dify触ってみた。
niftycorp
PRO
0
120
MLflowはどのようにLLMOpsの課題を解決するのか
taka_aki
0
180
エンジニア主導の企画立案を可能にする組織とは?
recruitengineers
PRO
1
360
# Azure Cosmos DB パフォーマンス最適化入門 - 設計・開発・運用の実践テクニック
shibayan
0
130
【Oracle Cloud ウェビナー】【入門&再入門】はじめてのOracle Cloud Infrastructure [+最新情報]
oracle4engineer
PRO
1
120
LangGraph × Bedrock による複数の Agentic Workflow を利用した Supervisor 型のマルチエージェントの実現/langgraph-bedrock-supervisor-agent
ren8k
4
460
開発者体験を定量的に把握する手法と活用事例
ham0215
0
170
Roomの監視可能なクエリのカスタマイズとレガシーコードへの適用
shiita0903
2
150
Featured
See All Featured
[RailsConf 2023] Rails as a piece of cake
palkan
53
5.3k
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
The Power of CSS Pseudo Elements
geoffreycrofte
75
5.5k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
30
4.6k
We Have a Design System, Now What?
morganepeng
51
7.4k
Mobile First: as difficult as doing things right
swwweet
223
9.5k
Designing Experiences People Love
moore
140
23k
Docker and Python
trallard
44
3.3k
Code Review Best Practice
trishagee
67
18k
The Language of Interfaces
destraynor
156
24k
Building Your Own Lightsaber
phodgson
104
6.3k
Measuring & Analyzing Core Web Vitals
bluesmoon
6
270
Transcript
JPEG streaming in Go !IBSVLBTBO4IVOTVLF.JDIJJ ͋Δ͍+1&(ͷϒϩοΫϊΠζͱͳʹ͔
Harukasan Shunsuke MICHII Lead engineer of pixiv Inc., Product manager
of ImageFlux, Infrastructure engineer, Kibana, Gopher
None
© pixiv Inc., SAKURA Internet Inc. ImageFlux service ͓٬༷ετϨʔδ ίϯςϯπΩϟογϡ
ը૾มॲཧ ΦϦδφϧը૾Λऔಘ มޙΩϟογϡΛ৴ ΤϯυϢʔβ ։ൃऀ ৴URLͷઃఆ ৴ݩΦϦδϯͷઃఆ ϨϙʔτͷӾཡ طଘετϨʔδΛͦͷ··ར༻Մೳ ը૾σʔλΛطଘͷετϨʔδαʔό͔ ΒҠಈ͢Δඞཁ͋Γ·ͤΜɻཧը໘͔ ΒΦϦδϯΛઃఆ͢Δ͚ͩͰ͙͢ʹը૾ม ͕ར༻ՄೳͰ͢ɻ URLΛม͑Δ͚ͩͰը૾ม ৴URLʹύϥϝʔλΛՃ͢Δ͚ͩͰ ҙͷը૾มΛߦ͑·͢ɻ εϚʔτϑΥϯɺPCͰผʑͷURLΛ͏͜ ͱͰɺ1ຕͷը૾͔Β༷ʑͳαΠζͷը૾ Λ৴͢Δ͜ͱ͕ՄೳͰ͢ɻ ը૾มॲཧ
© pixiv Inc., SAKURA Internet Inc. ImageFlux service ͓٬༷ετϨʔδ ίϯςϯπΩϟογϡ
ը૾มॲཧ ΦϦδφϧը૾Λऔಘ มޙΩϟογϡΛ৴ ΤϯυϢʔβ ։ൃऀ ৴URLͷઃఆ ৴ݩΦϦδϯͷઃఆ ϨϙʔτͷӾཡ طଘετϨʔδΛͦͷ··ར༻Մೳ ը૾σʔλΛطଘͷετϨʔδαʔό͔ ΒҠಈ͢Δඞཁ͋Γ·ͤΜɻཧը໘͔ ΒΦϦδϯΛઃఆ͢Δ͚ͩͰ͙͢ʹը૾ม ͕ར༻ՄೳͰ͢ɻ URLΛม͑Δ͚ͩͰը૾ม ৴URLʹύϥϝʔλΛՃ͢Δ͚ͩͰ ҙͷը૾มΛߦ͑·͢ɻ εϚʔτϑΥϯɺPCͰผʑͷURLΛ͏͜ ͱͰɺ1ຕͷը૾͔Β༷ʑͳαΠζͷը૾ Λ৴͢Δ͜ͱ͕ՄೳͰ͢ɻ ը૾มॲཧ
Decoding a JPEG image Resizing Encoding into a new JPEG
image Fetching from an origin Writing content into a client http.Transport libjpeg-turbo libswscale from ffmpeg http.ResponseWriter libjpeg-turbo
streaming in Go • io.Reader: reading content into the given
buffer • io.Writer: writing content from the given buffer typedef Reader interface { func Read(p []byte) (n int, err error) } typedef Writer interface { func Write(p []byte) (n int, err error) }
• Copying content from a reader into a writer io.Copy(dst,
src) 32KB buffer io.Reader io.Writer Read(buffer) Write(buffer) : 32KBͷόοϑΝ෦Ͱຖճmake
• Copying content from a reader into a writer •
To avoid allocating a buffer, use io.CopyBuffer: io.Copy(dst, src) 32KB buffer io.Reader io.Writer Read(buffer) Write(buffer) io.CopyBuffer(dst, src, buf) : 32KBͷόοϑΝ෦Ͱຖճmake
func ServeHTTP(w *http.ResponseWriter, r *http.Request) { req, _ := http.NewRequest("GET",
"http://www.example.com", nil) resp, _ := http.DefaultTransport.RoundTrip(req) io.CopyBuffer(w, resp.Body, buf) } HTTPίϐʔ͢Δ͚ͩͰετϦʔϛϯάͰ͖ΔʢΤϥʔॲཧ……ʣ
Decoding a JPEG image Resizing Encoding into a new JPEG
image Fetching from an origin Writing content into a client http.Transport libjpeg-turbo libswscale from ffmpeg http.ResponseWriter libjpeg-turbo
Streaming image scanlines • ScanlineReader interface typedef ScanlineReader interface {
func Config() *Config → plane sizes, pixel format,... func ReadScanLine(p [][]uint8) (n int, err error) → reading scanlines of planes into each buffer → PNG: JPEG: } R G B Y Cb Cr *blue difference chroma *red difference chroma
Decoding a JPEG image Resizing Encoding into a new JPEG
image Fetching from an origin Writing content into a client http.Transport libjpeg-turbo libswscale from ffmpeg http.ResponseWriter libjpeg-turbo
Decoding a JPEG image Resizing Encoding into a new JPEG
image Fetching from an origin Writing content into a client http.Transport Decoder.ReadScanLines Scaler.ReadScanLines http.ResponseWriter Encoder.ReadScanLines
func ServeHTTP(w *http.ResponseWriter, r *http.Request) { req, _ := http.NewRequest("GET",
"http://www.example.com", nil) resp, _ := http.DefaultTransport.RoundTrip(req) dec := jpeg.NewDecoder(resp.Body io.Reader) → ScanlineReader dec := scale.NewResize(dec ScanlineReader) → ScanlineReader enc := jpeg.NewEncoder() enc.Write(w io.Writer, dec ScanlineReader) }
streaming image scanlines • ScanlineReader interfaceΛ͏͜ͱͰը૾σʔλετϦʔϜͱ͠ ͯѻ͏͜ͱ͕Ͱ͖Δ • interfaceΛ͏͜ͱͰ֤ॲཧͷར༻ํ๏Λ౷ҰͰ͖Δ •
ඪ४ϥΠϒϥϦͷΠϯλʔϑΣʔεΑ͘ߟ͑ΒΕͯΔͷͰϚω͢Δ ͱྑ͍ײ͡ʹͳΔ
ͳͥετϦʔϛϯά͢Δඞཁ͕͋Δͷ͔ • ը૾σʔλΛ͍ͬΜʹॲཧ͢ΔͱશըૉͷϝϞϦ͕ඞཁ͕ͩ ετϦʔϜॲཧ͢ΕόοϑΝΛҰ֬อ͢Δ͚ͩͰࡁΉ • ֤৭ۭؒຖʹॲཧΛهड़Ͱ͖ɺແବͳ৭ม͕ൃੜ͠ͳ͍ JPEGಉ࢜ͷϦαΠζͷ߹Y'CbCr৭ۭؒͷ··ѻ͏͜ͱͰɺ RGBͷมʹΑΔϩε͕ൃੜ͠ͳ͍ • cgoͰϒϩοΫ͢Δ࣌ؒΛ͘͢Δ͜ͱ͕Ͱ͖Δ
Streaming JPEG scanlines typedef ScanlineReader interface { func Config() *Config
→ plane sizes, pixel format,... func ReadScanLine(p [][]uint8) (n int, err error) → JPEG: } Y Cb Cr *blue difference chroma *red difference chroma ͭ·Γɺ͜ͷΠϯλʔϑΣʔεͰJPEGΛσίʔυ͢ΕJPEGΛετϦʔϜॲཧͰ͖Δ
color conversion chroma subsampling huffman coding quantization DCT transformation ReadScanline
io.Reader RGB→Y'CbCrͷ৭ۭؒม ΫϩϚαϒαϯϓϦϯάॲཧ DCTม ྔࢠԽ ϋϑϚϯѹॖ
8x8ϐΫηϧ͝ͱʹDCTʢࢄίαΠϯมʣͰ ྲྀͱप͝ͱͷަྲྀʹม →ߴप΄Ͳߥ͘ͳΔΑ͏ʹྔࢠԽ͢Δ͜ͱͰใྔΛݮ →ϋϑϚϯූ߸Խͯ͠ѹॖ ΊͬͪΌࡶʹJPEGͷѹॖΛઆ໌͢Δͱ ߴप प
8x8ϐΫηϧ͝ͱʹDCTʢࢄίαΠϯมʣͰ ྲྀͱप͝ͱͷަྲྀʹม →ߴप΄Ͳߥ͘ͳΔΑ͏ʹྔࢠԽ͢Δ͜ͱͰใྔΛݮ →ϋϑϚϯූ߸Խͯ͠ѹॖ ΊͬͪΌࡶʹJPEGͷѹॖΛઆ໌͢Δͱ ߴप प
• JPEGͰ8x8ϐΫηϧʢ1ϒϩοΫʣ͝ͱʹDCTɺྔࢠԽΛߦ͏ • MCU: 8x8ըૉ͝ͱʹY'CbCrΛฒͨͷ • ͜ͷ8x8ըૉ͝ͱʹDCT/ྔࢠԽͰใྔΛམͱ͢ͷͰϒϩοΫڥք ʹϊΠζ͕ൃੜˠϒϩοΫϊΠζ MCU
color conversion chroma subsampling huffman coding quantization DCT transformation ReadScanline
io.Reader 8x8ըૉ͝ͱॲཧ͢Δ →ετϦʔϛϯάॲཧͰ͖Δʂ 1ըૉ͝ͱʹॲཧͰ͖Δ
/* ... static JDIMENSION read_scanlines(j_decompress_ptr dinfo, unsigned char *buf, int
stride, int height) { JSAMPROW *rows = alloca(sizeof(JSAMPROW) * height); int i; for (i = 0; i < height; i++) { rows[i] = &buf[i * stride]; } return jpeg_read_scanlines(dinfo, rows, height); } */ import "c" func (dec *Decoder) ReadScanline(p [][]uint8) (n int, err error) { n := C.read_scanline(dec.dinfo, p, dec.Stride, dec.Height); return n, nil } libjpegͷ߹jpeg_read_scanlinesΛݺͿ͚ͩ
Conclusion • GoͰඇৗʹ؆୯ʹετϦʔϜॲཧΛ࣮ݱͰ͖Δ • JPEGͷ߹8ߦͣͭσίʔυ/Τϯίʔυ͢Δ͜ͱ͕Ͱ͖Δ →ετϦʔϜͰ͖ΔΠϯλʔϑΣʔε͢ΕͪΐͬͱͣͭॲཧͰ͖Δ →libjpegετϦʔϜͰ͖ΔAPIΛ࣋ͬͯΔͷͰcgoͰݺͼग़͚ͩ͢ • ϒϩοΫϊΠζͱ8x8ըૉ͝ͱʹॲཧ͢ΔͷͰڥքʹग़ΔϊΠζ