CAOS 2015 Summer: Hayabusa Internals

http://hayabusa.io

Transcript

1. Hayabusa Internals Kohei Morino(森野耕平)
2. Tech overview
3. Hayabusa modules • hayabusa ◦ On-the-fly responsive image generation and delivery ◦ Express-compatible Node.js middleware ◦ Plug-in architecture • hayabusa-server ◦ Express-based web server using hayabusa
4. Hayabusa plug-ins • Resizing • Cropping • Format conversion • Optimization • PSD layers and slices • And more }Using ImageMagick
5. Flow ・・・ Original image Output image Data stream 1. Parse URL 2. Arrange modules Request File path + Params Modules
6. architecture Hayabusa Cache Server (Redis) Load Balancer CDN User GET https://hayabusa.io/foo/img1.webp Original Images S3 or HTTP server foo bar http://xxxxxxx/img1.png • Fetch image • Check changes • Processed image data • Metadata Process image Config
7. Caching (CDN) • Using CloudFront (for now) • ‘Vary’ header: User-Agent, Accept ◦ Cache used only when accessed by the same URL, User-Agent and Accept header ◦ Result image differs according to user’s environment GET https://hayabusa.io/foo/img1.auto png, webp… progressive / baseline (jpg)
8. Caching (Redis) • Use ElastiCache • Cache processed images ◦ LRU cache • Cache metadata ◦ Size ▪ for JPEG progressive/baseline decision ◦ Header (etag) ▪ Use for detection of original image changes ◦ Cache time, etc.
9. Creating cache • Cache is created at the first access ◦ It takes some time for processing the image • Ahead-of-time caching is supported through API ◦ Post URL list
10. Optimization • Wrap various modules for compression ◦ pngquant ▪ Reduces file sizes often as much as 70% ◦ optipng ▪ PNG lossless compression ◦ zopfli ▪ Better deflate algorithm ◦ gifsicle ▪ Compression for gif ◦ jpegrecompress ▪ Compress JPEG by re-encoding to the smallest size while keeping visual quality
11. Progressive vs Baseline Hayabusa chooses JPEG format by file size automatically. “Images of file size 10K and over have a better chance of being smaller when using the progressive JPEG format” X-axis: original image size Y-axis: difference when subtracting progressive from baseline sizes quoted from: http://www.bookofspeed.com/chapter5.html
12. Cascading default parameters Priority 1. Params in URL 2. Default values in directory setting 3. Default values in origin server setting 4. Plug-in defaults https://hayabusa.io/foo/img1.width-200.optipng-3.pngquant(color-256).・・・.png https://hayabusa.io/foo/img1.width-200.png simplify URL with defaults
13. Roadmap 1. Full PSD renderer 2. 9-slices 3. SpriteSheets / Atlas 4. CLI for local usage 5. GUI for designers 6. Unity assets 7. Sound, Movie, Font support
14. Thank you! hayabusa.io