Resizing Animated GIFs Without CGO or Third-Party Libraries

11/13/2024 Slides for the presentation at GoLab 2024 in Florence.

If you want to see the GIF images in the slide animated, please check out the Google Slides version.
https://docs.google.com/presentation/d/1sRzow8ytY6AP6HfmXMFBuRODgwg9_Tlhdjw03zWcPGc/edit?usp=sharing

Transcript

1. Takuto Nagami X: @logica0419 GitHub: @logica0419 Resizing Animated GIFs Without CGO or Third-Party Libraries
2. Why GIF resizing?
3. traQ Internal Messenger Application at traP (Student Community)
4. Animated icons & stamps (in traQ) • Icons: enable users to express themselves better • Stamps: more fun & live communication
5. De Facto Standard: CLI or CGO
6. Common choices • ImageMagick • MagickWand C API + CGO • libvips + CGO
7. Problems https://www.docker.com/company/ https://www.irasutoya.com/2014/03/blog-post_2687.html
8. Let’s resize animated GIFs only with Pure Go! Ideally, only with standard libraries & sub repositories
9. What is GIF image?
10. • Image (not video or multimedia) • Able to contain animation GIF ≓ Animated image https://illust55.com/1155/
11. What’s contained in the GIF image? Frame ↓ Image
12. What’s contained in the GIF image? • In image ◦ Image size (width, height) ◦ Global color palette • In frame ◦ Image data ◦ Delay ◦ Color palette ◦ Disposal
13. Processing images with Go
14. image/gif • Standard library ◦ https://pkg.go.dev/image/gif • gif.DecodeAll() -> *gif.GIF struct type GIF struct { Image []*image.Paletted ← Each frame Delay []int ← For each frame LoopCount int Disposal []byte ← Painpoint Config image.Config ← Image size & color palette BackgroundIndex byte }
15. golang.org/x/image/draw • One of sub repositories • draw.Draw() function ◦ Draw src image on dst image ◦ op - draw.Src / draw.Over func Draw(dst Image, r image.Rectangle, src image.Image, sp image.Point, op Op)
16. golang.org/x/image/draw • draw.Scaler interface ◦ Able to resize a single image type Scaler interface { Scale(dst Image, dr image.Rectangle, src image.Image, sr image.Rectangle, op Op, opts *Options) }
17. The simplest GIF resizing
18. Prepare the destination GIF image dst := &gif.GIF{ Image: make([]*image.Paletted, 0, len(src.Image)), Delay: src.Delay, LoopCount: src.LoopCount, Disposal: src.Disposal, Config: image.Config{ ColorModel: src.Config.ColorModel, Width: width, Height: height, }, BackgroundIndex: src.BackgroundIndex, }
19. Resize each frames for _, srcFrame := range src.Image { srcFrameBounds := srcFrame.Bounds() dstFrameBounds := image.Rect(0, 0, width, height) dstFrame := image.NewPaletted(dstFrameBounds, srcFrame.Palette) draw.CatmullRom.Scale(dstFrame, dstFrameBounds, srcFrame, srcFrameBounds, draw.Src, nil) dst.Image = append(dst.Image, dstFrame) }
20. However, there are 3 traps
21. Trap 1: Optimized GIF animation
22. Miku.gif https://piapro.jp/t/FB3J
23. After resizing • Resized into the same size
24. Oh no! She becomes long!
25. Why? • Due to delta frame optimization
26. Delta frame optimization • If all frames are resized to dist size ◦ frame 1: ◦ frame 2~: 　
27. Again: resize each frames for _, srcFrame := range src.Image { srcFrameBounds := srcFrame.Bounds() dstFrameBounds := image.Rect(0, 0, width, height) dstFrame := image.NewPaletted(dstFrameBounds, srcFrame.Palette) draw.CatmullRom.Scale(dstFrame, dstFrameBounds, srcFrame, srcFrameBounds, draw.Src, nil) dst.Image = append(dst.Image, dstFrame) }
28. That’s why!
29. Solution: Calculating each dst frame size
30. Calculating each dst frame size widthRatio := float64(width) / float64(srcWidth) heightRatio := float64(height) / float64(srcHeight) for _, srcFrame := range src.Image { dstFrameBounds := image.Rect( int(float64(srcFrameBounds.Min.X)*widthRatio), int(float64(srcFrameBounds.Min.Y)*heightRatio), int(float64(srcFrameBounds.Max.X)*widthRatio), int(float64(srcFrameBounds.Max.Y)*heightRatio), ) dstFrame := image.NewPaletted(dstFrameBounds, srcFrame.Palette) (Resize Image & append to dst) }
31. Calculating each dst frame size • With frame size calculation ◦ frame 1: ◦ frame 2~: 　
32. Solved!
33. Trap 2: Interpolation
34. parapara.gif https://errand.jp/web/design/1023/
35. After resizing
36. What are these black jaggy noises?
37. To solve this, you need little knowledge about interpolation Let me explain
38. Abstract concept of interpolation • Interpolation = mixing pixel colors • Kernel = func to define how pixels mixed ◦ Bicubic ◦ lanczos2 / lanczos3 • Mixing between transparent & non-transparent pixels seems to create noize
39. Seems like the noise happens when image has transparent pixels! Investigation still in progress…
40. Workaround: Piling up frames before resizing
41. Get transparent pixels out of mind Transparent pixels
42. GIF rendering: piling up frames Rendered previous frame Rendered current frame
43. What we resize now Rendered previous frame Rendered current frame
44. How about resizing this one? Rendered previous frame Rendered current frame
45. srcBounds := image.Rect(0, 0, srcWidth, srcHeight) dstBounds := image.Rect(0, 0, width, height) tempCanvas := image.NewNRGBA(srcBounds) Prepare canvas for rendering
46. Pile up current frame on canvas for _, srcFrame := range src.Image { (Calculate the dst frame size) draw.Draw(tempCanvas, srcFrameBounds, srcFrame, srcFrameBounds.Min, draw.Over)
47. Resize fittedFrame := image.NewPaletted(dstBounds, srcFrame.Palette) draw.CatmullRom.Scale(fittedFrame, dstBounds, tempCanvas, srcBounds, draw.Src, nil)
48. Trim (for delta frame optimization) dstFrame := image.NewPaletted(dstFrameBounds, srcFrame.Palette) draw.Draw(dstFrame, dstFrameBounds, fittedFrame, dstFrameBounds.Min, draw.Src) (Append resized frame to dst) }
49. But there is still a problem… https://patirabi.com/2021/10/10/061gif/
50. (Partially) Solved!
51. Trap 3: Disposal
52. Disposal • Also called Frame Disposal Methods ◦ How frames should be treated after rendering • 3 types ◦ 0x01: None ◦ 0x02: Background ◦ 0x03: Previous
53. 0x01: Disposal None • Do nothing, just go to next frame rendering https://legacy.imagemagick.org/Usage/anim_basics/
54. 0x02: Disposal Background • Overwrite area of frame with background color https://legacy.imagemagick.org/Usage/anim_basics/
55. • Delete current frame before next frame rendering ◦ Rolling back to “previous” 0x03: Disposal Previous https://legacy.imagemagick.org/Usage/anim_basics/
56. Inappropriate disposal processing • Mess up transparent background images https://sozai-good.com/illust/gifanimation/29065
57. Solution: Process Disposal Properly
58. bgColor := image.NewUniform( src.Config.ColorModel.(color.Palette)[src.BackgroundIndex]) for i, srcFrame := range src.Image { previous = deepCopyImage(tempCanvas) (Resizing process) switch src.Disposal[i] { case gif.DisposalNone: case gif.DisposalBackground: draw.Draw(tempCanvas, srcFrameBounds, bgColor, srcFrameBounds.Min, draw.Src) case gif.DisposalPrevious: draw.Draw(tempCanvas, srcBounds, previous, srcBounds.Min, draw.Src) } }
59. Solved!
60. A few more things to work… • Retaining the aspect ratio • Making resizing kernel selectable • Concurrent processing
61. resigif • https://github.com/logica0419/resigif • Please use this & give me some feedback!
62. Results: ImageMagick CLI vs resigif
63. Performance assessment • Experiment: 512px x 320px -> 256px x 160px ◦ ImageMagick CLI ▪ magick -filter Lanczos -scale 50% ◦ CLI using resigif (single thread) ◦ CLI using resigif (concurrent) ▪ Using lanczos3 resizing kernel (same as ImageMagick)
64. Performance assessment • Metrics ◦ CPU: max utilization (top) ◦ Memory: max utilization (top) ◦ Time: using time command ▪ Average of 5 trials
65. Performance assessment • Result https://pixabay.com/gifs/cube-square-puzzle-rubiks-11588/ ImageMagick 678 KB resigif 512 KB
66. Performance assessment Max CPU Max Mem Time ImageMagick 100% 1.9% 4.64s resigif Single 120% 1.9% 4.79s resigif Concurrent 800% 1.9% 1.09s
67. • Just a reference: Two measures conbined ◦ Deletion of ImageMagick ◦ Migration to distroless base image • 1/3 of the original Docker image size
68. End of the journey to resize animated GIFs only with Pure Go…
69. Special Thanks
70. Travel & hotel expenses sponsors • Chiba Institute of Technology PPA • pixiv Inc.
71. Reviewers / Sponsors / Supporters • Reviewers • Personal Sponsors • Supporters
72. Takuto Nagami X: @logica0419 GitHub: @logica0419 Thank you so much!
73. Play with resigif & follow my X! resigif - GitHub @logica0419 - X