SigLIP2 Explanation

Transcript

1. VBLab, Professor. Lee DoHoon Meng Socheata (지아) Department of Information Convergence Engineering Master’s degree in AI convergence, 2026 Presentation’s material 👇 mengsocheata@pusan.ac.kr www.linkedin.com/in/auranightsky
3. 연구 배경 및 관련 연구 조사 Research Background and Related Work 1. 연구 배경 (Research Background) 2. 연구 목적 및 문제 정의 (Research Objectives and Problem Definition) 3. 기존 연구와의 차별성 (Research Gap and Novelty)
4. 모델 구조 및 방법론, 결과 및 우수성 분석 Model Architecture, Methodology, Results, and Analysis of Strengths 1. 모델 구조 (Model Architecture) 2. 제안 방법론 (Proposed Methodology) 3. 실험 환경 및 평가 지표 (Experimental Setup and Evaluation Metrics) 4. 한계점 분석 (Limitations)
5. 연구 배경 및 관련 연구 조사 Research Background and Related Work
6. 1. 연구 배경 (Research Background) What is SigLIP2? • A family of SigLIP, Vision-Language Models (VLMs) • Developed by Google DeepMind, released in February 2025 • Builds upon SigLIP, introduced by Google Research in 2023 SigLIP Vision-Language Encoder Matching images with text descriptions Sigmoid-based loss
7. 2. 연구 목적 및 문제 정의 (Research Objectives and Problem Definition) Uses a fixed image size, which can blur or distort images and cause details to be lost. Struggle with understanding object’s precise locations and details Performs better in English but may struggle with other languages.
8. 3. 기존 연구와의 차별성 (Research Gap and Novelty) Dense features help the model understand details and locate objects within an image. SigLIP SigLIP2 Global alignment Decoder-based loss & Self-supervised losses Semantic Segmentation Object Localization Depth Estimation “A cat sitting on a sofa.” “A cat sitting on a sofa.”
9. 3. 기존 연구와의 차별성 (Research Gap and Novelty) Native aspect ratio and variable resolution Supports different image sizes without distorting their original shape. SigLIP SigLIP2 Without NaFlex: Fixed Resolution With NaFlex Variable Resolution Better Detail Retention Aspect Ratio Preservation
10. 3. 기존 연구와의 차별성 (Research Gap and Novelty) Strong multilingual vision-language encoders Understands images and text across multiple languages SigLIP • Limited Multilingual Performance • English-Focused Performance • Language Performance Gap SigLIP2 • Multilingual Training Data • 109 Languages • Improved Multilingual Understanding
11. 모델 구조 및 방법론, 결과 및 우수성 분석 Model Architecture, Methodology, Results, and Analysis of Strengths
12. 1. 모델 구조 (Model Architecture) MAP Head combines important features into one compact representation.
13. 1. 모델 구조 (Model Architecture) Sigmoid-based loss matches images with text, while LocCa generates detailed captions Sigmoid Loss Formula:
14. 1. 모델 구조 (Model Architecture) MAP (Multihead Attention Pooling) Head • Applies attention to encoder features. • Combines information from different patches/tokens. • Produces a single compact representation.
15. 1. 모델 구조 (Model Architecture) Self-Distillation • Teacher: Full image (global view). • Student: Cropped images (local views). • Goal: Improve local visual understanding. Full Image Cropped Image
16. 1. 모델 구조 (Model Architecture) Masked Prediction • • • Masks 50% of image patches. Student matches the teacher’s features at hidden patches. Goal: Learn fine-grained visual details.
17. 2. 제안 방법론 (Proposed Methodology)
18. 2. 제안 방법론 (Proposed Methodology)
19. 2. 제안 방법론 (Proposed Methodology) NaFlex • • • • Variable image resolution Preserves aspect ratio Document understanding OCR (text recognition)
20. 2. 제안 방법론 (Proposed Methodology)
21. 실험 Experiments
22. 3. 실험 환경 및 평가 지표 (Experimental Setup and Evaluation Metrics)
23. 3. 실험 환경 및 평가 지표 (Experimental Setup and Evaluation Metrics) Crossmodal-3600, a massively multilingual multimodal evaluation dataset
24. Testing Input Image
25. 4. 한계점 분석 ( Limitations) • High computational cost • Weaker performance on rare languages • Difficulty with typologically distant languages • Limited research on multilingual interpretability
26. Conclusion • Improved image-text understanding • Better classification and localization • Stronger multilingual support • Flexible image resolution (NaFlex) • Potential for real-world applications
27. References • SigLIP2 • multilingual-siglip2-encoders • youtube-siglip2 • github-siglip2 • huggingface-experiment • sigmoid-based loss • WebLI dataset • huggingface-attention-pooling
28. Thank you! Meng Socheata (지아) mengsocheata@pusan.ac.kr www.linkedin.com/in/auranightsky Presentation’s material 👇
29. Question 1. what are the potential applications for SigLIP2? 2. Why does SigLIP2 use a sigmoid-based loss instead of the softmax-based loss used in models like CLIP? What are the benefits? (Please explain)