Share
Embed
[mercari GEARS 2025] Foundations of AI - The Invisible Forces Driving Product Innovation
Transcript
33 slides
-
Foundations of AI: The Invisible Forces Driving Product Innovation Sho
Akiyama Mercari / Engineering Manager Andre Rusli Mercari / Research Engineer -
• A founding member of the newly established Applied AI
Research team • Previously worked as an ML engineer working in multiple different domains, including: LLM fine-tuning, AI-supported listing, embeddings for search, and on-device AI. • Engineering Manager for the AI/LLM team at Mercari. • Joined Mercari in 2024 as an individual contributor (MLOps and applied ML infrastructure). • Prior experience in mobile, web, and ML/MLOps engineering at companies including TRIAL Inc. (Fukuoka) and Cogent Labs (Tokyo). Engineering Manager, AI/LLM Sho Akiyama Andre Rusli, PhD Research Engineer, Applied AI -
The journey of product innovation sparked by images. 1) Origin
– from a forgotten piece of experimental code 2) Evolution – the rise of the embeddings revolution 3) Explosion – the long-awaited revival of Image Search 4) Expansion – from AI listing to contextual semantic search 5) Future – continuous research and knowledge sharing List of Chapters -
01 Dormant Idea A small experiment with SigLIP image embeddings
sat unused in a personal repo, initially considered experimental and not ready for production. 02 Spark of Curiosity Picked up again when unclaimed ideas surfaced. SigLIP was bootstrapped internally without external mandate. 03 First Customer Recommendation team became the earliest adopter, validating real-world impact for existing MobileNet use case to try SigLIP. Story kicked off with a casual team building lunch that turned into a roadmap discussion. -
Origin - The foundation of the journey wasn’t a top-down
initiative but grassroots engineering curiosity. - Lesson: small experiments can become infra-level breakthroughs if coupled with product alignment. -
Initial reusable pipeline: Image ingestion ↓ Embedding model ↓ ANN
index ↓ Image similarity search service(s) Embeddings moved from “experimental” to “critical” -
Embeddings moved from “experimental” to “critical infra.” Additionally, to
get people onboarded and excited within the company, “Embeddings as a Meme” → #z-embeddings Slack channel becomes shorthand for new use cases even with non-technical teams. -
Embeddings moved from “experimental” to “critical infra.” - Migration from
MobileNet to embeddings for Similar Looks - 1.5x increase in tap rate - +14% increase in purchase count via Item Detail Page -
Evolution One of the first clear demonstrations that embeddings ≠
research toy → direct GMV impact. -
Explosion: Features (1) 「Engineering」 ML / Infrastructure /
Backend / Client / Frontend (2) 「Product」 Metrics / Analysis / Milestones -
From “single-use critical infra.” to “reusable critical infra.” Reusable
inference for image embeddings -
Explosion (1) 「Engineering」 ML / Infrastructure / Backend
/ Client / Frontend (2) 「Product」 Metrics / Analysis / Milestones (3) 「Marketing」 Events / CM / Onboarding / User Growth -
Explosion (1) 「Engineering」 ML / Infrastructure / Backend
/ Client / Frontend (2) 「Product」 Metrics / Analysis / Milestones (3) 「Marketing」 Events / CM / Onboarding / User Growth (4) 「Research」 Conferences / Papers -
Explosion Presented at multiple conferences around the world • FOSSASIA
Summit in Bangkok, Thailand • 画像の認識・理解シンポジウム MIRU2025 in Kyoto, Japan • ACM KDD 2025 in Toronto, ON, Canada • 19th ACM Conference on Recommender Systems (RecSys 2025) in Prague, Czech Events / CM / SNS -
Beyond Image Search: Expanding to Other Features (2) 「Search」
Domain- specific Text Embeddings (1) 「AI Listing」 Enhancing Category Prediction -
image-to- images similarity (1) 「AI Listing」 Category Prediction
Upload images Predict categories 🪄 ✨ Generated! ✅✅ Category #1 Category #2 … Category #n Listing Completion Rate via AI Listing 📈🚀 -
(2) 「Search」 Context- Aware Text Embeddings “Onepiece summer”
“Onepiece goods” “Coach How To” “Coach Mens” “Coach” “Super Rare Chanel Belt Affordable Used” 👗🏖🩱 👒🧸 ⚽📈 🛍👜🧥 ? ? -
Solution: A fine-tuned text embeddings model Training set “Query”
& “Item Title” pairs Base model cl-nagoya/ruri-small-v2 Loss Functions MultipleNegativeRankingLoss MatryoshkaLoss (2) 「Search」 Context- Aware Text Embeddings -
Offline evaluation shows promising results
In addition to a comprehensive qualitative manual checks by internal experts. Model # of dim nDCG@k nDCG@k (long queries) Prec@k Recall@k Baseline 32 (PCA) 0.099 0.142 0.007 0.272 Fine-tuned 32 (MRL) 0.195 0.235 0.015 0.607 (2) 「Search」 Context- Aware Text Embeddings -
Product 🤝 Engineering 🤝 Research Beyond the Features: An Organizational
Shift The「Applied AI Research」Team is formed 🎊🎊🎊 -
The Applied AI Research (AAIR) Team A product-aligned but execution-autonomous
driver of AI innovation: delivering research that ships, by working ahead of the roadmap – not outside of it. 01 Semantic Understanding 02 Contextual Intelligence 03 Continuous AI Learning -
Multimodal embeddings Going beyond only-text, or only-image, understanding. Advanced multi-lingual
NLP/U Enabling our platform to reach more global audiences. User embeddings Personalized search and discovery, moderation (TnS), etc. and many more exciting projects! Going Forward