Upgrade to Pro — share decks privately, control downloads, hide ads and more …

[mercari GEARS 2025] Evals for LLMApps/Agents

Avatar for mercari mercari PRO
November 14, 2025
Technology
120
0

[mercari GEARS 2025] Evals for LLMApps/Agents

Avatar for mercari

mercari PRO

November 14, 2025

More Decks by mercari

See All by mercari
[DevDojo] Getting Started with BI: Looker Essentials - 2025
Avatar for mercari mercari
PRO
0
130
[DevDojo] Introduction to LLMs & AI Agents - 2025
Avatar for mercari mercari
PRO
0
160
[mercari GEARS 2025] Techniques for Reliable Code Generation Using AI Agents
Avatar for mercari mercari
PRO
0
280
[mercari GEARS 2025] Foundations of AI - The Invisible Forces Driving Product Innovation
Avatar for mercari mercari
PRO
0
260
[mercari GEARS 2025] Building Foundation for Mercari’s Global Expansion
Avatar for mercari mercari
PRO
1
440
[mercari GEARS 2025] The Past, Present, and Future of Anti-Phishing Measures at Mercari
Avatar for mercari mercari
PRO
0
140
[mercari GEARS 2025] Backend Standardization with MCP
Avatar for mercari mercari
PRO
0
210
[mercari GEARS 2025] Transforming customer engagement with Google Customer Engagement Suite
Avatar for mercari mercari
PRO
0
360
[mercari GEARS 2025] PJ Aurora’s Vision and Automated UI Quality Evaluation Agents
Avatar for mercari mercari
PRO
0
340

Other Decks in Technology

See All in Technology
4人目のSREはAgent
Avatar for tanimuyk tanimuyk
0
320
Why is RC4 still being used?
Avatar for tamaiyutaro tamaiyutaro
0
260
product engineering with qa
Avatar for Nealle nealle
0
130
#エンジニアBooks 30分でわかる
「技術記事を書く技術」 / engineer-books 2026-06-30
Avatar for Junichi Ito jnchito
1
150
トークン最適化のためのユーザーストーリー分析 / User Story Analysis for Token Optimization
Avatar for Tomohisa Omagari oomatomo
0
150
認証認可だけじゃない！ ID管理の構成要素と ライフサイクルを意識しよう
Avatar for ritou ritou
1
410
5分でわかる Amazon Connect_20260608
Avatar for hwangbyeonghun hwangbyeonghun
0
150
“全部コピーしない”ファイルデータの活用 ： — FSx for ONTAP × S3 Tables × Icebergで作るメタデータカタログ
Avatar for Yoshiki Fujiwara yoshiki0705
0
130
【FinOps】データドリブンな意思決定を目指して
Avatar for kaita z63d
3
550
AI時代における最適なQA組織の作り方
Avatar for ymty ymty
3
300
AI-DLCを “そのまま導入しなかった”話 ~組織に合わせてアジャストした 私たちの実践共有~
Avatar for hiroramos hiroramos4
PRO
1
480
BPaaSで進むAIオペレーションの現在地 AI実装が効く領域とスケーラビリティの選定と実装
Avatar for KentaroFujii kentarofujii
0
230

Featured

See All Featured
Hiding What from Whom? A Critical Review of the History of Programming languages for Music
Avatar for Tomoya Matsuura tomoyanonymous
2
870
Visual Storytelling: How to be a Superhuman Communicator
Avatar for David Neal reverentgeek
2
570
CSS Pre-Processors: Stylus, Less & Sass
Avatar for Bermon Painter bermonpainter
360
30k
Highjacked: Video Game Concept Design
Avatar for Ryan Kendrick rkendrick25
PRO
1
400
Avoiding the “Bad Training, Faster” Trap in the Age of AI
Avatar for Mike Taylor tmiket
0
190
For a Future-Friendly Web
Avatar for Brad Frost brad_frost
183
10k
The Anti-SEO Checklist Checklist. Pubcon Cyber Week
Avatar for Ryan Jones ryanjones
0
180
ピンチをチャンスに：未来をつくるプロダクトロードマップ #pmconf2020
Avatar for Aki / @LoveIdahoBurger aki_iinuma
128
56k
Speed Design
Avatar for Sergey Chernyshev sergeychernyshev
33
1.9k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
Avatar for Eileen M. Uchitelle eileencodes
38
2.9k
The innovator’s Mindset - 
Leading Through an Era of Exponential Change - McGill University 2025
Avatar for Jean-Marc De Jonghe jdejongh
PRO
1
210
ReactJS: Keep Simple. Everything can be a component!
Avatar for Pedro Nauck pedronauck
666
130k

Transcript

  1. Evals for LLMApps/Agents
 Jehandad Kamal
 Mercari / Engineer @ AI/LLM

    Team


  2. Mercari AI Listing Support https://about.mercari.com/en/press/news/articles/20240910_aisupport/

  3. Agents


  4. Agent Development Loop


  5. First SDLC for Agents
 ai.mercari.com

  6. Types of Evals ( by technique )
 ai.mercari.com

  7. Types of Evals (by perspective )


  8. Improved SDLC for Agents


  9. Thank You!