Upgrade to Pro — share decks privately, control downloads, hide ads and more …
Sign up for free

[mercari GEARS 2025] Evals for LLMApps/Agents

Sponsored · SiteGround - Reliable hosting with speed, security, and support you can count on. →
Avatar for mercari mercari PRO
November 14, 2025
Technology
140
0

[mercari GEARS 2025] Evals for LLMApps/Agents

Avatar for mercari

mercari PRO

November 14, 2025

More Decks by mercari

See All by mercari
[DevDojo] Getting Started with BI: Looker Essentials - 2025
Avatar for mercari mercari
PRO
0
180
[DevDojo] Introduction to LLMs & AI Agents - 2025
Avatar for mercari mercari
PRO
0
340
[mercari GEARS 2025] Techniques for Reliable Code Generation Using AI Agents
Avatar for mercari mercari
PRO
0
320
[mercari GEARS 2025] Foundations of AI - The Invisible Forces Driving Product Innovation
Avatar for mercari mercari
PRO
0
320
[mercari GEARS 2025] Building Foundation for Mercari’s Global Expansion
Avatar for mercari mercari
PRO
1
520
[mercari GEARS 2025] The Past, Present, and Future of Anti-Phishing Measures at Mercari
Avatar for mercari mercari
PRO
0
180
[mercari GEARS 2025] Backend Standardization with MCP
Avatar for mercari mercari
PRO
0
260
[mercari GEARS 2025] Transforming customer engagement with Google Customer Engagement Suite
Avatar for mercari mercari
PRO
0
440
[mercari GEARS 2025] PJ Aurora’s Vision and Automated UI Quality Evaluation Agents
Avatar for mercari mercari
PRO
0
410

Other Decks in Technology

See All in Technology
AIエージェントを最高のパートナーに育てる方法｜評価と判断軸を育てる5つのステップ
Avatar for 青木 晃一｜AI・Copilot活用 koichiaoki
1
160
Harness Engineering on Rails
Avatar for Joël Quenneville joelq
0
520
バイブコーディング時代のWebアプリ開発入門~Cloud Runで学ぶセキュアなビルドとデプロイ
Avatar for 尾﨑勇太/Yuta Ozaki waiwai2111
1
150
ほんとうの信頼性はヒーローが死んでからはじまる / True reliability begins after the hero dies
Avatar for VTRyo vtryo
0
120
Why Agent Cost Needs Observability
Avatar for NTT docomo Business nttcom
0
170
あけおめLINE 傾向とその対策
Avatar for nasa9084 nasa9084
0
330
3人で1000GPU超を統合運用する？マルチクラウド＆オンプレを跨ぐ、構築と運用のリアル！
Avatar for kazuki odo kazukun0716
1
190
What the customer really needed
Avatar for Yasunobu Kawaguchi kawaguti
PRO
3
230
「どのログを見ればいい？」 から始めた サーバーレス障害解析
Avatar for y_waka1 y_waka1
1
150
認知負荷を吸収し、プロダクトをまたぐPR Preview基盤の設計事例
Avatar for taiki45 taiki45
2
320
今話題のAI「Jev」って何？ 宇宙最速で学ぶ会
Avatar for みのるん minorun365
PRO
34
22k
AI Agent入門〜今更聞けないAgentの話〜
Avatar for Magy hiromimaganuma
0
120

Featured

See All Featured
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
Avatar for Rebecca Miller-Webster rmw
35
3.8k
Organizational Design Perspectives: An Ontology of Organizational Design Elements
Avatar for Dr. Kim W Petersen kimpetersen
PRO
1
830
Game over? The fight for quality and originality in the time of robots
Avatar for Wayne Barker wayneb77
1
290
Bash Introduction
Avatar for André Augusto Costa Santos 62gerente
615
220k
Navigating Weather and Climate Data
Avatar for Ryan Abernathey rabernat
0
530
Stop Working from a Prison Cell
Avatar for Chris Michel hatefulcrawdad
274
21k
Agile that works and the tools we love
Avatar for Rasmus Luckow-Nielsen rasmusluckow
331
22k
Fashionably flexible responsive web design (full day workshop)
Avatar for Andy Clarke malarkey
409
67k
Code Reviewing Like a Champion
Avatar for maltzj maltzj
528
40k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
Avatar for Dan Newman danielanewman
230
23k
Keith and Marios Guide to Fast Websites
Avatar for Keith Pitt keithpitt
413
23k
XXLCSS - How to scale CSS and keep your sanity
Avatar for Zaharenia Atzitzikaki sugarenia
250
1.3M

Transcript

  1. Evals for LLMApps/Agents
 Jehandad Kamal
 Mercari / Engineer @ AI/LLM

    Team


  2. Mercari AI Listing Support https://about.mercari.com/en/press/news/articles/20240910_aisupport/

  3. Agents


  4. Agent Development Loop


  5. First SDLC for Agents
 ai.mercari.com

  6. Types of Evals ( by technique )
 ai.mercari.com

  7. Types of Evals (by perspective )


  8. Improved SDLC for Agents


  9. Thank You!