Upgrade to Pro — share decks privately, control downloads, hide ads and more …

[mercari GEARS 2025] Evals for LLMApps/Agents

Sponsored · Ship Features Fearlessly Turn features on and off without deploys. Used by thousands of Ruby developers. →
Avatar for mercari mercari PRO
November 14, 2025
Technology
0
92

[mercari GEARS 2025] Evals for LLMApps/Agents

Avatar for mercari

mercari PRO

November 14, 2025
Tweet

More Decks by mercari

See All by mercari
[DevDojo] Getting Started with BI: Looker Essentials - 2025
Avatar for mercari mercari
PRO
0
75
[DevDojo] Introduction to LLMs & AI Agents - 2025
Avatar for mercari mercari
PRO
0
100
[mercari GEARS 2025] Techniques for Reliable Code Generation Using AI Agents
Avatar for mercari mercari
PRO
0
220
[mercari GEARS 2025] Foundations of AI - The Invisible Forces Driving Product Innovation
Avatar for mercari mercari
PRO
0
200
[mercari GEARS 2025] Building Foundation for Mercari’s Global Expansion
Avatar for mercari mercari
PRO
1
360
[mercari GEARS 2025] The Past, Present, and Future of Anti-Phishing Measures at Mercari
Avatar for mercari mercari
PRO
0
100
[mercari GEARS 2025] Backend Standardization with MCP
Avatar for mercari mercari
PRO
0
150
[mercari GEARS 2025] Transforming customer engagement with Google Customer Engagement Suite
Avatar for mercari mercari
PRO
0
230
[mercari GEARS 2025] PJ Aurora’s Vision and Automated UI Quality Evaluation Agents
Avatar for mercari mercari
PRO
0
210

Other Decks in Technology

See All in Technology
CloudFrontのHost Header転送設定でパケットの中身はどう変わるのか？
Avatar for nagisa_53 nagisa53
1
190
【社内勉強会】新年度からコーディングエージェントを使いこなす - 構造と制約で引き出すClaude Codeの実践知
Avatar for nwiizo nwiizo
24
12k
FastMCP OAuth Proxy with Cognito
Avatar for iganin hironobuiga
3
210
TUNA Camp 2026 京都Stage ヒューリスティックアルゴリズム入門
Avatar for terry-u16 terryu16
0
510
Phase11_戦略的AI経営
Avatar for overflow ,Inc overflowinc
0
1.6k
JEDAI認定プログラム JEDAI Order 2026 受賞者一覧 / JEDAI Order 2026 Winners
Avatar for Databricks Japan databricksjapan
0
350
Blue/Green Deployment を用いた PostgreSQL のメジャーバージョンアップ
Avatar for Ken Kato kkato1
0
140
俺の/私の最強アーキテクチャ決定戦開催 ― チームで新しいアーキテクチャに適合していくために / 20260322 Naoki Takahashi
Avatar for SHIFT EVOLVE shift_evolve
PRO
1
450
How to install a gem
Avatar for André Arko indirect
0
1.6k
スピンアウト講座02_ファイル管理
Avatar for overflow ,Inc overflowinc
0
1.4k
スピンアウト講座05_実践活用事例
Avatar for overflow ,Inc overflowinc
0
1.3k
Bref でサービスを運用している話
Avatar for Sugar Sato sgash708
0
200

Featured

See All Featured
Mozcon NYC 2025: Stop Losing SEO Traffic
Avatar for Sam Torres samtorres
0
190
Discover your Explorer Soul
Avatar for Emna emna__ayadi
2
1.1k
Producing Creativity
Avatar for Steve Smith orderedlist
PRO
348
40k
A better future with KSS
Avatar for Kyle Neath kneath
240
18k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
Avatar for Stéphanie Walter stephaniewalter
287
14k
Navigating the Design Leadership Dip - Product Design Week Design Leaders+ Conference 2024
Avatar for Andy Polaine apolaine
0
240
How To Speak Unicorn (iThemes Webinar)
Avatar for Michelle Schulp Hunt marktimemedia
1
410
Understanding Cognitive Biases in Performance Measurement
Avatar for Philip Tellis bluesmoon
32
2.8k
Performance Is Good for Brains [We Love Speed 2024]
Avatar for Tammy Everts tammyeverts
12
1.5k
Exploring the relationship between traditional SERPs and Gen AI search
Avatar for Ray Grieselhuber raygrieselhuber
PRO
2
3.7k
The Illustrated Guide to Node.js - THAT Conference 2024
Avatar for David Neal reverentgeek
1
320
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
Avatar for Tammy Everts tammyeverts
55
3.3k

Transcript

  1. Evals for LLMApps/Agents
 Jehandad Kamal
 Mercari / Engineer @ AI/LLM

    Team


  2. Mercari AI Listing Support https://about.mercari.com/en/press/news/articles/20240910_aisupport/

  3. Agents


  4. Agent Development Loop


  5. First SDLC for Agents
 ai.mercari.com

  6. Types of Evals ( by technique )
 ai.mercari.com

  7. Types of Evals (by perspective )


  8. Improved SDLC for Agents


  9. Thank You!