Upgrade to Pro — share decks privately, control downloads, hide ads and more …

[mercari GEARS 2025] Evals for LLMApps/Agents

Sponsored · Ship Features Fearlessly Turn features on and off without deploys. Used by thousands of Ruby developers. →
Avatar for mercari mercari PRO
November 14, 2025
Technology
0
83

[mercari GEARS 2025] Evals for LLMApps/Agents

Avatar for mercari

mercari PRO

November 14, 2025
Tweet

More Decks by mercari

See All by mercari
[DevDojo] Getting Started with BI: Looker Essentials - 2025
Avatar for mercari mercari
PRO
0
59
[DevDojo] Introduction to LLMs & AI Agents - 2025
Avatar for mercari mercari
PRO
0
74
[mercari GEARS 2025] Techniques for Reliable Code Generation Using AI Agents
Avatar for mercari mercari
PRO
0
190
[mercari GEARS 2025] Foundations of AI - The Invisible Forces Driving Product Innovation
Avatar for mercari mercari
PRO
0
180
[mercari GEARS 2025] Building Foundation for Mercari’s Global Expansion
Avatar for mercari mercari
PRO
1
310
[mercari GEARS 2025] The Past, Present, and Future of Anti-Phishing Measures at Mercari
Avatar for mercari mercari
PRO
0
83
[mercari GEARS 2025] Backend Standardization with MCP
Avatar for mercari mercari
PRO
0
120
[mercari GEARS 2025] Transforming customer engagement with Google Customer Engagement Suite
Avatar for mercari mercari
PRO
0
190
[mercari GEARS 2025] PJ Aurora’s Vision and Automated UI Quality Evaluation Agents
Avatar for mercari mercari
PRO
0
150

Other Decks in Technology

See All in Technology
Exadata Fleet Update
Avatar for oracle4engineer oracle4engineer
PRO
0
1.1k
ClickHouseはどのように大規模データを活用したAIエージェントを全社展開しているのか
Avatar for Miki Matsumoto mikimatsumoto
0
320
日本の85%が使う公共SaaSは、どう育ったのか
Avatar for たけだ taketakekaho
1
270
配列に見る bash と zsh の違い
Avatar for kazzpapa3 kazzpapa3
3
190
衛星画像即時マッピングサービスの実現に向けて
Avatar for Yuji Kobayashi lehupa
1
200
Bill One急成長の舞台裏 開発組織が直面した失敗と教訓
Avatar for SansanTech sansantech
PRO
2
430
1,000 にも届く AWS Organizations 組織のポリシー運用をちゃんとしたい、という話
Avatar for kazzpapa3 kazzpapa3
0
240
OpenShiftでllm-dを動かそう！
Avatar for jpishikawa jpishikawa
0
190
Amazon Rekognitionで 「信玄餅きなこ問題」を解決する
Avatar for usanchuu usanchuu
1
210
マネージャー視点で考えるプロダクトエンジニアの評価 / Evaluating Product Engineers from a Manager's Perspective
Avatar for hiro-torii hiro_torii
0
250
AI駆動開発を事業のコアに置く
Avatar for 鬼澤輔 tasukuonizawa
1
1.3k
こんなところでも（地味に）活躍するImage Modeさんを知ってるかい？- Image Mode for OpenShift -
Avatar for Masataka Tsukamoto tsukaman
1
230

Featured

See All Featured
How to Think Like a Performance Engineer
Avatar for Harry Roberts csswizardry
28
2.5k
Distributed Sagas: A Protocol for Coordinating Microservices
Avatar for Caitie McCaffrey caitiem20
333
22k
DevOps and Value Stream Thinking: Enabling flow, efficiency and business value
Avatar for Helen Beal helenjbeal
1
120
Design and Strategy: How to Deal with People Who Don’t "Get" Design
Avatar for Morgane Peng morganepeng
133
19k
GraphQLの誤解/rethinking-graphql
Avatar for sonatard sonatard
74
11k
Practical Tips for Bootstrapping Information Extraction Pipelines
Avatar for Matthew Honnibal honnibal
25
1.8k
Raft: Consensus for Rubyists
Avatar for Patrick Van Stee vanstee
141
7.3k
How People are Using Generative and Agentic AI to Supercharge Their Products, Projects, Services and Value Streams Today
Avatar for Helen Beal helenjbeal
1
130
Sam Torres - BigQuery for SEOs
Avatar for Tech SEO Connect techseoconnect
PRO
0
190
ラッコキーワード サービス紹介資料
Avatar for ラッコ株式会社 rakko
1
2.3M
The Language of Interfaces
Avatar for Des Traynor destraynor
162
26k
The Illustrated Children's Guide to Kubernetes
Avatar for Chris Short chrisshort
51
51k

Transcript

  1. Evals for LLMApps/Agents
 Jehandad Kamal
 Mercari / Engineer @ AI/LLM

    Team


  2. Mercari AI Listing Support https://about.mercari.com/en/press/news/articles/20240910_aisupport/

  3. Agents


  4. Agent Development Loop


  5. First SDLC for Agents
 ai.mercari.com

  6. Types of Evals ( by technique )
 ai.mercari.com

  7. Types of Evals (by perspective )


  8. Improved SDLC for Agents


  9. Thank You!