Upgrade to Pro — share decks privately, control downloads, hide ads and more …

[mercari GEARS 2025] Evals for LLMApps/Agents

Avatar for mercari mercari PRO
November 14, 2025
Technology
100
0

[mercari GEARS 2025] Evals for LLMApps/Agents

Avatar for mercari

mercari PRO

November 14, 2025

More Decks by mercari

See All by mercari
[DevDojo] Getting Started with BI: Looker Essentials - 2025
Avatar for mercari mercari
PRO
0
96
[DevDojo] Introduction to LLMs & AI Agents - 2025
Avatar for mercari mercari
PRO
0
130
[mercari GEARS 2025] Techniques for Reliable Code Generation Using AI Agents
Avatar for mercari mercari
PRO
0
250
[mercari GEARS 2025] Foundations of AI - The Invisible Forces Driving Product Innovation
Avatar for mercari mercari
PRO
0
230
[mercari GEARS 2025] Building Foundation for Mercari’s Global Expansion
Avatar for mercari mercari
PRO
1
390
[mercari GEARS 2025] The Past, Present, and Future of Anti-Phishing Measures at Mercari
Avatar for mercari mercari
PRO
0
120
[mercari GEARS 2025] Backend Standardization with MCP
Avatar for mercari mercari
PRO
0
180
[mercari GEARS 2025] Transforming customer engagement with Google Customer Engagement Suite
Avatar for mercari mercari
PRO
0
290
[mercari GEARS 2025] PJ Aurora’s Vision and Automated UI Quality Evaluation Agents
Avatar for mercari mercari
PRO
0
270

Other Decks in Technology

See All in Technology
20260516_SecJAWS_Days
Avatar for Takuya Yonezawa takuyay0ne
2
410
Vision Banana: Image Generators are Generalist Vision Learners
Avatar for Kazuyuki Miyazawa kzykmyzw
0
380
サンプリングは「作る」のか「使う」のか？ 分散トレースのコストと運用を両立する実践的戦略 / Why you need the tail sampling and why you don't want it
Avatar for ymotongpoo ymotongpoo
4
180
Purview Endpoint DLP 動かしてみた
Avatar for kozakigh kozakigh
0
390
20260515 ログイン機能だけではないアカウント管理を全体で考える～サービス設計者向け～
Avatar for OpenID Foundation Japan oidfj
0
400
バイブコーディング、仕様駆動、その先へ - 「不確実性に対する検査‧適応のサイクル」を設計する
Avatar for Koichiro Matsuoka littlehands
1
160
パーソルキャリア IT/テクノロジー職向け 会社紹介資料｜Company Introduction Deck
Avatar for PERSOL CAREER Dev | techtekt techtekt
PRO
0
120
続 運用改善、不都合な真実 〜 物理制約のない運用改善はほとんど無価値 / 20260518-ssmjp-kaizen-no-value-without-physical-constraints
Avatar for opelab opelab
2
180
データモデリング通り #5オンライン勉強会： AIに『ビジネスの文脈』を教え込むデータモデリング
Avatar for データ横丁 datayokocho
0
270
How to learn AWS Well-Architected with AWS BuilderCards: Security Edition
Avatar for Kosuke ENOMOTO coosuke
PRO
0
140
Redmine次期バージョン7.0の注目新機能解説 — UI/UX強化と連携強化を中心に
Avatar for MAEDA Go vividtone
1
110
"スキルファースト"で作る、AIの自走環境
Avatar for subroh_0508 subroh0508
0
120

Featured

See All Featured
Paper Plane (Part 1)
Avatar for Katie Cordina Lindsey katiecoart
PRO
0
7.5k
Navigating Team Friction
Avatar for Lara Hogan lara
192
16k
Java REST API Framework Comparison - PWX 2021
Avatar for Matt Raible mraible
34
9.3k
JAMstack: Web Apps at Ludicrous Speed - All Things Open 2022
Avatar for David Neal reverentgeek
1
440
Breaking role norms: Why Content Design is so much more than writing copy - Taylor Woolridge
Avatar for UX Y'all uxyall
0
280
The Illustrated Children's Guide to Kubernetes
Avatar for Chris Short chrisshort
51
52k
The Pragmatic Product Professional
Avatar for Laura Van Doore lauravandoore
37
7.3k
Designing for humans not robots
Avatar for Tammie Lister tammielis
254
26k
Navigating the Design Leadership Dip - Product Design Week Design Leaders+ Conference 2024
Avatar for Andy Polaine apolaine
0
310
Jess Joyce - The Pitfalls of Following Frameworks
Avatar for Tech SEO Connect techseoconnect
PRO
1
150
The Organizational Zoo: Understanding Human Behavior Agility Through Metaphoric Constructive Conversations (based on the works of Arthur Shelley, Ph.D)
Avatar for Dr. Kim W Petersen kimpetersen
PRO
0
320
Become a Pro
Avatar for Speaker Deck speakerdeck
PRO
31
5.9k

Transcript

  1. Evals for LLMApps/Agents
 Jehandad Kamal
 Mercari / Engineer @ AI/LLM

    Team


  2. Mercari AI Listing Support https://about.mercari.com/en/press/news/articles/20240910_aisupport/

  3. Agents


  4. Agent Development Loop


  5. First SDLC for Agents
 ai.mercari.com

  6. Types of Evals ( by technique )
 ai.mercari.com

  7. Types of Evals (by perspective )


  8. Improved SDLC for Agents


  9. Thank You!