Upgrade to Pro — share decks privately, control downloads, hide ads and more …

[mercari GEARS 2025] Evals for LLMApps/Agents

Sponsored · Ship Features Fearlessly Turn features on and off without deploys. Used by thousands of Ruby developers. →
Avatar for mercari mercari PRO
November 14, 2025
Technology
120
0

[mercari GEARS 2025] Evals for LLMApps/Agents

Avatar for mercari

mercari PRO

November 14, 2025

More Decks by mercari

See All by mercari
[DevDojo] Getting Started with BI: Looker Essentials - 2025
Avatar for mercari mercari
PRO
0
130
[DevDojo] Introduction to LLMs & AI Agents - 2025
Avatar for mercari mercari
PRO
0
170
[mercari GEARS 2025] Techniques for Reliable Code Generation Using AI Agents
Avatar for mercari mercari
PRO
0
290
[mercari GEARS 2025] Foundations of AI - The Invisible Forces Driving Product Innovation
Avatar for mercari mercari
PRO
0
270
[mercari GEARS 2025] Building Foundation for Mercari’s Global Expansion
Avatar for mercari mercari
PRO
1
460
[mercari GEARS 2025] The Past, Present, and Future of Anti-Phishing Measures at Mercari
Avatar for mercari mercari
PRO
0
150
[mercari GEARS 2025] Backend Standardization with MCP
Avatar for mercari mercari
PRO
0
220
[mercari GEARS 2025] Transforming customer engagement with Google Customer Engagement Suite
Avatar for mercari mercari
PRO
0
390
[mercari GEARS 2025] PJ Aurora’s Vision and Automated UI Quality Evaluation Agents
Avatar for mercari mercari
PRO
0
360

Other Decks in Technology

See All in Technology
VPCセキュリティ対応の最新事情
Avatar for nagisa_53 nagisa53
1
330
ファミコンでPHPを動かす / PHP on the Famicom
Avatar for HASEGAWA Tomoki tomzoh
2
650
AI時代におけるエンジニアの新たな役割──FDEとクオリアの探求/登壇資料（戸井田 裕貴）
Avatar for Hacobu hacobu
PRO
0
450
AIエージェントがあれば技術書なんてすぐ書けるでしょ→無理でした
Avatar for watany watany
4
470
reFACToring
Avatar for moznion moznion
0
500
変更し続けられるシステムをどう保つか — AI時代のSSoTという設計原則
Avatar for かわうそ kawauso
1
1.3k
41歳でAWSが好きすぎてITエンジニアになったおっさんの話
Avatar for Yuuki Yamashita yama3133
1
770
クラウドを使う側から、作る側へ / 大吉祥寺.pm 2026前夜祭
Avatar for FUJIWARA Shunichiro fujiwara3
6
1.5k
歴史から理解するクラウドインフラのしくみ
Avatar for t.kizawa kizawa2020
0
150
コンテナ・K8s研修【MIXI 26新卒技術研修】#2
Avatar for MIXI ENGINEERS mixi_engineers
PRO
1
170
20260720_クラウド女子会×PyLadiesTokyoコラボ Amazon Bedrock ハンズオン用資料
Avatar for yuuuka yuuka51
2
120
AIコード生成×サプライチェーン攻撃 — PHPが直面する“二重の信頼問題
Avatar for Shinya Saita shinyasaita
0
490

Featured

See All Featured
職位にかかわらず全員がリーダーシップを発揮するチーム作り / Building a team where everyone can demonstrate leadership regardless of position
Avatar for MADOX madoxten
64
56k
Color Theory Basics | Prateek | Gurzu
Avatar for Gurzu gurzu
0
400
Save Time (by Creating Custom Rails Generators)
Avatar for Garrett Dimon garrettdimon
PRO
32
3.9k
HTML-Aware ERB: The Path to Reactive Rendering @ RubyCon 2026, Rimini, Italy
Avatar for Marco Roth marcoroth
3
360
Building a Scalable Design System with Sketch
Avatar for Laura Van Doore lauravandoore
463
34k
Principles of Awesome APIs and How to Build Them.
Avatar for Keavy McMinn keavy
128
18k
brightonSEO & MeasureFest 2025 - Christian Goodrich - Winning strategies for Black Friday CRO & PPC
Avatar for Christian Goodrich cargoodrich
3
760
Bootstrapping a Software Product
Avatar for Garrett Dimon garrettdimon
PRO
307
120k
New Earth Scene 8
Avatar for Sydney Stone popppiees
3
2.4k
GitHub's CSS Performance
Avatar for Jon Rohan jonrohan
1033
470k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
Avatar for mongodb mongodb
49
10k
Context Engineering - Making Every Token Count
Avatar for Addy Osmani addyosmani
9
1k

Transcript

  1. Evals for LLMApps/Agents
 Jehandad Kamal
 Mercari / Engineer @ AI/LLM

    Team


  2. Mercari AI Listing Support https://about.mercari.com/en/press/news/articles/20240910_aisupport/

  3. Agents


  4. Agent Development Loop


  5. First SDLC for Agents
 ai.mercari.com

  6. Types of Evals ( by technique )
 ai.mercari.com

  7. Types of Evals (by perspective )


  8. Improved SDLC for Agents


  9. Thank You!