Upgrade to Pro — share decks privately, control downloads, hide ads and more …
Sign up for free

AI Agentにおける評価指標とAgent GPA

Sponsored · Ship Features Fearlessly Turn features on and off without deploys. Used by thousands of Ruby developers. →
Avatar for tsho tsho
February 26, 2026
Technology
600
1

AI Agentにおける評価指標とAgent GPA

第61回 MLOps 勉強会の発表です。AI Agentの評価指標であるAgent GPAの説明です。

Avatar for tsho

tsho

February 26, 2026

More Decks by tsho

See All by tsho
20260930_Gemma4_Hands-on
Avatar for tsho tsho
0
270
Benchmarking Vector Databases: pgvector vs. LanceDB
Avatar for tsho tsho
0
180
Google IO 2026で進化をとげた Google AI Studio
Avatar for tsho tsho
0
38
20260321_エンベディングってなに？RAGってなに？エンベディングの説明とGemini Embedding 2 の紹介
Avatar for tsho tsho
0
300
Scale SciPy with jax.shard_map - jax.shard_mapで SciPy をスケール
Avatar for tsho tsho
0
180
Developer Advocate / Community Managerなるには？
Avatar for tsho tsho
0
890
25/04/12 - Build with AI Hands-on Appendix
Avatar for tsho tsho
1
110
Unit testしてますか？
Avatar for tsho tsho
1
700

Other Decks in Technology

See All in Technology
AI導入は済んだ。次は統制を作り替える番だ_AI×開発組織 Summit 2026
Avatar for PERSOL CAREER Dev | techtekt techtekt
PRO
0
390
JAWSUG TOHOKU 青森支部 「LoGoチャット」のAWS移行で 乗り越えたこと
Avatar for たけだ taketakekaho
0
350
パスキーの自動テストはどこまでいけるのか
Avatar for teamLab teamlab
PRO
0
290
スプシの手作業からの解脱 ~更新はClaude Codeに、見せ方と使い方はGASアプリに任せる~
Avatar for KentaroFujii kentarofujii
0
160
Optunaの最新リリースv5.0について
Avatar for Preferred Networks pfn
PRO
0
120
Refining Active Record
Avatar for Shugo Maeda shugo
0
170
OptunaHub: ブラックボックス最適化のためのプラットフォーム
Avatar for Preferred Networks pfn
PRO
0
140
FinOps Agentが生んだ技術負債:コスト自動調査の落とし穴と解消までの道のり
Avatar for T.REX oasis1994liveforever
2
160
オントロジーって 結局なにもの？ AI エージェントに「意味」を渡す知識設計の話
Avatar for TomoyukiMurakami greenmidori
0
240
コンテナ運用におけるネットワーク・コンテナコストの削減方法／How to Reduce Network and Container Costs in Container Operations
Avatar for モブエンジニア masakiokuda
2
320
SharedWorkerを使ったキャッシュでネイティブアプリに負けないユーザー体験を目指す
Avatar for KAKEHASHI kakehashi
PRO
1
110
1つのリクエストを追って理解する - llm-dのP/D分離
Avatar for akhino akhino
1
170

Featured

See All Featured
Become a Pro
Avatar for Speaker Deck speakerdeck
PRO
31
6.3k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
Avatar for Marc Duiker marcduiker
35
2.6k
Fight the Zombie Pattern Library - RWD Summit 2016
Avatar for Marcelo Somers marcelosomers
234
18k
Abbi's Birthday
Avatar for Violet Bock coloredviolet
4
10k
Applied NLP in the Age of Generative AI
Avatar for Ines Montani inesmontani
PRO
4
2.5k
Accessibility Awareness
Avatar for Sarah Abderemane sabderemane
1
250
HTML-Aware ERB: The Path to Reactive Rendering @ RubyCon 2026, Rimini, Italy
Avatar for Marco Roth marcoroth
5
830
How to Ace a Technical Interview
Avatar for Jacob Kaplan-Moss jacobian
280
24k
Learning to Love Humans: Emotional Interface Design
Avatar for Aarron Walter aarron
275
41k
Design of three-dimensional binary manipulators for pick-and-place task avoiding obstacles (IECON2024)
Avatar for konakalab konakalab
0
630
Highjacked: Video Game Concept Design
Avatar for Ryan Kendrick rkendrick25
PRO
1
480
Side Projects
Avatar for Sacha Greif sachag
456
43k

Transcript

  1. © 2026 Snowflake Inc. All Rights Reserved AI Agent における評価手法と

    Agent GPA Sho Tanaka Feb 2026

  2. © 2026 Snowflake Inc. All Rights Reserved AI/ML, Dataの登壇やデモ開発を担当 -

    ex-Google gTech Ads, ML/Data - MLOps community 運営 (2020~) - Google Developer Expert, AI/ML tsho / 田中 翔 (Sho Tanaka) Linkedin.com/in/tsho Lead Developer Advocate @ Snowflake

  3. © 2026 Snowflake Inc. All Rights Reserved AI Agentとは?

  4. © 2026 Snowflake Inc. All Rights Reserved

  5. © 2026 Snowflake Inc. All Rights Reserved AI Agent の活用事例

    メルカリにおけるデータアナリティクス AI エージェント「Socrates」と ADK 活用 事例 - Speaker Deck コクヨ、ジンズなどがAIエージェント自社 開発 「Snowflake Intelligence」日本提 供

  6. © 2026 Snowflake Inc. All Rights Reserved

  7. © 2026 Snowflake Inc. All Rights Reserved AIOpsは2016年ごろにガー トナーが定義したものもあ るので注意

  8. © 2026 Snowflake Inc. All Rights Reserved

  9. © 2026 Snowflake Inc. All Rights Reserved 評価手法

  10. © 2026 Snowflake Inc. All Rights Reserved AI Agent /

    LLM による代表的な評価指標

  11. © 2026 Snowflake Inc. All Rights Reserved 例：ADK の評価指標 Why

    Evaluate Agents - Agent Development Kit (ADK) LLM-as-a-judge Final_response_match_v2, rubric_based_final_response_qual ity_v1 etc. Code-based / Deterministic コード・ルールベース/一致 tool_trajectory_avg_score Traditional NLP Metrics 従来の自然言語処理指標 response_match_score Human Evaluation 人間による評価 (機能として明示的な「指標」はない が、Web UI (Trace View) で支援)

  12. © 2026 Snowflake Inc. All Rights Reserved Agent GPA と

    TruLens

  13. © 2026 Snowflake Inc. All Rights Reserved

  14. © 2026 Snowflake Inc. All Rights Reserved

  15. © 2026 Snowflake Inc. All Rights Reserved

  16. © 2026 Snowflake Inc. All Rights Reserved Agent GPA の論文

    What Is Your Agent's GPA? A Framework for Evaluating Agent Goal-Plan-Action Alignment

  17. © 2026 Snowflake Inc. All Rights Reserved OSS としても提供中 https://github.com/truera/trulens

    https://www.trulens.org/

  18. © 2026 Snowflake Inc. All Rights Reserved https://www.trulens.org/getting_started/quickstarts/web-search-agent-evaluation/#10-add- evaluations

  19. © 2026 Snowflake Inc. All Rights Reserved

  20. © 2026 Snowflake Inc. All Rights Reserved さいごに

  21. © 2026 Snowflake Inc. All Rights Reserved Snowflake 上で Private

    Preview として提供中 What’s Your Agent’s GPA? A Framework for Evaluating AI Agent Reliability

  22. © 2026 Snowflake Inc. All Rights Reserved 参考

  23. © 2026 Snowflake Inc. All Rights Reserved CS 329T: Trustworthy

    Machine Learning

  24. © 2026 Snowflake Inc. All Rights Reserved https://learn.deeplearning.ai/

  25. © 2026 Snowflake Inc. All Rights Reserved THANK YOU