Upgrade to Pro — share decks privately, control downloads, hide ads and more …

AI Agentにおける評価指標とAgent GPA

Avatar for tsho tsho
February 26, 2026
Technology
510
1

AI Agentにおける評価指標とAgent GPA

第61回 MLOps 勉強会の発表です。AI Agentの評価指標であるAgent GPAの説明です。

Avatar for tsho

tsho

February 26, 2026

More Decks by tsho

See All by tsho
Google IO 2026で進化をとげた Google AI Studio
Avatar for tsho tsho
0
7
20260321_エンベディングってなに？RAGってなに？エンベディングの説明とGemini Embedding 2 の紹介
Avatar for tsho tsho
0
240
Scale SciPy with jax.shard_map - jax.shard_mapで SciPy をスケール
Avatar for tsho tsho
0
140
Developer Advocate / Community Managerなるには？
Avatar for tsho tsho
0
820
25/04/12 - Build with AI Hands-on Appendix
Avatar for tsho tsho
1
94
Unit testしてますか？
Avatar for tsho tsho
1
670

Other Decks in Technology

See All in Technology
41歳でAWSが好きすぎてITエンジニアになったおっさんの話
Avatar for Yuuki Yamashita yama3133
1
730
Type-safe IaC for Dart
Avatar for Nozomi Koborinai coborinai
0
180
AI_Dev_Day_製造業領域でのAI活用から見た活用の罠と成功に導く実践知.pdf
Avatar for KintoTech_Dev kintotechdev
0
150
そのドキュメント、自動化しませんか？
Avatar for yuk yuksew
1
400
生成 AI 時代にいま一度「問い合わせ」について考えてみる
Avatar for kazzpapa3 kazzpapa3
1
130
副作用のある Lambda でも Lambda Power Tuning は使えるのか / lambda-power-tuning-side-effects
Avatar for koukihosaka koukihosaka
1
140
Amplify Gen2でbackend.tsにCDKを定義する/しない事によるCDKの挙動の違いとユースケース
Avatar for Makky12 smt7174
1
480
GoでCコンパイラを作った話
Avatar for repunit11 repunit
0
150
壊して学ぶAWS CDK: そのcdk deployで消えるもの、残るもの
Avatar for Kazuki Adachi k_adachi_01
1
470
複数プロダクトで進めるAI機能実装 ── 実践から得たリアルな学びとロードマップ実現への挑戦 / AICon2026_yanari
Avatar for Rakus_Dev rakus_dev
1
270
Jitera Company Deck
Avatar for Jitera Inc. jitera
0
270
なぜ、あなたのAPIは使われないのか？ AX時代の設計原則、ガードレール、運用体制
Avatar for Yoichi Kawasaki yokawasa
1
200

Featured

See All Featured
Designing for Timeless Needs
Avatar for Cassini Nazir cassininazir
1
390
How to Talk to Developers About Accessibility
Avatar for Jason CranfordTeague jct
2
420
Chasing Engaging Ingredients in Design
Avatar for Sebastian Deterding codingconduct
0
240
How to optimise 3,500 product descriptions for ecommerce in one day using ChatGPT
Avatar for Katarina Dahlin katarinadahlin
PRO
1
3.7k
Ethics towards AI in product and experience design
Avatar for Skipper Chong Warson skipperchong
2
330
Money Talks: Using Revenue to Get Sh*t Done
Avatar for Nikki Halliwell nikkihalliwell
0
410
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
Avatar for Rebecca Miller-Webster rmw
35
3.7k
Visual Storytelling: How to be a Superhuman Communicator
Avatar for David Neal reverentgeek
2
590
Getting science done with accelerated Python computing platforms
Avatar for Jacob Tomlinson jacobtomlinson
2
340
Art, The Web, and Tiny UX
Avatar for Lynn Fisher lynnandtonic
304
22k
Public Speaking Without Barfing On Your Shoes - THAT 2023
Avatar for David Neal reverentgeek
1
460
Speed Design
Avatar for Sergey Chernyshev sergeychernyshev
33
1.9k

Transcript

  1. © 2026 Snowflake Inc. All Rights Reserved AI Agent における評価手法と

    Agent GPA Sho Tanaka Feb 2026

  2. © 2026 Snowflake Inc. All Rights Reserved AI/ML, Dataの登壇やデモ開発を担当 -

    ex-Google gTech Ads, ML/Data - MLOps community 運営 (2020~) - Google Developer Expert, AI/ML tsho / 田中 翔 (Sho Tanaka) Linkedin.com/in/tsho Lead Developer Advocate @ Snowflake

  3. © 2026 Snowflake Inc. All Rights Reserved AI Agentとは?

  4. © 2026 Snowflake Inc. All Rights Reserved

  5. © 2026 Snowflake Inc. All Rights Reserved AI Agent の活用事例

    メルカリにおけるデータアナリティクス AI エージェント「Socrates」と ADK 活用 事例 - Speaker Deck コクヨ、ジンズなどがAIエージェント自社 開発 「Snowflake Intelligence」日本提 供

  6. © 2026 Snowflake Inc. All Rights Reserved

  7. © 2026 Snowflake Inc. All Rights Reserved AIOpsは2016年ごろにガー トナーが定義したものもあ るので注意

  8. © 2026 Snowflake Inc. All Rights Reserved

  9. © 2026 Snowflake Inc. All Rights Reserved 評価手法

  10. © 2026 Snowflake Inc. All Rights Reserved AI Agent /

    LLM による代表的な評価指標

  11. © 2026 Snowflake Inc. All Rights Reserved 例：ADK の評価指標 Why

    Evaluate Agents - Agent Development Kit (ADK) LLM-as-a-judge Final_response_match_v2, rubric_based_final_response_qual ity_v1 etc. Code-based / Deterministic コード・ルールベース/一致 tool_trajectory_avg_score Traditional NLP Metrics 従来の自然言語処理指標 response_match_score Human Evaluation 人間による評価 (機能として明示的な「指標」はない が、Web UI (Trace View) で支援)

  12. © 2026 Snowflake Inc. All Rights Reserved Agent GPA と

    TruLens

  13. © 2026 Snowflake Inc. All Rights Reserved

  14. © 2026 Snowflake Inc. All Rights Reserved

  15. © 2026 Snowflake Inc. All Rights Reserved

  16. © 2026 Snowflake Inc. All Rights Reserved Agent GPA の論文

    What Is Your Agent's GPA? A Framework for Evaluating Agent Goal-Plan-Action Alignment

  17. © 2026 Snowflake Inc. All Rights Reserved OSS としても提供中 https://github.com/truera/trulens

    https://www.trulens.org/

  18. © 2026 Snowflake Inc. All Rights Reserved https://www.trulens.org/getting_started/quickstarts/web-search-agent-evaluation/#10-add- evaluations

  19. © 2026 Snowflake Inc. All Rights Reserved

  20. © 2026 Snowflake Inc. All Rights Reserved さいごに

  21. © 2026 Snowflake Inc. All Rights Reserved Snowflake 上で Private

    Preview として提供中 What’s Your Agent’s GPA? A Framework for Evaluating AI Agent Reliability

  22. © 2026 Snowflake Inc. All Rights Reserved 参考

  23. © 2026 Snowflake Inc. All Rights Reserved CS 329T: Trustworthy

    Machine Learning

  24. © 2026 Snowflake Inc. All Rights Reserved https://learn.deeplearning.ai/

  25. © 2026 Snowflake Inc. All Rights Reserved THANK YOU