Upgrade to Pro — share decks privately, control downloads, hide ads and more …

AI Agentにおける評価指標とAgent GPA

Avatar for tsho tsho
February 26, 2026
Technology
1
380

AI Agentにおける評価指標とAgent GPA

第61回 MLOps 勉強会の発表です。AI Agentの評価指標であるAgent GPAの説明です。

Avatar for tsho

tsho

February 26, 2026
Tweet

More Decks by tsho

See All by tsho
20260321_エンベディングってなに？RAGってなに？エンベディングの説明とGemini Embedding 2 の紹介
Avatar for tsho tsho
0
100
Scale SciPy with jax.shard_map - jax.shard_mapで SciPy をスケール
Avatar for tsho tsho
0
100
Developer Advocate / Community Managerなるには？
Avatar for tsho tsho
0
700
25/04/12 - Build with AI Hands-on Appendix
Avatar for tsho tsho
1
75
Unit testしてますか？
Avatar for tsho tsho
1
650

Other Decks in Technology

See All in Technology
20260311 技術SWG活動報告（デジタルアイデンティティ人材育成推進WG Ph2 活動報告会）
Avatar for OpenID Foundation Japan oidfj
0
370
【Λ（らむだ）】最近のアプデ情報 / RPALT20260318
Avatar for Λ lambda
0
110
エンジニアリングマネージャーの仕事
Avatar for YuheiNakasaka yuheinakasaka
0
110
Postman v12 で変わる API開発ワークフロー (Postman v12 アップデート) / New API development workflow with Postman v12
Avatar for Yoichi Kawasaki yokawasa
0
140
複数クラスタ運用と検索の高度化：ビズリーチにおけるElastic活用事例 / ElasticON Tokyo2026
Avatar for Visional Engineering ＆ Design visional_engineering_and_design
0
170
「通るまでRe-run」から卒業！落ちないテストを書く勘所
Avatar for asumikam asumikam
2
210
NewSQL_ ストレージ分離と分散合意を用いたスケーラブルアーキテクチャ
Avatar for hacomono Inc. hacomono
PRO
4
400
Go標準パッケージのI/O処理をながめる
Avatar for matumoto matumoto
0
230
1GB RAMのラズピッピで何ができるのか試してみよう / 20260319-rpijam-1gb-rpi-whats-possible
Avatar for Akira Ouchi akkiesoft
0
500
コンテキスト・ハーネスエンジニアリングの現在
Avatar for Hirosato Gamo hirosatogamo
PRO
4
510
2026年もソフトウェアサプライチェーンのリスクに立ち向かうために / Product Security Square #3
Avatar for GMO Flatt Security flatt_security
1
670
Keycloak を使った SSO で CockroachDB にログインする / CockroachDB SSO with Keycloak
Avatar for kota2and3kan kota2and3kan
0
160

Featured

See All Featured
Visual Storytelling: How to be a Superhuman Communicator
Avatar for David Neal reverentgeek
2
470
Understanding Cognitive Biases in Performance Measurement
Avatar for Philip Tellis bluesmoon
32
2.8k
<Decoding/> the Language of Devs - We Love SEO 2024
Avatar for Nikki Halliwell nikkihalliwell
1
160
Breaking role norms: Why Content Design is so much more than writing copy - Taylor Woolridge
Avatar for UX Y'all uxyall
0
210
From Legacy to Launchpad: Building Startup-Ready Communities
Avatar for Dug Song dugsong
0
180
Exploring anti-patterns in Rails
Avatar for Elle Meredith aemeredith
2
290
Ecommerce SEO: The Keys for Success Now & Beyond - #SERPConf2024
Avatar for Aleyda Solis aleyda
1
1.9k
Introduction to Domain-Driven Design and Collaborative software design
Avatar for Kenny Baas-Schwegler baasie
1
640
The Art of Programming - Codeland 2020
Avatar for Erika Heidi erikaheidi
57
14k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
Avatar for Kevin Liebholz kevinliebholz
37
6.3k
Helping Users Find Their Own Way: Creating Modern Search Experiences
Avatar for Dan Newman danielanewman
31
3.1k
The AI Revolution Will Not Be Monopolized: How open-source beats economies of scale, even for LLMs
Avatar for Ines Montani inesmontani
PRO
3
3.1k

Transcript

  1. © 2026 Snowflake Inc. All Rights Reserved AI Agent における評価手法と

    Agent GPA Sho Tanaka Feb 2026

  2. © 2026 Snowflake Inc. All Rights Reserved AI/ML, Dataの登壇やデモ開発を担当 -

    ex-Google gTech Ads, ML/Data - MLOps community 運営 (2020~) - Google Developer Expert, AI/ML tsho / 田中 翔 (Sho Tanaka) Linkedin.com/in/tsho Lead Developer Advocate @ Snowflake

  3. © 2026 Snowflake Inc. All Rights Reserved AI Agentとは?

  4. © 2026 Snowflake Inc. All Rights Reserved

  5. © 2026 Snowflake Inc. All Rights Reserved AI Agent の活用事例

    メルカリにおけるデータアナリティクス AI エージェント「Socrates」と ADK 活用 事例 - Speaker Deck コクヨ、ジンズなどがAIエージェント自社 開発 「Snowflake Intelligence」日本提 供

  6. © 2026 Snowflake Inc. All Rights Reserved

  7. © 2026 Snowflake Inc. All Rights Reserved AIOpsは2016年ごろにガー トナーが定義したものもあ るので注意

  8. © 2026 Snowflake Inc. All Rights Reserved

  9. © 2026 Snowflake Inc. All Rights Reserved 評価手法

  10. © 2026 Snowflake Inc. All Rights Reserved AI Agent /

    LLM による代表的な評価指標

  11. © 2026 Snowflake Inc. All Rights Reserved 例：ADK の評価指標 Why

    Evaluate Agents - Agent Development Kit (ADK) LLM-as-a-judge Final_response_match_v2, rubric_based_final_response_qual ity_v1 etc. Code-based / Deterministic コード・ルールベース/一致 tool_trajectory_avg_score Traditional NLP Metrics 従来の自然言語処理指標 response_match_score Human Evaluation 人間による評価 (機能として明示的な「指標」はない が、Web UI (Trace View) で支援)

  12. © 2026 Snowflake Inc. All Rights Reserved Agent GPA と

    TruLens

  13. © 2026 Snowflake Inc. All Rights Reserved

  14. © 2026 Snowflake Inc. All Rights Reserved

  15. © 2026 Snowflake Inc. All Rights Reserved

  16. © 2026 Snowflake Inc. All Rights Reserved Agent GPA の論文

    What Is Your Agent's GPA? A Framework for Evaluating Agent Goal-Plan-Action Alignment

  17. © 2026 Snowflake Inc. All Rights Reserved OSS としても提供中 https://github.com/truera/trulens

    https://www.trulens.org/

  18. © 2026 Snowflake Inc. All Rights Reserved https://www.trulens.org/getting_started/quickstarts/web-search-agent-evaluation/#10-add- evaluations

  19. © 2026 Snowflake Inc. All Rights Reserved

  20. © 2026 Snowflake Inc. All Rights Reserved さいごに

  21. © 2026 Snowflake Inc. All Rights Reserved Snowflake 上で Private

    Preview として提供中 What’s Your Agent’s GPA? A Framework for Evaluating AI Agent Reliability

  22. © 2026 Snowflake Inc. All Rights Reserved 参考

  23. © 2026 Snowflake Inc. All Rights Reserved CS 329T: Trustworthy

    Machine Learning

  24. © 2026 Snowflake Inc. All Rights Reserved https://learn.deeplearning.ai/

  25. © 2026 Snowflake Inc. All Rights Reserved THANK YOU