Upgrade to Pro — share decks privately, control downloads, hide ads and more …

レンズの下のLLM / LLM under the Lens

Avatar for Henry Cui Henry Cui
December 30, 2023
Programming
0
190

レンズの下のLLM / LLM under the Lens

Avatar for Henry Cui

Henry Cui

December 30, 2023
Tweet

More Decks by Henry Cui

See All by Henry Cui
プロダクション言語モデルの情報を盗む攻撃 / Stealing Part of a Production Language Model
Avatar for Henry Cui zchenry
1
210
Direct Preference Optimization
Avatar for Henry Cui zchenry
0
390
Diffusion Model with Perceptual Loss
Avatar for Henry Cui zchenry
0
420
Go with the Prompt Flow
Avatar for Henry Cui zchenry
0
170
Mojo Dojo
Avatar for Henry Cui zchenry
0
220
ことのはの力で画像の異常検知 / Anomaly Detection by Language
Avatar for Henry Cui zchenry
0
590
驚愕の事実！LangChainが抱える問題 / Problems of LangChain
Avatar for Henry Cui zchenry
0
260
MLOps初心者がMLflowを触る / MLflow Brief Introduction
Avatar for Henry Cui zchenry
0
130
{{guidance}}のガイダンス / Guidance of guidance
Avatar for Henry Cui zchenry
0
170

Other Decks in Programming

See All in Programming
私はどうやって技術力を上げたのか
Avatar for Yusuke Wada yusukebe
43
18k
技術的負債の正体を知って向き合う / Facing Technical Debt
Avatar for irof irof
0
120
Goで実践するドメイン駆動開発 AIと歩み始めた新規プロダクト開発の現在地
Avatar for imkaoru imkaoru
4
760
CSC509 Lecture 04
Avatar for Javier Gonzalez-Sanchez javiergs
PRO
0
300
CSC509 Lecture 03
Avatar for Javier Gonzalez-Sanchez javiergs
PRO
0
330
After go func(): Goroutines Through a Beginner’s Eye
Avatar for Vaibhav Gupta 97vaibhav
0
240
Things You Thought You Didn’t Need To Care About That Have a Big Impact On Your Job
Avatar for Holly Cummins hollycummins
0
180
CSC509 Lecture 02
Avatar for Javier Gonzalez-Sanchez javiergs
PRO
0
410
2分台で1500examples完走！爆速CIを支える環境構築術 - Kaigi on Rails 2025
Avatar for Hayato OKUMOTO falcon8823
3
3.4k
フロントエンド開発に役立つクライアントプログラム共通のノウハウ / Universal client-side programming best practices for frontend development
Avatar for nrs nrslib
7
3.9k
登壇は dynamic! な営みである / speech is dynamic
Avatar for Daichi KUDO da1chi
0
110
クラシルを支える技術と組織
Avatar for raku rakutek
0
190

Featured

See All Featured
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
Avatar for Tammy Everts tammyeverts
54
3k
Designing Experiences People Love
Avatar for Jonathan Moore moore
142
24k
KATA
Avatar for Megan Lloyd mclloyd
32
15k
Improving Core Web Vitals using Speculation Rules API
Avatar for Sergey Chernyshev sergeychernyshev
19
1.2k
Stop Working from a Prison Cell
Avatar for Chris Michel hatefulcrawdad
271
21k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
Avatar for mongodb mongodb
48
9.7k
Why You Should Never Use an ORM
Avatar for John Nunemaker jnunemaker
PRO
59
9.6k
Being A Developer After 40
Avatar for Adrian Kosmaczewski akosma
91
590k
Optimising Largest Contentful Paint
Avatar for Harry Roberts csswizardry
37
3.4k
It's Worth the Effort
Avatar for 3n 3n
187
28k
No one is an island. Learnings from fostering a developers community.
Avatar for Antonio thoeni
21
3.5k
The MySQL Ecosystem @ GitHub 2015
Avatar for Sam Lambert samlambert
251
13k

Transcript

  1. レンズの下のLLM 機械学習の社会実装勉強会第30回 Henry 2023/12/30

  2. LLM開発 ▪ LLM開発に必要な機能 • プロンプトエンジニアリングの繰り返し • 実験管理・性能評価・結果比較 ▪ これらの機能を達成する急成長のレポジトリ trulens

    2

  3. TruLens-Eval ▪ カルフォルニアにある会社TruEraのプロダクト • MLのMonitor. Debug. Test.にフォーカス ▪ TruLens-EvalはLLMの実験管理のために開発された ▪

    TruLens-Explainは深層モデルの解釈性のために開発された ▪ 今日はTruLens-Evalに入門 3

  4. TruLens-Evalを使う ▪ ライブラリインストール pip install trulens-eval==0.19.0 • 最新版の0.20.0では不明のエラーでimportできなかった ▪ シンプルなllm_app

    • 2つの引数はサポートされてない ▪ Feedbackのカスタマイズ ▪ trulens-evalコマンドでstreamlitを開く • streamlit-javascriptが必要 4

  5. まとめ ▪ TruLens-Evalの基本的な使い方 ▪ 余力ある方はLangChainなどとの組み合わせ 5