LLMの評価-近藤憲児

2023/05/11【第1回】ChatGPT活用LT会 by LLM福岡

Transcript

1. גࣜձࣾεΧΠσΟεΫ r  --. ͷධՁ גࣜձࣾεΧΠσΟεΫ "*Τϯδϯ։ൃνʔϜ ۙ౻ݑࣇ ʲୈճʳ$IBU(15׆༻-5ձ CZ
   --.෱Ԭ
2. גࣜձࣾεΧΠσΟεΫ r  ۙ౻ݑࣇ ͜ΜͲ͏͚Μ͡ • εΧΠσΟεΫͷ"*Τϯδϯ։ൃνʔϜϚ ωʔδϟʔ • ੜ࢈ܭըΛ "* ͷྗͰཱҊ͢ΔαʔϏε ࠷దϫʔΫε Λ࡞ͬͯ·͢ɻ • 6EFNZ
   ʹίʔεΛग़ͯ͠·͢ • 5XJUUFS ΍ͬͯ·͢ 👉 !LPOEPLFOKJCBJ
3. גࣜձࣾεΧΠσΟεΫ r  લճͷ-5ձͷࢿྉ΋ͥͻޚཡ͍ͩ͘͞ɻ IUUQTTQFBLFSEFDLDPNLFOKJLPOEPCBJTVLBJEFJTVLVOPMMNOPRVSJ[VNJKJO UFOHYJBOFS
4. גࣜձࣾεΧΠσΟεΫ r  ࠷ద2"͘Μ
5. גࣜձࣾεΧΠσΟεΫ r  ࠷ద2"͘Μͷ֓ཁ • ࠷దϫʔΫεͷϚχϡΞ ϧʹؔ͢Δ࣭໰ʹ౴͑ͯ ͘ΕΔ 4MBDL
   #PU • ϲ݄લʹ࡞ͬͯɺࣾ಺Έ ΜͳͰར༻Ͱ͖Δঢ়ଶʹ ͨ͠ɻ 1P$ ͷ໨త • ࠷ऴతʹ͸͓٬༷ʹ΋࢖ ͑Δঢ়ଶʹঢ՚͍ͤͨ͞ɻ ͦͷͨΊͷݕূɻ • ·ͨ --.
   ͷ׆༻ઌͷΠ ϝʔδΛɺࣾһશһ͕๲ Β·ͤΒΕΔΑ͏ʹ͢Δ ͨΊɻ 質問 最適QAくん (最適ワークスの マニュアルを熟知 した Slack Bot) 回答
6. גࣜձࣾεΧΠσΟεΫ r  --. ͷධՁͷԞਂ͞
7. גࣜձࣾεΧΠσΟεΫ r  ྫ͑͹ WFDUPS
   TFBSDI
   Λ IZCSJE
   TFBSDI
   ʹม͑ΔͳͲ͠ ͯɺ )BMMVDJOBUJPO Λ๷͙౒ྗ Λͨ͠ͱ͢Δɻ • ͦΕʹΑͬͯʮຊ౰ʹվળ͞ Εͨʯͱ͍͏ͷΛɺͲͷΑ͏ ʹଞਓʹઆ໌͢Δʁ • Ξ΢τϓοτͷྑ͠ѱ͠͸Ͳ ͷΑ͏ʹͯ͠٬؍తʹܭଌͰ ͖Δʁ • ͞ΒʹͦΕΛࣗಈͰ΍Δͱ͠ ͨΒͲͷΑ͏ʹͰ͖Δʁ ධՁʹͩ͜ΘΓͨ͘ͳΔཧ༝
8. גࣜձࣾεΧΠσΟεΫ r  ͜͜Ͱݴ͍ͬͯΔධՁͱ͸ɺൺֱͷ͜ͱɻ ൺֱͷϙΠϯτ͸৚݅Λݻఆ͢Δ͜ͱɻҰ෦Λݻఆͯͦ͜͠Ҏ֎ Λม͑ͯൺֱ͢Δɻ৚݅͸ϑΣΞͰͳ͖Ό͍͚ͳ͍ɻ ͱ͜ΖͰɺධՁͱ͍͏ͷ͸ɺࣾձௐࠪσʔλͷ෼ੳ΍ػցֶशͷ ਓʹ͸͓ͳ͡Έ͚ͩͲɺͦ΋ͦ΋೉͍͠ɻ ˠ ʮποίϛͲ͜Ζ͕Ұͭ΋ͳ͍ɺඇͷଧͪॴͷͳ͍ධՁ͸ɺ ͜ͷੈͷதʹଘࡏ͠ͳ͍ʯͱ͍͏͘Β͍ʹࢥ͓͍͍͍ͬͯͯɻ
9. גࣜձࣾεΧΠσΟεΫ r  ྫ͑͹ɺ࠷ۙಡΜͩຊͷྫ 数字のセンスを磨く〜データの読み⽅・活かし⽅ (光⽂社新書 1241) 新書 – 2023/2/15 筒井 淳也 (著) ʮ݁ࠗͨ͠ޙʹ਌ͱಉډ͠Α͏ͱ͢ Δ෉්͕ݮ͍ͬͯΔʯ ͱ͍͏͜ͱΛ਺ࣈͰࣔ͢ʹ͸ɺͲͷΑ ͏ͳൺֱΛߦ͑͹Α͍͔ʁ ˠ ؆୯Ͱ͋Ζ͏ɻੈ୅ͰॅΜͰ͍Δ Ո଒ ௚ܥՈ଒ ͷׂ߹ͷਪҠΛൺֱ͠ ͯΈΕ͹Α͍͡Όͳ͍͔ɻ ˠ ͍΍ɺͦ΋ͦ΋݁ࠗ͠ͳ͍ਓ΋૿ ͍͑ͯΔͷ͔ͩΒɺͦΕ͕ཧ༝Ͱݮͬ ͍ͯΔ͔΋͠Εͳ͍ɻͦΕΛߟྀ͠ͳ ͍ͱϑΣΞͳൺֱͰ͸ͳ͍ɻ
10. גࣜձࣾεΧΠσΟεΫ r  ྫ͑͹ɺ࠷ۙಡΜͩຊͷྫ ˠ Ͱ͸ɺʮ݁ࠗͯ͠਌ͱಉډ͢Δ ͔Ͳ͏͔ͷҙࢥ͕͋Δਓͷͳ͔Ͱɺ ಉډͨ͠ਓʯͷׂ߹ΛΈΕ͹Α͍ɻ ˠ ͦΕ͸ྑͦ͞͏ɻͨͩɺܑఋ͕ ͨ͘͞Μ͍Δ৔߹͸ɺͦͷ͏ͪͷ Ұਓ͔͠਌ͱಉډ͠ͳ͍ͷ͕ৗͳ ͷ͔ͩΒɺͦ͜Λߟྀ͠ͳ͍ͷ͸ ϑΣΞͰ͸ͳ͍ɻ ˠ ΋ͬͱݴ͑͹ɺ෉්ͷ͏ͪͲͪ Β͔ͷ਌ͱಉډ͢Δબ୒ͳͷ͔ͩ Βɺ͜ͷબ୒͸Λ௒͑Δ͜ͱ ͸ͳ͍ ˠ ΋ͬͱݴ͑͹ɺʮ݁ࠗޙ͸͠͹ Β͘ผډ͕ͩɺߴྸʹͳͬͯ਌ͱ ಉډ͢ΔʯਓΛɺͲͷΑ͏ʹΧ΢ ϯτ͢Δ͔ͰɺϑΣΞ͕͞มΘͬ ͯ͘Δɻ ˠ ΋ͬͱݴ͑͹ɺ཭ࠗͨ͠ਓΛͲ ͏ߟ͑Δ͔΋ߟྀ͠ͳ͚Ε͹ͳΒ ͳ͍ɻʮ཭ࠗͯ͠ࢠڙͱҰॹʹ࣮ Ոʹؼͬͨਓʯ͸ʮ݁ࠗͨ͠ޙʹ ਌ͱಉډ͠Α͏ͱ͢Δ෉්͕ݮͬ ͍ͯΔʯͱ͍͏࠷ॳͷ໰୊ҙࣝʹ ֘౰͢Δਓ͔ʁ ˠ ͓ෲ͍ͬͺ͍ʂ
11. גࣜձࣾεΧΠσΟεΫ r  ग़ྗͨ݁͠Ռͱʮࣄ࣮ͱͷൺֱʯΛߦ͑͹Α͍ɻ ؆୯ͦ͏ʹݟ͑Δɻ --. ͷΞ΢τϓοτͷධՁ͸Ͳ͏͔ʁ
12. גࣜձࣾεΧΠσΟεΫ r  Ͱ΋ɺҎԼͷΑ͏ͳ࣭໰ʹର͢Δ౴͑͸ɺͲ͏ʁ ʮ؍ଌՄೳͳӉ஦શମʹ͓͍ͯੜ໋͸ଘࡏ͍ͯ͠Δ͔ʁʯ ʮ࠷௿௞ۚ๏͸ࣦۀʹͲͷΑ͏ͳӨڹΛ༩͑Δ͔ʁʯ ͜ͷΑ͏ͳ࣭໰ʹ͸ෳ਺ͷ߹ཧతͳճ౴͕͋ΓಘΔɻͦΕΒճ ౴͸૬ޓʹໃ६͠͏Δ΋ͷͰ΋͋Δɻ ˠ ͜ΕΒέʔε͸ɺ୯७ͳʮࣄ࣮ͱͷൺֱʯͱ͍ͬͨ΋ͷ͔ ΒҰาઌΛߟ͑Ͷ͹ͳΒͳ͍ɻ
13. גࣜձࣾεΧΠσΟεΫ r  ͞Βʹݴ͑͹ɺʮ࠷௿௞ۚ๏͸ࣦۀʹͲͷΑ͏ͳӨڹΛ༩͑Δ ͔ʁʯ΁ͷճ౴͕ " ʮ࠷௿௞͕ۚߴͯ͘΋ޏ༻ΛݮΒ͞ͳ͍ʯ Ͱ͸ͳ͘ # ʮ࠷௿௞ۚ๏͸ޏ༻Λ࣮࣭తʹݮΒ͢Θ͚Ͱ͸ͳ͍Α͏ʹݟ ͑·͕͢ɺ͜ͷࠜڌʹ൓ର͢Δਓ΋ଟ਺ଘࡏ͠·͢ʯ ͩͬͨ৔߹ɻ " ͸ࣗ෼ʹࣗ৴͍࣋ͪ͗ͯ͢ΔͷͰͩΊ # ͸ࣗ৴ͷແ͞Λૉ௚ʹද໌͍ͯ͠ΔͷͰྑͦ͞͏ɻ ˠ # ͷճ౴Λྑ͍΋ͷͱͯ͠ධՁ͍ͯ͋͛ͨ͠ؾ࣋ͪʹͳΔɻ
14. גࣜձࣾεΧΠσΟεΫ r  ͡Ό͋ɺ͖ͬ͞ͷ͕͍͍ͱͯ͠ɺ --. ͷΞ΢τϓοτ͕ ʮࢲ "* ʹ͸஍ٿ͕ฏ໘ͩͱࢥ͍ͬͯ·͕͢ɺੈքதͷ΄ͱ ΜͲͷਓʑɺՊֶऀΛؚΊͯ൓ରҙݟ͕͋Γ·͢ɻʯ ͩͬͨΒɺ͜Ε͸ຊ౰ʹྑ͍ճ౴ʁ஍ٿ͕ฏ໘ͩͱ৴͍ͯ͡Δ "* ͷग़ྗ͢Δճ౴Λɺ͍͘Βݠڏ͔ͩΒͱݴͬͯ΋๻Β͸ڐ༰ Ͱ͖Δ͔ʁ ·ͨɺ͜͏͍͏෩ʹࣗ৴ͷແ͞Λ͍͍ͪͪ఻͑Δ͜ͱ͸΄Μͱ ʹྑ͍ճ౴ͩΖ͏͔ʁ༧๷ઢΛషΓ·ͬͨ͘Ξ΢τϓοτ͹͔ ΓΛग़͢͜ͱ͸ɺຊ౰ʹΑ͍͜ͱ͔ʁ௕ͨΒ͍͠αʔϏεར༻ ن໿ͷΑ͏ͳճ౴Λ͢Δͷ͸ɺ๻Β͕ٻΊͯΔ "* ͔ʁ
15. גࣜձࣾεΧΠσΟεΫ r  ͱ͍͏෩ʹɺ໎͍Ͳ͜Ζ͕ͳͦ͞͏ͳͱ͜Ζ΋ɺগ͠ઌʹ͸প ͕͋Δɻ ͱ͍͏͜ͱͰɺ·ͣ׬ᘳͳποίϛͷͳ͍ධՁΛ໨ࢦ͢ͷ͸΍ ΊΔɻ ͨͩɺ໰୊͕͋ΔͷͰධՁΛ΍Βͳ͍ɺͱ͍͏Θ͚Ͱ͸ͳ͍ɻ ͢΂ͯͷൺֱ͕ແҙຯɺͱ͍͏Θ͚Ͱ΋ͳ͍ɻ ˠ ੍໿΍ݶքΛߟྀ͠ͳ͕ΒɺධՁΛ͠·͠ΐ͏ɻ
16. גࣜձࣾεΧΠσΟεΫ r  ධՁͷ࣮ફ
17. גࣜձࣾεΧΠσΟεΫ r  2"$IBJO • -BOH$IBJO Λ ࢖ ͬ ͨ ධ Ձ ͷ ํ๏ • --. ʹධՁ͞ ͤ Δ ɺ ͱ ͍ ͏ ΞΠσΟΞ • ˠ ධՁͷࣗಈ Խ͕Մೳʹ
18. גࣜձࣾεΧΠσΟεΫ r  You are a teacher grading a quiz. You are given a question, the student's answer, and the true answer, and are asked to score the student answer as either CORRECT or INCORRECT. (…) Grade the student answers based ONLY on their factual accuracy. Ignore differences in punctuation and phrasing between the student answer and true answer. It is OK if the student answer contains more information than the true answer, as long as it does not contain any conflicting statements. Begin! QUESTION: 1+1 STUDENT ANSWER: 3 TRUE ANSWER: 2 GRADE: ઌఔͷίʔυΛ࣮ߦͨ͠ͱ͖ʹ࣮ࡍʹཪଆͰ࣮ߦ͞ΕΔ QSPNQU
19. גࣜձࣾεΧΠσΟεΫ r  • ʮ--. Ͱ --. ΛධՁ͢ΔͬͯͲ͏ͳͷʁʯͱ΋ࢥ͕ͬͨɺ ࣮ࡍɺ (15 ΋ (15 ࣗ਎Λ࢖ͬͯ )BMMVDJOBUJPO ͷධՁ Λͨ͠Β͍͠ɻ • ͜ͷ͋ͨΓ͕ɺલஈͰʮ׬ᘳͳධՁΛٻΊͳ͍ʯͱ͍͏͜ͱΛ͘Ͳ͘ Ͳͱઆ໌ͨ͠ཧ༝ GPT-4 System Card, OpenAI, March 23, 2023 ੈͷதతʹ͸࣮ࡍͲ͏΍ͬͯΔ͔ɻ
20. גࣜձࣾεΧΠσΟεΫ r  • ଞͷධՁपΓͷ՝୊ͱͯ͠ɺσʔληοτΛूΊΔͱ͜Ζ͕ ͋ΔɻͦΕΛίϛϡχςΟͷྗͰղܾ͠Α͏ͱ͍ͯ͠Δɻ • ྫ͑͹ FWBMT • ྫ͑͹ -BOH$IBJO%BUBTFUT ੈͷதతʹ͸࣮ࡍͲ͏΍ͬͯΔ͔ɻ
21. גࣜձࣾεΧΠσΟεΫ r  3FGFSFODFT • -BOH$IBJO ͷ &WBMVBUJPO
    IUUQTQZUIPOMBOHDIBJODPNFOMBUFTUVTF@DBTFTFWBMVBUJPO IUNM • 5SVUIGVM "*
    IUUQTBSYJWPSHBCT • ͪΌΜͱ )BMMVDJOBUJPO ΍ӕͷఆٛΛߟ͑ΔͳΒ͹·ͣ͜ΕΛಡΉͷ͕͍ ͍ɻ • (15
    4ZTUFN
    $BSE
    IUUQTDEOPQFOBJDPNQBQFSTHQU TZTUFNDBSEQEG • PQFOBJFWBMT
    IUUQTHJUIVCDPNPQFOBJFWBMT • -BOH$IBJO%BUBTFUT
    IUUQTIVHHJOHGBDFDP-BOH$IBJO%BUBTFUT
22. גࣜձࣾεΧΠσΟεΫ r