Share
Embed
さくらのクラウド高火力プランを使って 大規模言語モデル(LLM)を動かしてみよう
オープンソースカンファレンス2023 オンライン北海道 セミナープログラム登壇資料
https://event.ospn.jp/osc2023-online-do/session/958849
Hikaru Ashino
June 17, 2023
Transcript
47 slides
-
2 • ୲ۀ • αʔϏεͷόοΫΤϯυ։ൃɺΠϯϑϥ։ൃ • ӴσʔλϓϥοτϑΥʔϜʮTellusʢςϧʔεʣʯ • ͘͞ΒͷVPSɺ͘͞ΒͷΫϥυ •
ܦྺ • 2012 - 2016 ITܥઐֶߍ4՝ఔͰֶͿ ߴઐ࢜ଔ • OSC2012 SendaiͰॳΊͯOSSͷੈքΛΔ • 2013 - 2016 MSPͱϗεςΟϯάΛߦ͏ձࣾʹΞϧόΠτೖࣾ • OpenStackɺLinux KVMΛ༻͍ͨԾڥͷߏஙӡ༻ • OSSͰߏங͞ΕͨγεςϜͷӡ༻ɺτϥϒϧγϡʔςΟϯά • 2016 - ݱ৬ ͘͞ΒΠϯλʔωοτʹ৽ଔೖࣾ • 2017 - 2019 ܳज़ܥେֶӃʹͯ2ݚڀ͢Δ ܳज़ֶम࢜ Twitter: @tar_xzvff -
ձࣾհ 3 ձࣾ֓ཁ ຊࣾॴࡏ େࡕେࡕࢢ۠കా1-12-12 ౦ژݐകాϏϧ 11F (202110݄Ҡస) ۀ݄ 199612݄23
ʢձࣾઃཱ: 19998݄17ʣ ্݄ 200510݄12ʢϚβʔζʣ 201511݄27ʢ౦ূҰ෦ʢݱϓϥΠϜ ࢢʣࢢมߋʣ ࢿຊۚ 22ԯ5,692ສԁ ैۀһ ࿈݁ 710໊ʢ20223݄ʣ άϧʔϓձࣾ ΞΠςΟʔΤϜגࣜձࣾ גࣜձࣾS2i ᓎՖҠಈి৴༗ݶެ࢘ ήώϧϯגࣜձࣾ Ϗοτελʔגࣜձࣾ ϓϥφειϦϡʔγϣϯζגࣜձࣾ IzumoBASEגࣜձࣾ BBSakura Networksגࣜձࣾ ΠϯλʔωοτΠϯϑϥͷఏڙΛओͳࣄۀ༰ͱ͠·ͯ͠ɺ େࡕɺ౦ژɺੴङͷ3Ҭʹ5ͭͷσʔληϯλʔΛల։ -
ձࣾհ 4 VPSɾΫϥυ σʔληϯλʔ ৽αʔϏε ԾԽٕज़Λ༻͍ɺ 1ͷཧαʔό্ ʹෳͷԾαʔό Λߏங͠ɺԾઐ༻ αʔόͱ͚ͯͨ͠
ྖҬͷ༗αʔϏε ߴੑೳαʔόͱ֦ு ੑͷߴ͍ωοτϫʔ ΫΛѹతͳίετ ύϑΥʔϚϯεͰར ༻Ͱ͖ΔIaaSܕύϒ ϦοΫɾΫϥυɾ αʔϏε ߴੑೳͰ֦ுੑͱ৴པੑͷߴ ͍αʔόΛ·Δ͝ͱಠͯ͠ ར༻͢Δ͜ͱ͕Ͱ͖ɺࣗ༝ʹ ΧελϚΠζͯ͠ར༻Մೳͳ αʔϏε ϋδϯά ϦϞʔτϋδϯά σʔληϯλʔʹ͓٬༷ઐ ༻ͷϋδϯάεϖʔεΛ֬ อ͠ɺωοτϫʔΫػث αʔόͳͲͷػࡐΛࣗ༝ʹஔ ͚ΔαʔϏε ػցֶशɺσʔλղੳɺߴਫ਼γϛϡϨʔγϣϯ༻్ʹಛԽͨ͠GPUࡌͷ ઐ༻αʔόαʔϏε ઐ༻αʔό ͘͞ΒͷηΩϡΞϞόΠϧίωΫτ ΫϥυʹμΠϨΫτʹଓ͠ɺηΩϡΞͰ͋ΓͭͭҙͷωοτϫʔΫ ଓՄೳͳSIMΛఏڙ͢ΔɺIoT͚ϞόΠϧαʔϏε ̖̞ ਓೳ IoT Ϩϯλϧαʔό 1ͷαʔόΛෳͷܖऀ ͰαʔόΛڞ༗·ͨ༗͢ Δ͜ͱ͕Ͱ͖ɺཧ͘͞Β Πϯλʔωοτʹͤͯ͏ αʔϏε ઐۀͰશํҐʹύϒϦοΫͳαʔϏεΛఏڙ͍ͯ͠Δࠃ།Ұͷࣄۀऀ͔ͩΒͦ͜Έ߹Θͤͯબࢶ͕͕ΔɺͦΕ͕ʮ͘͞ΒΠϯλʔωοτʯͷڧΈ ݄*4."1औಘ ΠϯλʔωοτΠϯϑϥͷఏڙΛࣄۀυϝΠϯʹɺ େࡕ/౦ژ/ੴङʹσʔληϯλʔΛల։ɻ େࡕ/౦ژ/ੴङΛ100GbpsͰ݁ͼͭͭɺ ର֎ଓͷ૯ܭ 1.84Tbps ͷωοτϫʔΫͰ ࠃͷΠϯλʔωοττϥϑΟοΫΛࢧ͍͑ͯ·͢ɻ (20234݄ݱࡏ) ੴङσʔληϯλʔ3߸౩ʢӈଆʣ ੴङσʔληϯλʔશܠ Ӊ Tellus(ςϧʔε)ɺຊൃͷӴσʔλϓϥοτϑΥʔϜͰ͢ɻ ӴσʔλͷఏڙΛ͡Ίͱ͠ɺσʔλΛར༻ͨ͠ ৽ͨͳϏδωεΛग़͢ΔڥΛ͝༻ҙ͍ͯ͠·͢ɻ -
ΞδΣϯμ • ʮ͘͞ΒͷΫϥυߴՐྗϓϥϯʯ֓ཁɺࣄྫ • େنݴޠϞσϧ(LLM)ͱ • ओͳେنݴޠϞσϧ(LLM)ʹ͍ͭͯ • ͘͞ΒͷΫϥυͰେنݴޠϞσϧΛಈ͔ͨ͢ΊͷखॱͷղઆɺσϞ •
େنݴޠϞσϧΛηϧϑϗετ͢ΔϝϦοτ • ͓Βͤ 5 -
ʮ͘͞ΒͷΫϥυߴՐྗϓϥϯʯ֓ཁ 7 ͘͞ΒͷΫϥυͰར༻Ͱ͖ΔGPUαʔόϓϥϯ ϋΠ パ ϑΥʔϚϯεͳ GPUʮNVIDIA V100ʯΛॳظඅ༻ෆཁ で 1
࣌ؒ୯Ґ͔ΒΫϥ ド ͷ͍উखͦͷ··ʹར༻ で ͖Δαʔ ビ ε εϖοΫ ఏڙκʔϯ ੴङୈ̍κʔϯ CPU 4vCPU ϝϞϦ 56GB GPUΧʔυ NVIDIA V100 (32GB) x 1 GPUࡌϝϞϦ 32GB ༻్ྫ: ػցֶशɺσΟʔϓϥʔχϯάɺHPC -
ʮ͘͞ΒͷΫϥυߴՐྗϓϥϯʯࣄྫ • ओͳҰྫ • ػցֶशʢϞσϧͷ࡞ɺਪʣ • ੜAI(Generative AI)ܥ • େنݴޠϞσϧ(LLM)
• Ի߹ • ݚڀ։ൃ • ಈըΤϯίʔυ • VRܥ 9 -
ʮ͘͞ΒͷΫϥυߴՐྗϓϥϯʯࣄྫ • ੜAI(Generative AI)ܥ ը૾ੜ 12 ͘͞ΒͷΫϥυʬGPUϓϥϯʭϋϯζΦϯ ʙStable DiffusionΛ༻ͨ͠ϑΝϯδʔΞʔτͷ࡞Γํ https://qiita.com/zembutsu/items/ae7809501b7e07700cf9
Stable Diffusion web UI https://github.com/AUTOMATIC1111/stable-diffusion-webui -
ʮ͘͞ΒͷΫϥυߴՐྗϓϥϯʯࣄྫ • ੜAI(Generative AI)ܥ ίʔσΟϯάΞγελϯτ 13 ͘͞ΒͷΫϥυͷGPUαʔό(Tesla V100)Ͱ Tabby(GitHub Copilotͷସ)Λಈ͔͢
https://qiita.com/tar_xzvf/items/85ad8b41168921822ef3 Tabby https://github.com/TabbyML/tabby -
ओͳେنݴޠϞσϧ(LLM)ʹ͍ͭͯ • GPT(Generative Pre-trained Transformer) • OpenAPI͕։ൃͨ͠ϓϦτϨʔχϯάϞσϧ • https://huggingface.co/gpt2 •
ChatGPT͜ΕΛ͍ͬͯΔ(GPT3,GPT3.5,GPT4) • BERT(Bidirectional Encoder Representations from Transformers) • Google͕։ൃͨ͠ϓϦτϨʔχϯάϞσϧ • https://huggingface.co/bert-base-uncased • OPT(Open Pre-trained Transformer) • META͕։ൃͨ͠ϓϦτϨʔχϯάϞσϧ • https://huggingface.co/facebook/opt-13b 18 -
ΦʔϓϯιʔεେنݴޠϞσϧ(LLM)ʹ͍ͭͯ • Dolly https://huggingface.co/databricks/dolly-v2-12b • Datablicks͕ࣾެ։ 120,70,30ԯύϥϝʔλ • ༻ར༻Մೳ •
OpenLLaMAɹhttps://huggingface.co/openlm-research/open_llama_7b • OpenLM Research͕ެ։ 130(600ԯτʔΫϯͰֶश),70,30ԯύϥϝʔλɺMeta AI ͷ LLaMA ͷΦʔϓϯιʔε࣮ • ༻ར༻Մೳ • Falcon https://huggingface.co/tiiuae/falcon-40b • Technology Innovation Institute͕ެ։ 400,70ԯύϥϝʔλ • ༻ར༻Մೳ • StableLM https://huggingface.co/stabilityai/stablelm-tuned-alpha-7b • Stability AI͕ެ։ 70,30ԯύϥϝʔλ • ༻ར༻Մೳ • OpenCALM l https://huggingface.co/cyberagent/open-calm-7b • CyberAgent͕ެ։ͨ͠ຊޠେنݴޠϞσϧɻ࠷େ68ԯύϥϝʔλ • ༻ར༻Մೳ • ຊޠGPT-2/BERTͷࣄલֶशϞσϧ https://huggingface.co/rinna • rinna͕ࣾެ։ • MPT-7B https://huggingface.co/mosaicml/mpt-7b • MosaicML͕ެ։ɺ70ԯύϥϝʔλɻจΛಘҙͱ͍ͯ͠Δ • ༻ར༻Մೳ 19 -
͘͞ΒͷΫϥυͰେنݴޠϞσϧΛಈ͔ͨ͢Ίͷखॱ • ࠓճCyberAgent͕ެ։͍ͯ͠ΔOpenCALMΛಈ͔ͯ͠Έ·͢ • OpenCALM • গͳ͍ຊޠେنݴޠϞσϧͷҰͭ • Hugging Face※Ͱެ։͞Ε͍ͯΔ
21 ※ػցֶशϞσϧΛެ։ɺڞ༗Ͱ͖ΔϓϥοτϑΥʔϜ αΠόʔΤʔδΣϯτɺಠࣗͷຊޠLLMʢେنݴޠϞσϧʣΛ։ൃ ―ࣗવͳຊޠͷจষੜΛ࣮ݱ― https://www.cyberagent.co.jp/news/detail/id=28797 -
͘͞ΒͷΫϥυͰେنݴޠϞσϧΛಈ͔ͨ͢Ίͷखॱ̏ • GPUαʔόͷ࡞ • αʔόՃը໘Ͱ֤߲Λબ 25 ໊߲ αʔόϓϥϯ GPUϓϥϯ
σΟεΫ-σΟεΫϓϥϯ SSDϓϥϯ σΟεΫ-σΟεΫιʔε ΞʔΧΠϒ ΞʔΧΠϒબ Ubuntu 22.04.1 LTS σΟεΫ-σΟεΫαΠζ 100GB σΟεΫमਖ਼-σΟεΫमਖ਼Λ͢Δ νΣοΫ ཧϢʔβͷύεϫʔυ ҙͷύεϫʔυΛೖྗ ϗετ໊ɺެ։伴 ※ඞཁʹԠͯ͡ೖྗɾબ αʔόͷใ-໊લ Θ͔Γ͍͢ҙͷ໊લ ࡞ 1 -
͘͞ΒͷΫϥυͰେنݴޠϞσϧΛಈ͔ͨ͢Ίͷखॱ̒ • GPUυϥΠόͷΠϯετʔϧ • NVIDIAࣾͷυΩϡϝϯτ௨ΓʹΠϯετʔϧΛߦ͍·͢ • https://docs.nvidia.com/datacenter/tesla/tesla-installation-notes/index.html 28 $ sudo
apt-get install linux-headers-$(uname -r) $ distribution=$(. /etc/os-release;echo $ID$VERSION_ID | sed -e 's/\.//g') $ wget https://developer.download.nvidia.com/compute/cuda/repos/$distribution/x86_64/cuda-keyring_1.0-1_all.deb $ sudo dpkg -i cuda-keyring_1.0-1_all.deb $ sudo apt-get update $ sudo apt-get -y install cuda-drivers -
͘͞ΒͷΫϥυͰେنݴޠϞσϧΛಈ͔ͨ͢Ίͷखॱ̓ • PythonϥΠϒϥϦͷΠϯετʔϧ • OpenCALM(LLM)ΛPythonͰѻ͏ͨΊʹϥΠϒϥϦΛΠϯετʔϧ͠·͢ 29 $ sudo apt install
python3-pip $ pip3 install transformers accelerate -
͘͞ΒͷΫϥυͰେنݴޠϞσϧΛಈ͔ͨ͢Ίͷखॱ̔ • αϯϓϧίʔυͷ࣮ߦ • Hugging FaceͰެ։͞Ε͍ͯΔɺOpenCALMͷαϯϓϧίʔυΛಈ͔͠·͢ • https://huggingface.co/cyberagent/open-calm-7b 30 import
torch from transformers import AutoModelForCausalLM, AutoTokenizer model = AutoModelForCausalLM.from_pretrained("cyberagent/open-calm-7b", device_map="auto", torch_dtype=torch. fl oat16) tokenizer = AutoTokenizer.from_pretrained("cyberagent/open-calm-7b") inputs = tokenizer("AIʹΑͬͯࢲୡͷΒ͠ɺ", return_tensors="pt").to(model.device) with torch.no_grad(): tokens = model.generate( **inputs, max_new_tokens=64, do_sample=True, temperature=0.7, top_p=0.9, repetition_penalty=1.05, pad_token_id=tokenizer.pad_token_id, ) output = tokenizer.decode(tokens[0], skip_special_tokens=True) print(output) -
͘͞ΒͷΫϥυͰେنݴޠϞσϧΛಈ͔ͨ͢Ίͷखॱ • αϯϓϧίʔυΛ༻͍ͯOpenCALM(LLM)͕͘͞ΒͷΫϥυ্Ͱಈ͖·ͨ͠ • ৄࡉͳखॱͬͱ؆୯ʹಈ͔ͨ͢ΊͷखॱͷهࣄΛ༻ҙ͠·ͨ͠ • cloud-initͷίʔυ༻ҙ͍ͯ͠·͢ɺίϐϖͰڥ͕ߏஙͰ͖·͢ 32 ͘͞ΒͷΫϥυͷGPUαʔό(Tesla V100)Ͱ
αΠόʔΤʔδΣϯτ͕Ұൠެ։ͨ͠ຊޠLLM(OpenCALM)Λಈ͔ͯ͠ΈΔ https://qiita.com/tar_xzvf/items/09ee2bf146c4a3319492 -
͘͞ΒͷΫϥυͰେنݴޠϞσϧͷσϞɹΞʔΩςΫνϟ • ͘͞ΒͷΫϥυߴՐྗϓϥϯͷαʔό্ʹOpenCALMΛ༻ҙAPIΛ࣮ • Fast APIͰ transformerϞσϧ(OpenCALM)Λݺͼग़͢APIΛ࣮ • Fast APIϦΫΤετʹจষΛडɺͦΕΛOpenCALMʹೖྗ
• OpenCALM͔Βͷग़ྗΛAPIϨεϙϯεͱͯ͠ฦ͢ 36 OpenCALM ͓ݩؾ? ͓ݩؾ? ݩؾͰ͢ ݩؾͰ͢ -
େنݴޠϞσϧΛηϧϑϗετ͢ΔϝϦοτ • ಛఆλεΫΛ࣮ߦ͢ΔͨΊʹΧελϚΠζͨ͠LLMΛ͍͍ͨχʔζ • λεΫݻ༗ͷσʔληοτͰϑΝΠϯνϡʔχϯά͢Δ • σʔληΩϡϦςΟͱϓϥΠόγʔ • ֎෦ʹग़ͤͳ͍σʔλΛͬͨLLMͷ׆༻ •
σʔλ࿙ӮͳͲͷ؍͔Β • ΦϯϓϨϛεͷԸܙ • ͠େنݴޠϞσϧΛಈ͔ͤΔΠϯϑϥ͕͋ΔͷͰ͋Εɺैྔ՝ۚͳͲؾʹ ͤͣʹϞσϧΛಈ͔͢͜ͱ͕Ͱ͖Δɺίετίϯτϩʔϧ • ࠷దԽ(ࣗ৫ʹஔ͘ͷͰɺԠ͕ͯ͘վળͰ͖ΔՄೳੑ͕͋Δ) • ͨͩ͠ిؾ͕͍͢͜͝ͱʹͳΓͦ͏💸 39 -
ײ • ಈ͔͢·Ͱͷखॱ͕γϯϓϧͰ͙͢ʹಈ͔ͤͨ • GPUυϥΠόͱPythonϥΠϒϥϦͷΠϯετʔϧ͚ͩ • ࣗ༝͕ߴͦ͏ɺ൚༻తʹΈࠐΜͩΓͰ͖ͦ͏ • HTTP APIͱͯ͠ଞαʔϏεͱܨ͍ͩΓɺslack
botʹͨ͠Γ • ͏͢͜͠ਫ਼͕΄͍͠ • Ԡ͕ͪΐͬͱظͣΕͷ࣌͋Δ • ϑΝΠϯνϡʔχϯάͯ͠Έ͍ͨ • ۀͰ׆༻ͯ͠Έ͍ͨ • ࣗࣾΫϥυͰಈ͘͜ͱ͕͔Γخ͍͠ • ҟͳΔGPUͰࠓޙݕূͯ͠Έ͍ͨͰ͢ • LLMΛ׆༻ͯ͠Έ͍ͨɺਂ۷Γͯ͠Έ͍ͨɺ࡞ͬͯΈ͍ͨ • ·ͣػցֶशΛ͡Ί͔Βֶͼ͍ͨͱࢥ͍·͢ 40 -
ࠂ • 6/24() Open Source Conference 2023 Hokkaido (లࣔ) 43
GPUαʔόʔΛ࣌ؒ୯ҐͰར༻Մೳͳʮ͘͞ΒͷΫϥυߴՐྗϓϥϯʯΛ͝հ͠·͢ɻ ͋ΘͤͯɺάοζΫʔϙϯͷߦ͍·͢ɻ ࠓճͷσϞలࣔ͠·͢ʂ https://ospn.connpass.com/event/285754/ -
͘͞ΒΠϯλʔωοτͰΤϯ ジ χΞ࠾༻ΛڧԽ͍ͯ͠·͢ ͘͞ΒΠϯλʔωοτ৽ͨͳΞΠ デ Ξͷग़ʹڧ͍ҙͱΛ࣋ͬͯઓ͢Δ͓٬༷Λ じ Ίɺࢲͨͪͱͭͳ が Γͷ͋Δ͢
べ ͯͷਓͨͪͷͨΊʹɺະདྷͷ͋Δ べ ͖࢟Λ͍ඳ͖ͳ が Β ― ʮΓ͍ͨ͜ͱʯΛʮ で ͖Δʯʹม͑Δ ― ͋ΒΏΔΞ プ ϩʔνΛ “Πϯλʔωοτ”Λ௨ じ ͯఏ ڙ͠·͢ɻ SAKURA internet ࣾձΛࢧ͑Δ ύϒϦοΫΫϥυΛ Ұॹʹ࡞Γ·ͤΜ͔ʁ ιϑτΣΞ։ൃɺ Πϯϑϥج൫͔Β ϑϩϯτΤϯυ·Ͱ ࠾༻ڧԽத! ৄ͘͠WebαΠτʹͯɺΧδϡΞϧ໘ஊͬͯ·͢ 👉 www.sakura.ad.jp/lp/22engineer/