Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
アバターミーティングを支える技術
Search
ねおりん
January 13, 2021
Technology
0
610
アバターミーティングを支える技術
Gotanda.unity #16
https://meetup.unity3d.jp/jp/events/1268
https://youtu.be/FTHfVpBfpo0
ねおりん
January 13, 2021
Tweet
Share
More Decks by ねおりん
See All by ねおりん
UIElements 最高! 一番好きなUIフレームワークです!
noir_neo
0
620
24時間くらいで作ったゲーム 2019年
noir_neo
0
1.3k
24時間でゲームをつくるための Unity
noir_neo
2
1.9k
Multiuser AR Experience を ARKit + Unity で実装する
noir_neo
0
2.6k
これで安心! hubot 沼の堕ち方〜娘がいると開発が楽しいね!
noir_neo
0
350
MDLT#004
noir_neo
0
1.5k
FU2 展示反省
noir_neo
0
120
FU2進捗141004
noir_neo
0
150
FU2進捗140920
noir_neo
0
130
Other Decks in Technology
See All in Technology
研究開発部メンバーの働き⽅ / Sansan R&D Profile
sansan33
PRO
3
17k
データプレーンプログラミングとは? DPU&スイッチASICの開発経験から語る
ebiken
PRO
1
270
MCP で繋ぐ Figma とデザインシステム〜LLM を使った UI 実装のリアル〜
kimuson
2
1.3k
SmartHRの複数のチームにおけるMCPサーバーの活用事例と課題
yukisnow1823
2
1.2k
OTel meets Wasm: プラグイン機構としてのWebAssemblyから見る次世代のObservability
lycorptech_jp
PRO
1
300
プラットフォームとしての Datadog / Datadog as Platforms
aoto
PRO
1
340
データ戦略部門 紹介資料
sansan33
PRO
1
3.1k
大事なのは、AIの精度だけじゃない!〜1円のズレも許されない経理領域とAI〜
jun_nemoto
11
5.2k
新卒から4年間、20年もののWebサービスと向き合って学んだソフトウェア考古学 - PHPカンファレンス新潟2025 / new graduate 4year software archeology
oguri
2
360
やさしいClaude Code入門
minorun365
PRO
32
25k
AIに実況させる / AI Streamer
motemen
3
1.4k
AIのための オンボーディングドキュメントを整備する - hirotea
hirotea
9
2.3k
Featured
See All Featured
How to train your dragon (web standard)
notwaldorf
92
6k
Stop Working from a Prison Cell
hatefulcrawdad
269
20k
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
Product Roadmaps are Hard
iamctodd
PRO
53
11k
Side Projects
sachag
454
42k
Reflections from 52 weeks, 52 projects
jeffersonlam
349
20k
A better future with KSS
kneath
239
17k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
29
1.7k
Fireside Chat
paigeccino
37
3.5k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
14
1.5k
A designer walks into a library…
pauljervisheath
205
24k
BBQ
matthewcrist
88
9.7k
Transcript
ΞόλʔϛʔςΟϯάΛࢧ͑ Δٕज़ 2021/1/13 Gotanda.unity #16 Ͷ͓ΓΜ 1
noir_neo • Ͷ͓ΓΜ • Ϋϥελʔגࣜձࣾ • ιϑτΣΞΤϯδχΞ • Ϋϥελʔͷϋτ •
2021ͷඪʮ͓෩࿊ͷ࣌ܭͷి Λ͑Δʯ 2
ͱ͜ΖͰΈͳ͞ΜɺϏ σΦϛʔςΟϯά͠· ͢ΑͶ? 3
ฐࣾɺVRϛʔςΟϯά વΓ·͕͢ɺϏσΦϛʔ ςΟϯά͠·͢ VRHMDඃͬͯϛʔςΟϯάϊʔτऔΔͷେมͳͨΊɻ 4
͜Μͳײ͡ → 5
6
ҰൠతͳϏσΦϛʔςΟϯά ͷޫܠͰ͢Ͷ? (ͦ͏Ͱ͋ͬͯ΄͍͠) 7
͍ΘΏΔ VTuber ΞϓϦ ͍Ζ͍Ζ͋Δɻ 8
͍Ζ͍Ζࢼͨ݁͠ՌɺࣗͰ ࡞Γ࢝Ίͨ ͦ͏͍͏͜ͱ͋Δɻ 9
͢͜ͱ • ͍ΘΏΔ VTuber ΞϓϦͷߏྫ • ݟɺͩ͜ΘΓϙΠϯτͳͲ 10
ݟग़͠ • γεςϜ֓ཁ • إͱखͷτϥοΩϯά • ΞόλʔͷಡΈࠐΈɾදࣔɾΞχϝʔγϣϯ • ө૾ͷૹग़ 11
γεςϜ֓ཁ 12
13
إͱखͷτϥοΩϯά • ARKit • Leap Motion 14
ARKit Face Tracking • iOS σόΠεͷϑϩϯτΧϝϥͰϑΣΠετϥοΩϯά͢Δͭ • Unity Ͱ ARFoundation,
ARKit Face Tracking plugin Λ͏ 15
ARKit Face Tracking • Ϋϥογϡ͢Δ • σʔλڸ • දͷϚοϐϯά 16
Ϋϥογϡ͢Δ • ͠Β͘ಈ͔ͯ͠Δͱ EXC_BAD_ACCESS ͰམͪΔ • ARFoundation 3.x ͳΒͳ͍ •
͜Ε͕͋ͬͯ ARFoundation 4.x ʹ্͛ΒΕͣɺ Unity2020.2 ʹ ্͛ΒΕͳ͍ " • Issue ͰiOS14ͷͱݴΘΕ͍ͯΔ(मਖ਼Λ͔ͭ͠ͳͦ͞͏) > github.com/Unity-Technologies/arfoundation-samples/issues/ 707 17
Face Tracking ͷσʔλڸ • ARKit ͷ Face Tracking ͰऔಘͰ͖Δσʔλࠨӈసͯ͠ೖͬͯ ͘Δ
• ΧϝϥͷલͰӈΛ͘ͱࠨճసΛɺӈΛด͡ΔͱࠨΛด ͡ΔΑ͏ͳσʔλΛฦ͢ 18
إͷ pose Λڸࣸ͠ʹ͢Δ rotation.x *= -1; rotation.y *= -1; position.z
*= -1; 19
BlendShapeLocation ͷࠨӈΛೖΕସ ͑Δ 20
{ BlendShapePreset.Blink_L, new [] { ARKitBlendShapeLocation.EyeBlinkRight } }, { BlendShapePreset.Blink_R,
new [] { ARKitBlendShapeLocation.EyeBlinkLeft } } 21
ʮࠨΛݟ͍ͯΔʯɺʮ(͔ͬͯ)ࠨͷ͕ଆΛݟ͍ͯΔʯͱ ʮ(͔ͬͯ)ӈͷ͕֎ଆΛݟ͍ͯΔʯ { BlendShapePreset.LookLeft, new [] { ARKitBlendShapeLocation.EyeLookInLeft, ARKitBlendShapeLocation.EyeLookOutRight }
}, { BlendShapePreset.LookRight, new [] { ARKitBlendShapeLocation.EyeLookOutLeft, ARKitBlendShapeLocation.EyeLookInRight } }, 22
දͷϚοϐϯά • ARKitͰऔಘͰ͖Δ Blend Shape Ωʔ(ଟ͘ͷ)Ξόλʔͷ࣋ͭͦ ΕͱҟͳΔ • ಉ͡ Blend
Shape ΛΞόλʔʹ༻ҙͯ͠ɺͦͷ··ύϥϝʔ λʔΛөͤ͞ΔʮύʔϑΣΫτγϯΫʯͱݺΕΔ֓೦ ͋Δ 23
දͷϚοϐϯά • aiueo ͷαϯϓϧΛऔ͓͍ͬͯͯ࠷ۙࣅ͢ΔͷΛ࠾༻ɺͰͩ ͍͍͍͍ͨײ͡ʹͳͬͨ • ۃʹֻ͚ͨΓɺ2൪ʹࣅͯΔͷΛࠞͥͨΓɺͪΌΜͱޱ Λด͡ΔΑ͏ʹ͢Δͷʹຊͬͱ͍Ζ͍ΖͬͯΔ • ͜͜ͷڍಈਓʹΑͬͯΈ͔ΕΔͱࢥ͏ͷͰࣗ࡞ͯͯ͠
໘ന͍ͱ͜Ζ 24
Leap Motion • ϋϯυδΣενϟʔίϯτϩʔϥʔ • 2012ͷσόΠε • ڈ(20209݄) IR 170
ͱ͍͏݁ߏੑೳ্͕͕ͬͨධՁΩοτ͕ ൃച͞ΕͯΞπ͍ • Unity ϓϥάΠϯ͕ఏڙ͞Ε͍ͯΔ 25
Leap Motion • جຊతʹ LeapServiceProvider or LeapXRServiceProvider ίϯϙʔωϯτΛ࣮͑؆୯ • LeapServiceProvider
صஔ͖ • LeapXRServiceProvider टͱ͔಄ʹϚϯτ͢Δ༻ • ࣗͰ࣮͢Δ(pure class ʹ͍ͨ͠ͱ͔ɺصஔ͖ͱटϚϯτ ΛΓସ͑ΒΕΔΑ͏ʹ͍ͨ͠)߹ɺ PolicyFlag ΛΓସ͑ Δͷͱɺ࠷దͳ Interpolate ͷΦϓγϣϯ͕ҧ͏ͬΆ͍ʹҙ 26
ΞόλʔͷಡΈࠐΈɾදࣔɾ Ξχϝʔγϣϯ • VRM • Custom UTI • Animation Rigging
27
VRM • VRΞϓϦέʔγϣϯ͚ͷਓܕ3DΞόλʔσʔλΛѻ͏ͨΊͷ ϑΝΠϧϑΥʔϚοτ • UniVRM (C#ʹΑΔඪ४࣮)Λ͏ • (શʹࣗ༻ͩͬͨΒΞϓϦʹຒΊࠐΜͰྑ͍͕ɺͲ͏ͤΞ όλʔΑ͘मਖ਼͢ΔͷͰϥϯλΠϜϩʔυʹରԠ͓ͯ͘͠ͱ
ָ) 28
Custom UTI • iOS ͷڞ༗ϘλϯΛԡͨ͠ͱ͖ʹग़ͯ ͘ΔϦετʹ Unity ΞϓϦΛฒΔ • ͜ΕͰVRMϑΝΠϧΛ։͚Δͱຯʹ
ศར • (શʹࣗ༻ͩͬͨΒΞϓϦʹຒΊ ࠐΜͰྑ͍͕ɺͲ͏ͤΞόλʔΑ ͘मਖ਼͢ΔͷͰϥϯλΠϜϩʔυʹର Ԡ͓ͯ͘͠ͱָ) 29
Info.plist Λ͕Μͬͯॻ͘ [PostProcessBuild] public static void OnPostProcessBuild(BuildTarget target, string path)
{ if (target != BuildTarget.iOS) return; var plistPath = path + "/Info.plist"; var plist = new PlistDocument(); plist.ReadFromString(File.ReadAllText(plistPath)); var rootDict = plist.root; var exportedTypeDeclarations = rootDict.CreateArray("UTExportedTypeDeclarations"); var exportedTypeDeclarationsDict = exportedTypeDeclarations.AddDict(); var typeConformsTo = exportedTypeDeclarationsDict.CreateArray("UTTypeConformsTo"); typeConformsTo.AddString("public.data"); exportedTypeDeclarationsDict.SetString("UTTypeDescription", "VRM"); exportedTypeDeclarationsDict.CreateArray("UTTypeIconFiles"); var typeIdentifier = $"{PlayerSettings.applicationIdentifier}.custom"; exportedTypeDeclarationsDict.SetString("UTTypeIdentifier", typeIdentifier); var typeTagSpecification = exportedTypeDeclarationsDict.CreateDict("UTTypeTagSpecification"); var typeTagSpecificationArray = typeTagSpecification.CreateArray("public.filename-extension"); typeTagSpecificationArray.AddString("vrm"); var documentTypes = rootDict.CreateArray("CFBundleDocumentTypes"); var documentTypesDict = documentTypes.AddDict(); documentTypesDict.SetString("CFBundleTypeName", "VRM"); documentTypesDict.SetString("CFBundleTypeRole", "Viewer"); documentTypesDict.SetString("LSHandlerRank", "Alternate"); var itemContentTypes = documentTypesDict.CreateArray("LSItemContentTypes"); itemContentTypes.AddString(typeIdentifier); File.WriteAllText(plistPath, plist.WriteToString()); } https:/ /gist.github.com/noir-neo/ 91e7a09e87e15280c1b2c70648b833f1 30
͋ͱURLͷϋϯυϦϯάͱಉ͡ @interface UrlHandlerAppController : UnityAppController @end @implementation UrlHandlerAppController - (BOOL)application:(UIApplication
*)application openURL:(NSURL *)url options:(NSDictionary<UIApplicationOpenURLOptionsKey, id> *)options { // TODO: Handling return YES; } @end ͨͩ͜͜͠Ͱड͚औͬͨURI࣍ճىಈ࣌ʹ͑ͳ͍ͷͰΞό λʔΛΩϟογϡ͓ͯ͘͠ΈͳͲ͕ผ్ඞཁ 31
Animation Rigging • "ΞχϝʔγϣϯԽ͞Ε͍ͯΔεέϧτϯʹର࣮ͯ͠ߦ࣌ʹϓϩ γʔδϟϧͳϞʔγϣϯΛηοτΞοϓͰ͖ΔΑ͏ʹ͢Δ" • ཁΞχϝʔγϣϯΛϥϯλΠϜͰ͍͍ײ͡ʹ্ॻ͖Ͱ͖Δ • Unity2020.2 Ͱ
verified ʹͳΓ·ͨ͠Ͷ • σϑΥϧτͷΞχϝʔγϣϯʹରͯ͠(τϥοΩϯάͨ͠)֤Ϙʔ ϯͷճసΛ্ॻ͖͢Δͷʹศར 32
Animation Rigging • TwoBoneIK ͕ඪ४Ͱೖ͍ͬͯΔͷͰͷ IK ʹ͑Δ(݁ߏ͍͍ ײ͡) • (಄ͱख͔ΒମશମΛΑ͠ͳʹIK͢ΔɺΈ͍ͨͳͷ
FinalIK ͱ͔͏ํָ͕) • มͳճసΛೖΕͯ muscle ͷൣғͰಈ͔ͯ͘͠ΕΔͷͰ࠷ѱͷ ഁͷํ͠ͳ͍ 33
ө૾ͷૹग़ • ө૾ͷϛϥʔ • Spout / Syphon / NDI 34
ࣗͰݟΔө૾ڸʹ͢Δ • Unity Ͱ Camera ͷө૾ΛϑϦοϓ͢Δʹ projectionMatrix Λ͍ ͡ΔΒ͍͠ •
culling ͱ͔ΊΜͲͦ͘͞͏ • Render Texture ʹඳ͍ͨͷΛ Canvas ͷ UV Ͱసͤ͞Δͷ͕ ָ • ͍ͭͰʹɺૹग़͢Δө૾ʹ UI ΛೖΕͨ͘ͳ͍ͱ͔͋Δͷ Ͱແବͳ͍ 35
Spout / Syphon / NDI • ө૾(ͷϑϨʔϜ)Λڞ༗͢Δϓϩτίϧ • Spout(Windows) /
Syphon(mac) Ϛγϯ • NDI ϩʔΧϧωοτϫʔΫ • Unity Ͱ Klak{Spout|Syphon|NDI} Λ͏ • keijiroਆ ! 36
Spout / Syphon / NDI • ಁաө૾ѻ͑ͯ৴ιϑτʹೖΕΒΕΔͷͰάϦʔϯ όοΫͰൈ͍ͨΓ͠ͳͯ͘Α͍ͷͰΑ͍ 37
·ͱΊ • Unity ͰΩϛ͚ͩͷόʔνϟϧΧϝϥΛ࡞ͬͯɺϏσΦϛʔςΟ ϯάͰࠩΛ͚Α͏ʂ 38
39