Multiuser AR Experience を ARKit + Unity で実装する

(リアルタイム) Multiuser AR Experience を ARKit + Unity で (比較的簡単に) 実装する方法
主に自作プラグイン https://github.com/noir-neo/UnityARKitMultipeerConnectivity の紹介

デモ動画: https://www.youtube.com/watch?v=DzNEOHz8JwA

Transcript

1. Multiuser AR Experience Λ ARKit + Unity Ͱ࣮૷͢Δ 2018/10/5 Unity x ARKit ೖ໳ Meetup Ͷ͓ΓΜ (@noir_neo) 1
2. Ͷ͓ΓΜ • @noir_neo • Shoma SATO • AR Ͱਓ޻஌ೳͷঁͷࢠͱ฻Β͍ͨ͠ 2
3. (ϦΞϧλΠϜ) Multiuser AR Experience Λ ARKit + Unity Ͱ (ൺֱత؆୯ʹ) ࣮૷͢Δํ ๏ (ओʹࣗ࡞ϓϥάΠϯͷ঺հ) 3
4. ֓ཁ • ARKit 2.0 ͱ Multiuser Experience • ࣮૷Ҋ 1. UNet 2. ϦΞϧλΠϜ௨৴ήʔϜαʔό 3. MultipeerConnectivity ←ࠓ೔ͷϝΠϯ 4. Cloud Anchors 4
5. ARKit 2.0 ͱ Multiuser Experience 5
6. ARKit 2.0 ͷ AR World Map • ෳ਺σόΠεؒͰ AR ੈքΛڞ༗Ͱ͖ΔΑ͏ʹͳͬͨ • ͨͩ͠ ARKit ͕ఏڙ͍ͯ͠Δػೳ͸2ͭ • AR ੈքͷอଘɾಡࠐ (serialize / deserialize) • AR ੈքͷ࠶ݱ (relocalize) 6
7. ͜Ε͚ͩ͡Όڞ༗͸Ͱ͖ͳ ͍! ࣗ෼ͷ୺຤಺Ͱอଘͯ͋͠ͱͰ࠶ݱͰ͖Δ͚ͩ 7
8. ͡Ό͋Ͳ͏΍ͬͯڞ༗͢Δ ͷ? ։ൃऀʹҕͶΒΕΔ ! 8
9. ࣮૷Ҋ 9
10. 1. UNet • Unity ͕ఏڙ͍ͯ͠ΔϚϧνϓϨΠϠʔػೳ • ͷϩʔΧϧ௨৴Λ࢖͏ɺͱ͍͏࿩ • Unity Technologies ͷαϯϓϧ1Ͱ΋౰વ࢖ΘΕ͍ͯΔ 1 https:/ /github.com/Unity-Technologies/SharedSpheres 10
11. 1. UNet Pros • ݱঢ়Ұ൪ΧδϡΞϧʹ࢖͑ͦ͏ • σϑΥϧτͰೖͬͯΔ • ϦΞϧλΠϜ௨৴ήʔϜΛ࡞Γ΍͍͢ 11
12. 1. UNet Cons • ಉ͡ LAN ʹܨ͕͍ͬͯΔඞཁ͕͋Δ • ARWorldMap (਺ඦKB)ΛૹΔʹ͸޻෉͕ඞཁ • νϟϯΫʹͯ͠ૹΔ(ઌड़ͷαϯϓϧʹৄ͘͠ॻ͔Ε͍ͯΔ) • ஈ֊తʹഇࢭ༧ఆ 12
13. 2. ϦΞϧλΠϜ௨৴ήʔϜαʔό • ϦΞϧλΠϜ௨৴ήʔϜαʔόͷ SaaS Λ࢖͏ɺͱ͍͏࿩ • Photon ΍ϞϊϏοτΤϯδϯͳͲ 13
14. 2. ϦΞϧλΠϜ௨৴ήʔϜαʔό Pros • Πϯλʔωοτʹܨ͕͍ͬͯΕ͹OK • ϦΞϧλΠϜ௨৴ήʔϜΛ࡞Γ΍͍͢ 14
15. 2. ϦΞϧλΠϜ௨৴ήʔϜαʔό Cons • ಉۭؒ͡ʹ͍ΔϢʔβΛಉ͡ϧʔϜʹೖΕΔͨΊʹ޻෉͕ඞཁ • ϚονϯάαʔόʔΛཱͯͯҐஔ৘ใΛ࢖ͬͯղܾ͢Δͱ ͔ɺۙڑ཭௨৴Λ࢖͏ͱ͔ • Ϣʔβಉ࢜ͰϧʔϜ ID Λղܾͯ͠΋Β͏ • ϥϯχϯάίετ(αʔόʔ୅)͕͔͔Δ 15
16. 3. MultipeerConnectivity • iOS Ͱ୺຤ؒ P2P Wi-Fi ΍ Bluetooth ௨৴Λ͢Δ framework • Apple ͷ Swift ͷαϯϓϧ2Ͱ࢖͍ͬͯΔ 2 https:/ /developer.apple.com/documentation/arkit/creatingamultiuserarexperience 16
17. 3. MultipeerConnectivity UnityARKitMultipeerConnectivity • Swift ͷαϯϓϧΛ Unity Ͱ࠶࣮૷(͢ΔͨΊͷϓϥάΠϯ) • noir-neo/UnityARKitMultipeerConnectivity3 3 https:/ /github.com/noir-neo/UnityARKitMultipeerConnectivity 17
18. UnityARKitMultipeerConnectivity Features • ARWorldMap ͷૹड৴ • ARUserAnchor ͷૹड৴ • peer ͷ઀ଓɾ੾அΠϕϯτ 18
20. public class ARWorldMapReceiver : MonoBehaviour { [SerializeField] UnityARCameraManager arCameraManager; void Start() { UnityMCSessionNativeInterface.GetMcSessionNativeInterface().WorldMapReceivedEvent += Relocalize; } void Relocalize(ARWorldMap worldMap) { UnityARSessionNativeInterface.ARSessionShouldAttemptRelocalization = true; var config = arCameraManager.sessionConfiguration; config.worldMap = worldMap; UnityARSessionRunOption runOption = UnityARSessionRunOption.ARSessionRunOptionRemoveExistingAnchors | UnityARSessionRunOption.ARSessionRunOptionResetTracking; UnityARSessionNativeInterface.GetARSessionNativeInterface() .RunWithConfigAndOptions(config, runOption); } } 20
21. public static class ARWorldMapSender { public static void SendARWorld() { var mcSession = UnityMCSessionNativeInterface.GetMcSessionNativeInterface(); var arSession = UnityARSessionNativeInterface.GetARSessionNativeInterface(); arSession.GetCurrentWorldMapAsync(worldMap => { mcSession.SendToAllPeers(worldMap.nativePtr); }); } } 21
22. UnityARKitMultipeerConnectivity Issues4 • ARUserAnchor ͷ࡟আɺߋ৽ • ARWorldMap ΛϢʔβʹૢ࡞͍͍ͤͣ͞ײ͡ʹڞ༗͢Δ • ୺຤Ґஔͷಉظ (?) 4 https:/ /github.com/noir-neo/UnityARKitMultipeerConnectivity/issues 22
23. 3. MultipeerConnectivity Pros • ωοτϫʔΫʹܨ͕͍ͬͯΔඞཁ͕ͳ͍ • ۙ͘ʹ͍Ε͹উखʹܨ͕Δ 23
24. 3. MultipeerConnectivity Cons • ׬શʹ iOS ʹґଘ͢Δ • ϦΞϧλΠϜ௨৴ήʔϜΛ࡞Δʹ͸ػೳ͕଍Γͳ͍ • ΋͏গ͠͸ͦͷ͋ͨΓͷػೳ΋૿΍͍ͨ͠ 24
25. 4. Cloud Anchors • ARCore SDK for Unity ͔Β iOS Ͱ΋ར༻Ͱ͖Δ5 • Google ͷαʔόʔ(Firebase)ʹϗετ͢Δ 5 https:/ /developers.google.com/ar/develop/unity/cloud-anchors/quickstart-unity-ios 25
26. 4. Cloud Anchors Pros • Android ͱ΋ڞ༗Ͱ͖Δ(Β͍͠)(͍͢͝) (·ͩ৮Εͯͳ͍ͷͰͪΌΜͱΘ͔ͬͯ·ͤΜ ) 26
27. ·ͱΊ • ARKit ʹ͸ AR World Λอଘɾ࠶ݱ͢Δػೳ͕͋Δ͚ͩ • AR World Λ Unity Ͱૹड৴͢Δํ๏͸͍͔ͭ͋͘Δ • Ұ௕Ұ୹ͳͷͰཁ݅ʹ߹Θͤͯ࢖͍෼͚·͠ΐ͏ • ۙڑ཭௨৴Λ࢖͏ϓϥάΠϯΛॻ͍͍ͯΔͷͰΑ͚Ε͹ࢼͯ͠ Έ͍ͯͩ͘͞ ✨ • noir-neo/UnityARKitMultipeerConnectivity 27
28. ωοτϫʔΫ ϥϯχϯάίετ ήʔϜͷ࡞Γ΍͢͞ ϚονϝΠΫ UNet ಉ͡ LAN ʹ͍Δඞ ཁ ! """ ࣗಈԽՄೳ ϦΞϧλΠϜ௨৴ ήʔϜαʔό Πϯλʔωοτ઀ ଓ # """ ࣮૷͔Ϣʔβ͕େม ʹͳΔ Multipeer Connectivity Wi-Fi ͔ Bluetooth ༗ޮ ! - ࣗಈ Cloud Anchors Πϯλʔωοτ઀ ଓ ! (?) - ࣗಈ (?) 28