Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
ARKit in visionOS
Search
shu223
December 19, 2023
Programming
0
5
ARKit in visionOS
「Mobile勉強会 Wantedly × チームラボ × Sansan #12」での発表資料です。
https://sansan.connpass.com/event/303229/
shu223
December 19, 2023
Tweet
Share
More Decks by shu223
See All by shu223
GPT-4o with iOS
shu223
0
24
WhisperKit がだいぶ良いので紹介する
shu223
0
73
Animating Shapes with Simple Equations
shu223
0
13
ローカルLLM on iOS の現状まとめ
shu223
0
35
Core ML版Stable DiffusionをiOSで快適に動かす
shu223
0
43
iOSではじめるフォトグラメトリ #iOSDC
shu223
0
13
MLOps for Core ML #iOSDC 2022
shu223
0
10
slidify-sample
shu223
1
1.2k
機械学習のブルーオーシャン Core ML by 堤 修一 #iOSDC Japan 2020
shu223
12
5.5k
Other Decks in Programming
See All in Programming
Scala アプリケーションのビルドを改善してデプロイ時間を 1/4 にした話 | How I improved the build of my Scala application and reduced deployment time by 4x
nomadblacky
1
180
マイグレーションコード自作して File-Based Routing に自動移行!! ~250 ページの歴史的経緯を添えて~
cut0
1
260
令和トラベルにおけるLLM活用事例:社内ツール開発から得た学びと実践
ippo012
0
150
rails_girls_is_my_gate_to_join_the_ruby_commuinty
maimux2x
0
200
Android開発以外のAndroid開発経験の活かしどころ
konifar
2
970
オートマトン学習しろ / Do automata learning
makenowjust
3
130
Developer Joy == Developer Productivity (really!)
hollycummins
1
180
LangGraphでのHuman-in-the-Loopの実装
os1ma
3
1.1k
Method Swizzlingを行うライブラリにおけるマルチモジュール設計
yoshikma
0
120
RAGの回答精度評価用のQAデータセットを生成AIに作らせた話
kurahara
0
250
GoのIteratorに詳しくなってしまう
inatonix
1
200
私のEbitengineの第一歩
qt_luigi
0
450
Featured
See All Featured
Music & Morning Musume
bryan
46
6k
Fontdeck: Realign not Redesign
paulrobertlloyd
80
5.1k
Facilitating Awesome Meetings
lara
49
5.9k
Web development in the modern age
philhawksworth
204
10k
Happy Clients
brianwarren
96
6.6k
Debugging Ruby Performance
tmm1
72
12k
Building a Scalable Design System with Sketch
lauravandoore
458
32k
Ruby is Unlike a Banana
tanoku
96
11k
Why Our Code Smells
bkeepers
PRO
334
56k
The Power of CSS Pseudo Elements
geoffreycrofte
71
5.3k
Designing for Performance
lara
604
68k
Embracing the Ebb and Flow
colly
83
4.4k
Transcript
ARKit in visionOS ΞϨVision ProͰ࣮ݱͰ͖Δͷ͔ అ मҰʢ@shu223ʣ
ࣗݾհ • అ मҰ • @shu223 (GitHub, Qiita, Zenn, note,
, YouTube, Podcast, etc...) • ॻ੶ʢۀग़൛4ɺݸਓग़൛ଟ @BOOTHʣ:
ARKit in visionOSͷαϯϓϧू Appleެࣜͷ ARKit in visionOSνϡʔτϦΞϧ Λ ࣮ࡍʹಈ͘Ξ ϓϦͱ࣮ͯ͠
ͨ͠αϯϓϧɿ github.com/shu223/visionOS-Sampler
ຊͷΞδΣϯμ • visionOSʹ͓͚ΔARKit͕iOSͰͲ͏มΘ͔ͬͨ • visionOSͰʮΞϨʯ࣮ݱͰ͖Δͷ͔
visionOSΞϓϦͷ࡞Γํ • Swift, SwiftUIΛͬͯॻ͚Δ • ಉ͡ίʔυϕʔεͰiOS / visionOS྆ϦϦʔεՄೳ • iOSͷ΄ͱΜͲͷϑϨʔϜϫʔΫvisionOSͰ͑Δ
• ྫɿ Core ML, Core Bluetooth, Vision, etc...
visionOSͰ͑ͳ͍ϑϨʔϜϫʔΫ 1 ActivityKit AdSupport AppClip AutomatedDeviceEnrollment BusinessChat CarKey CarPlay Cinematic
ClockKit CoreLocationUI CoreMediaIO CoreNFC CoreTelephony DeviceActivity DockKit ExposureNotification FamilyControls FinanceKit FinanceKitUI ManagedSettings ManagedSettingsUI Messages MLCompute NearbyInteraction OpenAL ProximityReader RoomPlan SafetyKit ScreenTime SensorKit ServiceManagement Social Twitter WidgetKit WorkoutKit 1 Bringing your existing apps to visionOS | Apple Developer Documentation
ʢવʣARKit͑Δ
iOSͰARKitͬͯͨ͠༨༟ʂ
✋
iOSͰͷARKitͷฏ໘ݕग़ͷίʔυ // ηογϣϯͷίϯϑΟΪϡϨʔγϣϯΛੜ let configuration = ARWorldTrackingConfiguration() configuration.planeDetection = [.horizontal,
.vertical] // ηογϣϯ։࢝ sceneView.session.run(configuration)
iOSͰͷARKitͷฏ໘ݕग़ͷίʔυ // ΞϯΧʔ͕Ճ͞ΕΔͱσϦήʔτϝιου͕ݺΕΔ func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode,
for anchor: ARAnchor) { // ίϯϑΟΪϡϨʔγϣϯʹԠͨ͡ΞϯΧʔ͕ಘΒΕΔ guard let planeAnchor = anchor as? ARPlaneAnchor else { ... } ... }
iOSͰͷARKitͷฏ໘ݕग़ͷίʔυ // ηογϣϯͷίϯϑΟΪϡϨʔγϣϯΛੜ let configuration = ARWorldTrackingConfiguration() configuration.planeDetection = [.horizontal,
.vertical] ⚠ ARWorldTrackingConfiguration ɾɾɾvisionOSͰ ͑ͳ͍ʢARConfiguration ࣗମ͕͑ͳ͍ʣ
iOSͰͷARKitͷฏ໘ݕग़ͷίʔυ // ηογϣϯ։࢝ sceneView.session.run(configuration) ⚠ ARSCNView ɾɾɾvisionOSͰ͑ͳ͍ ⚠ ARSession ɾɾɾಉ্
iOSͰͷARKitͷฏ໘ݕग़ͷίʔυ // ΞϯΧʔ͕Ճ͞ΕΔͱσϦήʔτϝιου͕ݺΕΔ func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode,
for anchor: ARAnchor) { // ίϯϑΟΪϡϨʔγϣϯʹԠͨ͡ΞϯΧʔ͕ಘΒΕΔ guard let planeAnchor = anchor as? ARPlaneAnchor else { ... } ... } ⚠ ARSCNViewDelegate ɾɾɾvisionOSͰ͑ͳ͍ ⚠ ARAnchor, ARPlaneAnchor ɾɾɾಉ্
⚠ ARKit in iOSͷAPI΄΅͑ͳ͍
visionOSͰͷฏ໘ݕग़ͷίʔυ // ηογϣϯͱDataProviderͷॳظԽ let session = ARKitSession() let provider =
PlaneDetectionProvider( alignments: [.horizontal, .vertical] ) // ηογϣϯ։࢝ try await session.run([provider])
visionOSͰͷฏ໘ݕग़ͷίʔυ // Data Provider͔ΒΞϯΧʔ͕ಘΒΕΔ for await update in provider.anchorUpdates {
let planeAnchor = update.anchor ... }
ͯ͢visionOSઐ༻ͷ৽API
Ͳ͏มΘͬͨͷ͔ʁ
// iOS: ARSession session.run(configuration) // visionOS: ARKitSession try await session.run([provider])
// iOS: ARSession session.run(configuration) // visionOS: ARKitSession try await session.run([provider])
ͲͪΒrunstopͱ͍ͬͨϝιουΛ࣋ͭ
// iOS: ARSession session.run(configuration) // ARConfigurationΛ͢ // visionOS: ARKitSession try
await session.run([provider]) // DataProviderΛ͢
• ARKitSession ʢvisionOSʣ ARSession ʢiOSʣͲͪ ΒηογϣϯΛཧ • ͍ํݶΓͳ͘ࣅ͍ͯΔ • Anchor
ʢvisionOSʣͱ ARAnchor ʢiOSʣ͍͍ͩͨࣅͨ Α͏ͳͷ 2 2 visionOSʹ͓͚ΔARKitiOS൛ͱͲ͏มΘ͔ͬͨ
Ͳ͏ͯ͜͠͏ͳͬͨ... • ಉ͡ػೳͳͷʹ... • ࣮͍͍ͩͨࣅͨΑ͏ͳΜͳͷʹ... ಉ͡APIʹ͠ͱ͍ͨํ͕iOSͱίʔυΛڞ௨ԽͰ͖ͯྑ͔ͬͨͷ Ͱ...ʁ
ARKit in visionOSͷʮؾ࣋ͪʯΛཧղ͢Δ
ϙΠϯτͦͷ1: Vision Proʹ͓͍ͯARKitͷػೳৗ࣌ON
Debug Visualizationػೳ • Xcode 15.1 beta ͷػೳ • visionOSͷॾʑΛՄࢹԽͯ͘͠ΕΔ
Debug Visualization Λ༗ޮʹͯ͠৭ʑݟͯΈΔͱ...
ARKitͷػೳৗ࣌ON • ΞϓϦ͕ARKitͷػೳΛ͍ͬͯΑ͏ ͱ͍·͍ͱɺWorld TrackingScene Understanding͕ৗ࣌ߦΘΕ͍ͯΔ • ϋϯυτϥοΩϯάಉ༷
ϙΠϯτͦͷ2: ϓϥΠόγʔ
ARKitͱϓϥΠόγʔ ARKitɺϢʔβʔڐՄͳ͠ʹΞϓϦʹ σʔλΛఏڙ͠ͳ͍
ϓϥΠόγʔͱData Provider Ϣʔβʔ͕ڋ൱͍ͯ͠ΔσʔλΛఏڙ͢ ΔData ProviderͰηογϣϯ࣮ߦ͠Α ͏ͱ͢Δͱɺηογϣϯ͕ࣦഊ͢Δ
• Vision Pro͍͏ͳΕARKitσόΠε • ΞϓϦ։ൃऀ͕ARKitػೳΛ੍ޚ͢ΔͷͰͳ͘ɺৗ࣌ಈ ͍͍ͯΔ • ϓϥΠόγʔͷྀ • σʔλΛෆ༻ҙʹ։ൃऀʹ͞ͳ͍
// iOS: // ConfigurationͰʮ͍͍ͨARKitػೳΛߏʯ session.run(configuration) // visionOS: // Data ProviderΛ௨ͯ͡ʮ͍͍ͨARKitσʔλʹΞΫηεʯ
try await session.run([provider])
͜͜·Ͱͷ·ͱΊ • ARKit in visionOSiOSͱ·ͬͨ͘ҧ͏API܈Λ͏ • ͦͷઃܭࢥɺVision Pro͕ARKitσόΠεͰ͋Δͱߟ͑Δ ͱᡰʹམͪΔ
ʙΞϨVision ProͰ࣮ݱͰ͖Δͷ͔ʙ
ʢͨͿΜଟ͘ͷ։ൃऀ͕ʣΓ͍ͨ͜ͱ • ϓϩϑΟʔϧσʔλΛਓͷ಄্ʹද ࣔͨ͠Γ • ϞϊͷใΠϯετϥΫγϣϯΛ දࣔͨ͠Γ
ARKit in iOS • People Occulusion • Pose Detection •
Face Tracking • Object Detection • ϑϨʔϜը૾ʹରͯ͠Core MLར༻ ͍͘ΒͰखஈ͕͋Δ
ARKit in visionOS • People Occulusion • Pose Detection •
Face Tracking • Object Detection • ϑϨʔϜը૾ʹରͯ͠Core MLར༻ ͲΕෆՄ 4 4 นఱҪͷʮ໘ʯΛݕग़͠छผΛಛఆ͢Δ͜ͱՄೳ
ۤͷࡦ • DeviceAnchorΛར༻ • BLEUWBͷҐஔਪఆٕज़Λར༻ • ϚΠΫΞϨΠʹΑΔԻݯํਪఆ
DeviceAnchorΛར༻ DeviceAnchor The position and orientation of Apple Vision Pro.
• ૬खVision ProΛண͚͍ͯΕҐஔɾ͖͕Θ͔Δʁ • ͨͿΜࣗͷ͔͠औΕͳ͍ • ԾʹଞਓͷऔΕΔͱͯ͠ɺʮVision ProΛண͚͍ͯΔਓ ͚ͩʯଟ͘ͷϢʔεέʔεʹ߹Θͳ͍
BLEUWBͷҐஔਪఆٕज़Λར༻ • Core Bluetoothར༻Մೳ • ΦϒδΣΫτΛݻఆ͢Δ΄Ͳͷਫ਼ظͰ͖ͳ͍ • iBeaconͷΑ͏ʹʮ͍ۙͨΒʙ͢ΔʯఔͷϢʔεέʔ ε͕ଥ •
Nearby Interactionར༻ෆՄ
ϚΠΫΞϨΠʹΑΔԻݯํਪఆ • Vision ProͷϚΠΫϞϊϥϧͳͷͰແཧͦ͏
݁ɿ visionOS 2.0 ͪ 5 5 c.f. ࠓͦ͜watchOS
·ͱΊ • ARKit in visionOSͷ৽APIΛཧղ͢ΔϙΠϯτʮVision Pro ARKitσόΠεʯʮϓϥΠόγʔͷྀʯ • visionOS 1.0Ͱਓମʹରͯ͠ϓϩϑΟʔϧͷԿΒ͔
ͷΦϒδΣΫτΛઃஔ͢ΔϢʔεέʔεݫͦ͠͏