Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
ARKit in visionOS
Search
shu223
December 19, 2023
Programming
0
7
ARKit in visionOS
「Mobile勉強会 Wantedly × チームラボ × Sansan #12」での発表資料です。
https://sansan.connpass.com/event/303229/
shu223
December 19, 2023
Tweet
Share
More Decks by shu223
See All by shu223
GPT-4o with iOS
shu223
0
29
WhisperKit がだいぶ良いので紹介する
shu223
0
95
Animating Shapes with Simple Equations
shu223
0
17
ローカルLLM on iOS の現状まとめ
shu223
0
52
Core ML版Stable DiffusionをiOSで快適に動かす
shu223
0
52
iOSではじめるフォトグラメトリ #iOSDC
shu223
0
18
MLOps for Core ML #iOSDC 2022
shu223
0
11
slidify-sample
shu223
1
1.2k
機械学習のブルーオーシャン Core ML by 堤 修一 #iOSDC Japan 2020
shu223
12
5.6k
Other Decks in Programming
See All in Programming
色々なIaCツールを実際に触って比較してみる
iriikeita
0
330
ECS Service Connectのこれまでのアップデートと今後のRoadmapを見てみる
tkikuc
2
250
Webの技術スタックで マルチプラットフォームアプリ開発を可能にするElixirDesktopの紹介
thehaigo
2
1k
광고 소재 심사 과정에 AI를 도입하여 광고 서비스 생산성 향상시키기
kakao
PRO
0
170
WebフロントエンドにおけるGraphQL(あるいはバックエンドのAPI)との向き合い方 / #241106_plk_frontend
izumin5210
4
1.4k
3 Effective Rules for Using Signals in Angular
manfredsteyer
PRO
0
100
リアーキテクチャxDDD 1年間の取り組みと進化
hsawaji
1
220
NSOutlineView何もわからん:( 前編 / I Don't Understand About NSOutlineView :( Pt. 1
usagimaru
0
340
役立つログに取り組もう
irof
28
9.6k
macOS でできる リアルタイム動画像処理
biacco42
9
2.4k
ローコードSaaSのUXを向上させるためのTypeScript
taro28
1
630
Ethereum_.pdf
nekomatu
0
470
Featured
See All Featured
How STYLIGHT went responsive
nonsquared
95
5.2k
Designing for Performance
lara
604
68k
Build your cross-platform service in a week with App Engine
jlugia
229
18k
Reflections from 52 weeks, 52 projects
jeffersonlam
346
20k
Build The Right Thing And Hit Your Dates
maggiecrowley
33
2.4k
Intergalactic Javascript Robots from Outer Space
tanoku
269
27k
Building Adaptive Systems
keathley
38
2.3k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
47
5k
A designer walks into a library…
pauljervisheath
204
24k
Bootstrapping a Software Product
garrettdimon
PRO
305
110k
Stop Working from a Prison Cell
hatefulcrawdad
267
20k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
44
6.8k
Transcript
ARKit in visionOS ΞϨVision ProͰ࣮ݱͰ͖Δͷ͔ అ मҰʢ@shu223ʣ
ࣗݾհ • అ मҰ • @shu223 (GitHub, Qiita, Zenn, note,
, YouTube, Podcast, etc...) • ॻ੶ʢۀग़൛4ɺݸਓग़൛ଟ @BOOTHʣ:
ARKit in visionOSͷαϯϓϧू Appleެࣜͷ ARKit in visionOSνϡʔτϦΞϧ Λ ࣮ࡍʹಈ͘Ξ ϓϦͱ࣮ͯ͠
ͨ͠αϯϓϧɿ github.com/shu223/visionOS-Sampler
ຊͷΞδΣϯμ • visionOSʹ͓͚ΔARKit͕iOSͰͲ͏มΘ͔ͬͨ • visionOSͰʮΞϨʯ࣮ݱͰ͖Δͷ͔
visionOSΞϓϦͷ࡞Γํ • Swift, SwiftUIΛͬͯॻ͚Δ • ಉ͡ίʔυϕʔεͰiOS / visionOS྆ϦϦʔεՄೳ • iOSͷ΄ͱΜͲͷϑϨʔϜϫʔΫvisionOSͰ͑Δ
• ྫɿ Core ML, Core Bluetooth, Vision, etc...
visionOSͰ͑ͳ͍ϑϨʔϜϫʔΫ 1 ActivityKit AdSupport AppClip AutomatedDeviceEnrollment BusinessChat CarKey CarPlay Cinematic
ClockKit CoreLocationUI CoreMediaIO CoreNFC CoreTelephony DeviceActivity DockKit ExposureNotification FamilyControls FinanceKit FinanceKitUI ManagedSettings ManagedSettingsUI Messages MLCompute NearbyInteraction OpenAL ProximityReader RoomPlan SafetyKit ScreenTime SensorKit ServiceManagement Social Twitter WidgetKit WorkoutKit 1 Bringing your existing apps to visionOS | Apple Developer Documentation
ʢવʣARKit͑Δ
iOSͰARKitͬͯͨ͠༨༟ʂ
✋
iOSͰͷARKitͷฏ໘ݕग़ͷίʔυ // ηογϣϯͷίϯϑΟΪϡϨʔγϣϯΛੜ let configuration = ARWorldTrackingConfiguration() configuration.planeDetection = [.horizontal,
.vertical] // ηογϣϯ։࢝ sceneView.session.run(configuration)
iOSͰͷARKitͷฏ໘ݕग़ͷίʔυ // ΞϯΧʔ͕Ճ͞ΕΔͱσϦήʔτϝιου͕ݺΕΔ func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode,
for anchor: ARAnchor) { // ίϯϑΟΪϡϨʔγϣϯʹԠͨ͡ΞϯΧʔ͕ಘΒΕΔ guard let planeAnchor = anchor as? ARPlaneAnchor else { ... } ... }
iOSͰͷARKitͷฏ໘ݕग़ͷίʔυ // ηογϣϯͷίϯϑΟΪϡϨʔγϣϯΛੜ let configuration = ARWorldTrackingConfiguration() configuration.planeDetection = [.horizontal,
.vertical] ⚠ ARWorldTrackingConfiguration ɾɾɾvisionOSͰ ͑ͳ͍ʢARConfiguration ࣗମ͕͑ͳ͍ʣ
iOSͰͷARKitͷฏ໘ݕग़ͷίʔυ // ηογϣϯ։࢝ sceneView.session.run(configuration) ⚠ ARSCNView ɾɾɾvisionOSͰ͑ͳ͍ ⚠ ARSession ɾɾɾಉ্
iOSͰͷARKitͷฏ໘ݕग़ͷίʔυ // ΞϯΧʔ͕Ճ͞ΕΔͱσϦήʔτϝιου͕ݺΕΔ func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode,
for anchor: ARAnchor) { // ίϯϑΟΪϡϨʔγϣϯʹԠͨ͡ΞϯΧʔ͕ಘΒΕΔ guard let planeAnchor = anchor as? ARPlaneAnchor else { ... } ... } ⚠ ARSCNViewDelegate ɾɾɾvisionOSͰ͑ͳ͍ ⚠ ARAnchor, ARPlaneAnchor ɾɾɾಉ্
⚠ ARKit in iOSͷAPI΄΅͑ͳ͍
visionOSͰͷฏ໘ݕग़ͷίʔυ // ηογϣϯͱDataProviderͷॳظԽ let session = ARKitSession() let provider =
PlaneDetectionProvider( alignments: [.horizontal, .vertical] ) // ηογϣϯ։࢝ try await session.run([provider])
visionOSͰͷฏ໘ݕग़ͷίʔυ // Data Provider͔ΒΞϯΧʔ͕ಘΒΕΔ for await update in provider.anchorUpdates {
let planeAnchor = update.anchor ... }
ͯ͢visionOSઐ༻ͷ৽API
Ͳ͏มΘͬͨͷ͔ʁ
// iOS: ARSession session.run(configuration) // visionOS: ARKitSession try await session.run([provider])
// iOS: ARSession session.run(configuration) // visionOS: ARKitSession try await session.run([provider])
ͲͪΒrunstopͱ͍ͬͨϝιουΛ࣋ͭ
// iOS: ARSession session.run(configuration) // ARConfigurationΛ͢ // visionOS: ARKitSession try
await session.run([provider]) // DataProviderΛ͢
• ARKitSession ʢvisionOSʣ ARSession ʢiOSʣͲͪ ΒηογϣϯΛཧ • ͍ํݶΓͳ͘ࣅ͍ͯΔ • Anchor
ʢvisionOSʣͱ ARAnchor ʢiOSʣ͍͍ͩͨࣅͨ Α͏ͳͷ 2 2 visionOSʹ͓͚ΔARKitiOS൛ͱͲ͏มΘ͔ͬͨ
Ͳ͏ͯ͜͠͏ͳͬͨ... • ಉ͡ػೳͳͷʹ... • ࣮͍͍ͩͨࣅͨΑ͏ͳΜͳͷʹ... ಉ͡APIʹ͠ͱ͍ͨํ͕iOSͱίʔυΛڞ௨ԽͰ͖ͯྑ͔ͬͨͷ Ͱ...ʁ
ARKit in visionOSͷʮؾ࣋ͪʯΛཧղ͢Δ
ϙΠϯτͦͷ1: Vision Proʹ͓͍ͯARKitͷػೳৗ࣌ON
Debug Visualizationػೳ • Xcode 15.1 beta ͷػೳ • visionOSͷॾʑΛՄࢹԽͯ͘͠ΕΔ
Debug Visualization Λ༗ޮʹͯ͠৭ʑݟͯΈΔͱ...
ARKitͷػೳৗ࣌ON • ΞϓϦ͕ARKitͷػೳΛ͍ͬͯΑ͏ ͱ͍·͍ͱɺWorld TrackingScene Understanding͕ৗ࣌ߦΘΕ͍ͯΔ • ϋϯυτϥοΩϯάಉ༷
ϙΠϯτͦͷ2: ϓϥΠόγʔ
ARKitͱϓϥΠόγʔ ARKitɺϢʔβʔڐՄͳ͠ʹΞϓϦʹ σʔλΛఏڙ͠ͳ͍
ϓϥΠόγʔͱData Provider Ϣʔβʔ͕ڋ൱͍ͯ͠ΔσʔλΛఏڙ͢ ΔData ProviderͰηογϣϯ࣮ߦ͠Α ͏ͱ͢Δͱɺηογϣϯ͕ࣦഊ͢Δ
• Vision Pro͍͏ͳΕARKitσόΠε • ΞϓϦ։ൃऀ͕ARKitػೳΛ੍ޚ͢ΔͷͰͳ͘ɺৗ࣌ಈ ͍͍ͯΔ • ϓϥΠόγʔͷྀ • σʔλΛෆ༻ҙʹ։ൃऀʹ͞ͳ͍
// iOS: // ConfigurationͰʮ͍͍ͨARKitػೳΛߏʯ session.run(configuration) // visionOS: // Data ProviderΛ௨ͯ͡ʮ͍͍ͨARKitσʔλʹΞΫηεʯ
try await session.run([provider])
͜͜·Ͱͷ·ͱΊ • ARKit in visionOSiOSͱ·ͬͨ͘ҧ͏API܈Λ͏ • ͦͷઃܭࢥɺVision Pro͕ARKitσόΠεͰ͋Δͱߟ͑Δ ͱᡰʹམͪΔ
ʙΞϨVision ProͰ࣮ݱͰ͖Δͷ͔ʙ
ʢͨͿΜଟ͘ͷ։ൃऀ͕ʣΓ͍ͨ͜ͱ • ϓϩϑΟʔϧσʔλΛਓͷ಄্ʹද ࣔͨ͠Γ • ϞϊͷใΠϯετϥΫγϣϯΛ දࣔͨ͠Γ
ARKit in iOS • People Occulusion • Pose Detection •
Face Tracking • Object Detection • ϑϨʔϜը૾ʹରͯ͠Core MLར༻ ͍͘ΒͰखஈ͕͋Δ
ARKit in visionOS • People Occulusion • Pose Detection •
Face Tracking • Object Detection • ϑϨʔϜը૾ʹରͯ͠Core MLར༻ ͲΕෆՄ 4 4 นఱҪͷʮ໘ʯΛݕग़͠छผΛಛఆ͢Δ͜ͱՄೳ
ۤͷࡦ • DeviceAnchorΛར༻ • BLEUWBͷҐஔਪఆٕज़Λར༻ • ϚΠΫΞϨΠʹΑΔԻݯํਪఆ
DeviceAnchorΛར༻ DeviceAnchor The position and orientation of Apple Vision Pro.
• ૬खVision ProΛண͚͍ͯΕҐஔɾ͖͕Θ͔Δʁ • ͨͿΜࣗͷ͔͠औΕͳ͍ • ԾʹଞਓͷऔΕΔͱͯ͠ɺʮVision ProΛண͚͍ͯΔਓ ͚ͩʯଟ͘ͷϢʔεέʔεʹ߹Θͳ͍
BLEUWBͷҐஔਪఆٕज़Λར༻ • Core Bluetoothར༻Մೳ • ΦϒδΣΫτΛݻఆ͢Δ΄Ͳͷਫ਼ظͰ͖ͳ͍ • iBeaconͷΑ͏ʹʮ͍ۙͨΒʙ͢ΔʯఔͷϢʔεέʔ ε͕ଥ •
Nearby Interactionར༻ෆՄ
ϚΠΫΞϨΠʹΑΔԻݯํਪఆ • Vision ProͷϚΠΫϞϊϥϧͳͷͰແཧͦ͏
݁ɿ visionOS 2.0 ͪ 5 5 c.f. ࠓͦ͜watchOS
·ͱΊ • ARKit in visionOSͷ৽APIΛཧղ͢ΔϙΠϯτʮVision Pro ARKitσόΠεʯʮϓϥΠόγʔͷྀʯ • visionOS 1.0Ͱਓମʹରͯ͠ϓϩϑΟʔϧͷԿΒ͔
ͷΦϒδΣΫτΛઃஔ͢ΔϢʔεέʔεݫͦ͠͏