Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
ARKit in visionOS
Search
shu223
December 19, 2023
Programming
0
6
ARKit in visionOS
「Mobile勉強会 Wantedly × チームラボ × Sansan #12」での発表資料です。
https://sansan.connpass.com/event/303229/
shu223
December 19, 2023
Tweet
Share
More Decks by shu223
See All by shu223
GPT-4o with iOS
shu223
0
28
WhisperKit がだいぶ良いので紹介する
shu223
0
90
Animating Shapes with Simple Equations
shu223
0
16
ローカルLLM on iOS の現状まとめ
shu223
0
49
Core ML版Stable DiffusionをiOSで快適に動かす
shu223
0
49
iOSではじめるフォトグラメトリ #iOSDC
shu223
0
13
MLOps for Core ML #iOSDC 2022
shu223
0
10
slidify-sample
shu223
1
1.2k
機械学習のブルーオーシャン Core ML by 堤 修一 #iOSDC Japan 2020
shu223
12
5.5k
Other Decks in Programming
See All in Programming
生成 AI を活用した toitta 切片分類機能の裏側 / Inside toitta's AI-Based Factoid Clustering
pokutuna
0
570
破壊せよ!データ破壊駆動で考えるドメインモデリング / data-destroy-driven
minodriven
16
4.1k
Snowflake x dbtで作るセキュアでアジャイルなデータ基盤
tsoshiro
2
430
WEBエンジニア向けAI活用入門
sutetotanuki
0
300
Kubernetes for Data Engineers: Building Scalable, Reliable Data Pipelines
sucitw
1
200
カラム追加で増えるActiveRecordのメモリサイズ イメージできますか?
asayamakk
4
1.5k
VR HMDとしてのVision Pro+ゲーム開発について
yasei_no_otoko
0
100
飲食業界向けマルチプロダクトを実現させる開発体制とリアルな現状
hiroya0601
1
390
Amazon Neptuneで始めてみるグラフDB-OpenSearchによるグラフの全文検索-
satoshi256kbyte
4
330
アジャイルを支えるテストアーキテクチャ設計/Test Architecting for Agile
goyoki
7
2.8k
デプロイを任されたので、教わった通りにデプロイしたら障害になった件 ~俺のやらかしを越えてゆけ~
techouse
52
32k
Universal Linksの実装方法と陥りがちな罠
kaitokudou
1
220
Featured
See All Featured
The Art of Programming - Codeland 2020
erikaheidi
51
13k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
28
9.1k
Documentation Writing (for coders)
carmenintech
65
4.4k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
107
49k
YesSQL, Process and Tooling at Scale
rocio
167
14k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
27
790
It's Worth the Effort
3n
183
27k
How to train your dragon (web standard)
notwaldorf
88
5.7k
Building a Scalable Design System with Sketch
lauravandoore
459
33k
Product Roadmaps are Hard
iamctodd
PRO
48
10k
A Tale of Four Properties
chriscoyier
156
23k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
48k
Transcript
ARKit in visionOS ΞϨVision ProͰ࣮ݱͰ͖Δͷ͔ అ मҰʢ@shu223ʣ
ࣗݾհ • అ मҰ • @shu223 (GitHub, Qiita, Zenn, note,
, YouTube, Podcast, etc...) • ॻ੶ʢۀग़൛4ɺݸਓग़൛ଟ @BOOTHʣ:
ARKit in visionOSͷαϯϓϧू Appleެࣜͷ ARKit in visionOSνϡʔτϦΞϧ Λ ࣮ࡍʹಈ͘Ξ ϓϦͱ࣮ͯ͠
ͨ͠αϯϓϧɿ github.com/shu223/visionOS-Sampler
ຊͷΞδΣϯμ • visionOSʹ͓͚ΔARKit͕iOSͰͲ͏มΘ͔ͬͨ • visionOSͰʮΞϨʯ࣮ݱͰ͖Δͷ͔
visionOSΞϓϦͷ࡞Γํ • Swift, SwiftUIΛͬͯॻ͚Δ • ಉ͡ίʔυϕʔεͰiOS / visionOS྆ϦϦʔεՄೳ • iOSͷ΄ͱΜͲͷϑϨʔϜϫʔΫvisionOSͰ͑Δ
• ྫɿ Core ML, Core Bluetooth, Vision, etc...
visionOSͰ͑ͳ͍ϑϨʔϜϫʔΫ 1 ActivityKit AdSupport AppClip AutomatedDeviceEnrollment BusinessChat CarKey CarPlay Cinematic
ClockKit CoreLocationUI CoreMediaIO CoreNFC CoreTelephony DeviceActivity DockKit ExposureNotification FamilyControls FinanceKit FinanceKitUI ManagedSettings ManagedSettingsUI Messages MLCompute NearbyInteraction OpenAL ProximityReader RoomPlan SafetyKit ScreenTime SensorKit ServiceManagement Social Twitter WidgetKit WorkoutKit 1 Bringing your existing apps to visionOS | Apple Developer Documentation
ʢવʣARKit͑Δ
iOSͰARKitͬͯͨ͠༨༟ʂ
✋
iOSͰͷARKitͷฏ໘ݕग़ͷίʔυ // ηογϣϯͷίϯϑΟΪϡϨʔγϣϯΛੜ let configuration = ARWorldTrackingConfiguration() configuration.planeDetection = [.horizontal,
.vertical] // ηογϣϯ։࢝ sceneView.session.run(configuration)
iOSͰͷARKitͷฏ໘ݕग़ͷίʔυ // ΞϯΧʔ͕Ճ͞ΕΔͱσϦήʔτϝιου͕ݺΕΔ func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode,
for anchor: ARAnchor) { // ίϯϑΟΪϡϨʔγϣϯʹԠͨ͡ΞϯΧʔ͕ಘΒΕΔ guard let planeAnchor = anchor as? ARPlaneAnchor else { ... } ... }
iOSͰͷARKitͷฏ໘ݕग़ͷίʔυ // ηογϣϯͷίϯϑΟΪϡϨʔγϣϯΛੜ let configuration = ARWorldTrackingConfiguration() configuration.planeDetection = [.horizontal,
.vertical] ⚠ ARWorldTrackingConfiguration ɾɾɾvisionOSͰ ͑ͳ͍ʢARConfiguration ࣗମ͕͑ͳ͍ʣ
iOSͰͷARKitͷฏ໘ݕग़ͷίʔυ // ηογϣϯ։࢝ sceneView.session.run(configuration) ⚠ ARSCNView ɾɾɾvisionOSͰ͑ͳ͍ ⚠ ARSession ɾɾɾಉ্
iOSͰͷARKitͷฏ໘ݕग़ͷίʔυ // ΞϯΧʔ͕Ճ͞ΕΔͱσϦήʔτϝιου͕ݺΕΔ func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode,
for anchor: ARAnchor) { // ίϯϑΟΪϡϨʔγϣϯʹԠͨ͡ΞϯΧʔ͕ಘΒΕΔ guard let planeAnchor = anchor as? ARPlaneAnchor else { ... } ... } ⚠ ARSCNViewDelegate ɾɾɾvisionOSͰ͑ͳ͍ ⚠ ARAnchor, ARPlaneAnchor ɾɾɾಉ্
⚠ ARKit in iOSͷAPI΄΅͑ͳ͍
visionOSͰͷฏ໘ݕग़ͷίʔυ // ηογϣϯͱDataProviderͷॳظԽ let session = ARKitSession() let provider =
PlaneDetectionProvider( alignments: [.horizontal, .vertical] ) // ηογϣϯ։࢝ try await session.run([provider])
visionOSͰͷฏ໘ݕग़ͷίʔυ // Data Provider͔ΒΞϯΧʔ͕ಘΒΕΔ for await update in provider.anchorUpdates {
let planeAnchor = update.anchor ... }
ͯ͢visionOSઐ༻ͷ৽API
Ͳ͏มΘͬͨͷ͔ʁ
// iOS: ARSession session.run(configuration) // visionOS: ARKitSession try await session.run([provider])
// iOS: ARSession session.run(configuration) // visionOS: ARKitSession try await session.run([provider])
ͲͪΒrunstopͱ͍ͬͨϝιουΛ࣋ͭ
// iOS: ARSession session.run(configuration) // ARConfigurationΛ͢ // visionOS: ARKitSession try
await session.run([provider]) // DataProviderΛ͢
• ARKitSession ʢvisionOSʣ ARSession ʢiOSʣͲͪ ΒηογϣϯΛཧ • ͍ํݶΓͳ͘ࣅ͍ͯΔ • Anchor
ʢvisionOSʣͱ ARAnchor ʢiOSʣ͍͍ͩͨࣅͨ Α͏ͳͷ 2 2 visionOSʹ͓͚ΔARKitiOS൛ͱͲ͏มΘ͔ͬͨ
Ͳ͏ͯ͜͠͏ͳͬͨ... • ಉ͡ػೳͳͷʹ... • ࣮͍͍ͩͨࣅͨΑ͏ͳΜͳͷʹ... ಉ͡APIʹ͠ͱ͍ͨํ͕iOSͱίʔυΛڞ௨ԽͰ͖ͯྑ͔ͬͨͷ Ͱ...ʁ
ARKit in visionOSͷʮؾ࣋ͪʯΛཧղ͢Δ
ϙΠϯτͦͷ1: Vision Proʹ͓͍ͯARKitͷػೳৗ࣌ON
Debug Visualizationػೳ • Xcode 15.1 beta ͷػೳ • visionOSͷॾʑΛՄࢹԽͯ͘͠ΕΔ
Debug Visualization Λ༗ޮʹͯ͠৭ʑݟͯΈΔͱ...
ARKitͷػೳৗ࣌ON • ΞϓϦ͕ARKitͷػೳΛ͍ͬͯΑ͏ ͱ͍·͍ͱɺWorld TrackingScene Understanding͕ৗ࣌ߦΘΕ͍ͯΔ • ϋϯυτϥοΩϯάಉ༷
ϙΠϯτͦͷ2: ϓϥΠόγʔ
ARKitͱϓϥΠόγʔ ARKitɺϢʔβʔڐՄͳ͠ʹΞϓϦʹ σʔλΛఏڙ͠ͳ͍
ϓϥΠόγʔͱData Provider Ϣʔβʔ͕ڋ൱͍ͯ͠ΔσʔλΛఏڙ͢ ΔData ProviderͰηογϣϯ࣮ߦ͠Α ͏ͱ͢Δͱɺηογϣϯ͕ࣦഊ͢Δ
• Vision Pro͍͏ͳΕARKitσόΠε • ΞϓϦ։ൃऀ͕ARKitػೳΛ੍ޚ͢ΔͷͰͳ͘ɺৗ࣌ಈ ͍͍ͯΔ • ϓϥΠόγʔͷྀ • σʔλΛෆ༻ҙʹ։ൃऀʹ͞ͳ͍
// iOS: // ConfigurationͰʮ͍͍ͨARKitػೳΛߏʯ session.run(configuration) // visionOS: // Data ProviderΛ௨ͯ͡ʮ͍͍ͨARKitσʔλʹΞΫηεʯ
try await session.run([provider])
͜͜·Ͱͷ·ͱΊ • ARKit in visionOSiOSͱ·ͬͨ͘ҧ͏API܈Λ͏ • ͦͷઃܭࢥɺVision Pro͕ARKitσόΠεͰ͋Δͱߟ͑Δ ͱᡰʹམͪΔ
ʙΞϨVision ProͰ࣮ݱͰ͖Δͷ͔ʙ
ʢͨͿΜଟ͘ͷ։ൃऀ͕ʣΓ͍ͨ͜ͱ • ϓϩϑΟʔϧσʔλΛਓͷ಄্ʹද ࣔͨ͠Γ • ϞϊͷใΠϯετϥΫγϣϯΛ දࣔͨ͠Γ
ARKit in iOS • People Occulusion • Pose Detection •
Face Tracking • Object Detection • ϑϨʔϜը૾ʹରͯ͠Core MLར༻ ͍͘ΒͰखஈ͕͋Δ
ARKit in visionOS • People Occulusion • Pose Detection •
Face Tracking • Object Detection • ϑϨʔϜը૾ʹରͯ͠Core MLར༻ ͲΕෆՄ 4 4 นఱҪͷʮ໘ʯΛݕग़͠छผΛಛఆ͢Δ͜ͱՄೳ
ۤͷࡦ • DeviceAnchorΛར༻ • BLEUWBͷҐஔਪఆٕज़Λར༻ • ϚΠΫΞϨΠʹΑΔԻݯํਪఆ
DeviceAnchorΛར༻ DeviceAnchor The position and orientation of Apple Vision Pro.
• ૬खVision ProΛண͚͍ͯΕҐஔɾ͖͕Θ͔Δʁ • ͨͿΜࣗͷ͔͠औΕͳ͍ • ԾʹଞਓͷऔΕΔͱͯ͠ɺʮVision ProΛண͚͍ͯΔਓ ͚ͩʯଟ͘ͷϢʔεέʔεʹ߹Θͳ͍
BLEUWBͷҐஔਪఆٕज़Λར༻ • Core Bluetoothར༻Մೳ • ΦϒδΣΫτΛݻఆ͢Δ΄Ͳͷਫ਼ظͰ͖ͳ͍ • iBeaconͷΑ͏ʹʮ͍ۙͨΒʙ͢ΔʯఔͷϢʔεέʔ ε͕ଥ •
Nearby Interactionར༻ෆՄ
ϚΠΫΞϨΠʹΑΔԻݯํਪఆ • Vision ProͷϚΠΫϞϊϥϧͳͷͰແཧͦ͏
݁ɿ visionOS 2.0 ͪ 5 5 c.f. ࠓͦ͜watchOS
·ͱΊ • ARKit in visionOSͷ৽APIΛཧղ͢ΔϙΠϯτʮVision Pro ARKitσόΠεʯʮϓϥΠόγʔͷྀʯ • visionOS 1.0Ͱਓମʹରͯ͠ϓϩϑΟʔϧͷԿΒ͔
ͷΦϒδΣΫτΛઃஔ͢ΔϢʔεέʔεݫͦ͠͏