ARKit 2.0

Transcript

1. ARKit 2.0 @TachibanaKaoru 2018/6/11 WWDC GoriGori Catch-Up
2. About Me @TachibanaKaoru Freelance iOS Engineer ϘϧμϦϯά޷͖ ࠓ೥ͷςʔϚɿཱྀΛ͠ͳ͕Β࢓ࣄΛ͢Δ όϦౡɺαϯτϦʔχౡɺόϯίΫ
3. ARKit
4. What is ARKit? iOS11~ Device with A9 chip~ SceneKit or SpriteKit or Metal Unity ARKit Plugin
5. ARKit capable devices Phone X iPhone 8, iPhone 8 Plus iPhone 7, iPhone 7 Plus iPhone 6s, iPhone 6s Plus iPhone SE iPad Pro (all models) iPad (5th generation)
6. ARKit 1.0 ۭؒݕ஌ɺฏ໘ʢਫฏ໘ʣݕ஌ɺإද৘ݕ஌ 1.5 ฏ໘ʢਨ௚໘ݕ஌ʣɺը૾ݕ஌ 2.0 ը૾ݕ஌Ϟʔυ௥ՃɺARঢ়ଶอଘରԠɺ Object SacnରԠɺࢹઢɾઉݕ஌
7. AR Configuration Mode Camera iOS Device AROrientationTracking Back Camera iOS 11.0+ ARKit Capable Devices ARWorldTracking Back Camera iOS 11.0+ ARKit Capable Devices ARFaceTracking Front Camera iOS 11.0+ Only iPhone X ARImageTracking Back Camera iOS 12.0+ ARKit Capable Devices ARObjectScanning Back Camera iOS 12.0+ ARKit Capable Devices
8. USDZ
9. USDZ USD : Universal Scene Description 3D Model Format from Pixar PDF in 3D world! From iOS12 ~
10. USDZ on iOS12 Available on Safari, memo, Mail and so on.
11. USDZ on iOS12 Check USDZ files on “AR Quick Look Gallery” https://developer.apple.com/arkit/gallery/
12. USDZ on your App Available with QLPreviewController, WKWebView, SFSafariViewController
13. USDZ with QuickLook import QuickLook class ViewController: UIViewController { override func viewDidLoad() { super.viewDidLoad() } // Call Here to show QuickLook Preview Controller!! @IBAction func showQuickLook(_ sender: Any) { let quickViewCon = QLPreviewController() quickViewCon.dataSource = self self.present(quickViewCon, animated: true, completion: nil) } }
14. USDZ with QuickLook extension ViewController: QLPreviewControllerDataSource { func numberOfPreviewItems(in controller: QLPreviewController) -> Int{ return 1 } func previewController(_ controller: QLPreviewController, previewItemAt index: Int) -> QLPreviewItem{ return MyARItem() } } class MyARItem: NSObject, QLPreviewItem{ var previewItemURL: URL? { return Bundle.main.url(forResource: "coffee", withExtension: "usdz") } }
15. How does ARKit works?
16. AR Configuration Mode Camera iOS Device AROrientationTracking Back Camera iOS 11.0+ ARKit Capable Devices ARWorldTracking Back Camera iOS 11.0+ ARKit Capable Devices ARFaceTracking Front Camera iOS 11.0+ Only iPhone X ARImageTracking Back Camera iOS 12.0+ ARKit Capable Devices ARObjectScanning Back Camera iOS 12.0+ ARKit Capable Devices
17. AR Configuration Mode Camera iOS Device AROrientationTracking Back Camera iOS 11.0+ ARKit Capable Devices ARWorldTracking Back Camera iOS 11.0+ ARKit Capable Devices ARFaceTracking Front Camera iOS 11.0+ Only iPhone X ARImageTracking Back Camera iOS 12.0+ ARKit Capable Devices ARObjectScanning Back Camera iOS 12.0+ ARKit Capable Devices
18. run(_ configuration) Basics of ARKit Recap ARConfiguration ARFrame ARFrame ARFrame CMMotionManager AVCaptureSession currentFrame ARSession
19. Behind the Scenes—Inertial Odometry Motion sensor Time Image ARFrame Motion + Motion +
20. Behind the Scenes—Inertial Odometry Motion sensor Image ARFrame Motion + Motion + Time Image ARFrame Motion + Image ARFrame Motion + Image ARFrame Motion + Motion + Image ARFrame Motion + Motion +
21. Orientation Tracking API AROrientationTrackingConfiguration open class ARCamera : NSObject, NSCopying { open var transform: simd_float4x4 { get } open var eulerAngles: simd_float3 { get } … } AROrientationTrackingConfiguration
22. Behind the Scenes—Visual Inertial Odometry Motion data and computer vision Image ARFrame Motion + Image ARFrame Motion + Image ARFrame Motion + Image ARFrame Motion CV Image ARFrame Motion CV Time 1 m
23. Behind the Scenes—Visual Inertial Odometry Extracting robust features
24. Behind the Scenes—Visual Inertial Odometry Triangulation creates World Map
25. Behind the Scenes—Visual Inertial Odometry Initialization C W
26. AROrientationTracking ARWorldTracking
27. Share AR!
28. What is “share AR?” “share AR” means share World tracking information and 3D models Use ARWorldMap class
29. ARWorldMap ARWorlMap͕อ࣋͢Δ৘ใ 3Dͷಛ௃఺ͳͲͷτϥοΩϯά৘ ใͱɺ഑ஔ͞ΕͨΞϯΧʔ৘ใɻ ARWorldTrackingConfigurationͰͷΈ औಘ͢Δ͜ͱ͕Ͱ͖Δɻ Internal tracking data • Map of 3D feature points • Local appearance List of named anchors World Map Data A
30. Share the World Map // Retrieve world map from ARSession session.getCurrentWorldMap { worldMap, error in guard let worldMap = worldMap else { showAlert(error) return } // Serialize let data = try NSKeyedArchiver.archivedData(withRootObject: worldMap, requiringSecureCoding: true) }
31. Relocalize to World Map Setup configuration // Receive / Load world map let worldMap : ARWorldMap = ... // Create a session configuration let configuration = ARWorldTrackingConfiguration() configuration.initialWorldMap = worldMap // Run the session session.run(configuration) W
32. SwiftShot ARWorldMapΛMultipeerConnectivity্Ͱڞ༗͢ΔήʔϜ SwiftShot https://developer.apple.com/documentation/arkit/ swiftshot_creating_a_game_for_augmented_reality Inside SwiftShot: Creating an AR Game https://developer.apple.com/videos/play/wwdc2018/605
33. 3D Object Scan
34. 3D Object Scan ARKit Ͱ 3D ObjectΛScanͯ͠ɺ ARReferenceObjectΛ࡞Δ ARReferenceObjectΛ࢖ͬͯɺͦͷ෺ମ͕ Ͳ͜ʹଘࡏ͢Δ͔ݕ஌Ͱ͖Δ
35. 3D Object Scan ARKitͰscanͯ͠3DϞσϧ͕࡞੒Ͱ͖ Δʁ NO! ARKitͷ scanning͸ɺ෺ମ͔Βಛ௃఺ͷ ू߹Λอଘ͢Δ͚ͩɻ ͦͷ෺ମΛݕ஌͢Δ͜ͱ͸Ͱ͖ͯ΋ɺ ෺ମͷ3DϞσϧ͸࡞Εͳ͍ɻ How to Acquire an Object? Similar representation as a world map
36. How to get ARReferenceObject? AppleͷαϯϓϧίʔυɺScanning and Detecting 3D ObjectsΛ࢖͏ https://developer.apple.com/documentation/arkit/ scanning_and_detecting_3d_objects
37. How to use ARReferenceObject? Scanning and Detecting 3D Objects αϯϓϧΛ࢖ͬͯɺ ARReferenceObjectΛεΩϟϯɻΤΫεϙʔτͯ͠ɺ*.arobjectΛऔಘɻ XcodeͰɺARͷ৽نϓϩδΣΫτΛ࡞੒͠ɺAsset CatalogʹAR Resource GroupΛ࡞Γɺ͜ͷϑΝΠϧΛDrag&Dropɻ
38. Prepare for Scan override func viewWillAppear(_ animated: Bool) { super.viewWillAppear(animated) let configuration = ARWorldTrackingConfiguration() configuration.detectionObjects = ARReferenceObject.referenceObjects(inGroupNamed: "apple", bundle: nil)! sceneView.session.run(configuration) } }
39. Detection of Object func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) { if let objectAnchor = anchor as? ARObjectAnchor{ let foundObject = objectAnchor.referenceObject // check name" let objectname = foundObject.name // do something to the object! objectAnchor.addTitleNode(on: node) } }
40. To Scan Efficiently ໛༷͕͋Δ΋ͷ ςʔϒϧͷ্ʹͷΔ͘Β͍ͷେ͖͞ ॊΒ͔͍΋ͷͰ͸ͳ͘ɺݻ͍෺ମ ҆ఆͨࣨ͠಺র໌ Use iPhoneX !
41. Personal Recommendation Nintendo Amiibo
42. ARObjectScanningConfiguration ARObjectScanningConfiguration is for use only in development scenarios. High-fidelity spatial mapping has a high performance and energy cost, and disables ARKit features not necessary for reference object scanning. To produce AR experiences for end users, use ARWorldTrackingConfiguration. Appleͱͯ͠͸ɺObject ScanϞʔυ͸ɺ։ൃϑΣʔζͰ࢖͏΋ͷͱ૝ఆ͠ ͍ͯΔΑ͏Ͱ͢ɻ ։ൃऀ͕ಛఆͷΦϒδΣΫτΛScan͠ɺͦ͜Ͱநग़ͨ͠ ARReferenceObjectΛΞϓϦʹ͍ΕͯϦϦʔεɻ
43. ·ͱΊ ARKit 2.0Ͱ͸͓΋͠Ζ͍ػೳ͕͍Ζ͍Ζ௥Ճ͞Ε͍ͯ·͢ɻ AR Share Object Scan USDZ ϑΝΠϧΛ࢖͑͹࣮૷͕΄ͱΜͲ͍Γ·ͤΜɻ Ͱ͖Δָ͚ͩΛ͠·͠ΐ͏ʂ