Getting Started With MRTK (for Beginner)

Transcript

1. Getting Started With MRTK for Beginner
2. #AR_Fukuoka Hashtag
3. Goal Learn about object manipulation as the basic usage of MRTK https://youtu.be/qUQXZACp35A
4. Sample Data http://arfukuoka.lolipop.jp/mrtk/Sample.zip
5. Downloading MRTK (1/3) https://github.com/Microsoft/MixedRealityToolkit-Unity releases
6. Downloading MRTK (2/3) Microsoft Mixed Reality Toolkit v2.4.0 is used for this tutorial 少し下にスクロール
7. Downloading MRTK (3/3) Download Foundation.2.4.0.unitypackage Foundation.2.4.0.unitypackage
8. Creating Project (1/6) Launch UnityHub and click Project Project
9. Creating Project (2/6) Click ▽ aligned on the right side of NEW button. ▽
10. Creating Project (3/6) Select 2018.4.xxf1 (2018.4.23f1 is selected in this deck) 2018.4.xxf1
11. Creating Project (4/6) ①3D ②Set the project name (ex.ARFukuoka_MRTK) ③Select the directory to save the project
12. Creating Project (5/6) Create
13. Creating Project (6/6) Unity Editor will appear
14. Installation of MRTK (1/3) Double click the unitypackage. Click Import button of Import Unity Package dialog. Microsoft.MixedReality. Toolkit.Unity.Foundation .2.4.0.unitypackage Import
15. Installation of MRTK (2/3) Apply Click Apply button to switch the setting to xR.
16. Installation of MRTK (3/3) Close Close MRTK Project Configurator dialog.
17. Adding 3D Object into the Space (1/3) Right click blank of Hierarchy
18. Adding 3D Object into the Space (2/3) ①3D Object ②Cube
19. Adding 3D Object into the Space (3/3) Cube should appear
20. Run View from camera will be shown. Click to play Click it again to stop preview
21. Modification of Objectʼs Transform Click Cube
22. Modification of Objectʼs Transform Move Rotate Scale
23. Inputting Values to Transform Object Click Cube
24. Inputting Values to Transform Object p Properties of each object can be added/edited in Inspector p Position/Rotation/Scale are editable in Transform. Inspector
25. Inputting Values to Transform Object Set as following Positionを 0 0 1 Rotation 0 0 0 Scale 0.3 0.3 0.3 p Properties of each object can be added/edited in Inspector p Position/Rotation/Scale are editable in Transform.
26. Controlling Viewpoint of Scene Letʼs set the object at the center of scene tab
27. Controlling Viewpoint of Scene Double Click Cube
28. Controlling Viewpoint of Scene Object can be shown on the center of Scene tab.
29. Controlling Viewpoint of Scene [←] [→]: Move to Left/RIght [↑][↓]: Zoom In/Out [Alt]+Drag: Rotation +Drag: Move to ←→↑↓
30. Next Step Not only Transform but also appearance and behavior and properties can be edited in Inspector
31. Changing Color of Cube (1/4) ① Cube ② See Mesh Renderer ③ Open Materials
32. Changing Color of Cube (2/4) Element0
33. Changing Color of Cube (3/4) Appearance information is described in Material file
34. Changing Color of Cube (4/4) Detail of Material is shown at the bottom of Inspector. But Default-Material is not editable.
35. Itʼs necessary to create Material file to edit appearance of a object.
36. Creating Material (1/3) Create Material to describe color of Cube. ①Assets ②Right Click
37. Creating Material (2/3) Create Material to describe color of Cube. ①Create ②Material
38. Creating Material (3/3) New Material should appear Create Material to describe color of Cube.
39. Application of Material (1/2) Open Materials of Cube in Inspector ② See Default-Material ①Cube
40. Application of Material (2/2) Replace Default-Material with New Material. Drag & Drop into Default-Material
41. Editing Color of Cube (1/3) ①Cube ②Material should be New Material
42. Editing Color of Cube (2/3) Open NewMaterial Click □ next to Albedo
43. Editing Color of Cube (3/3) Color should be changed
44. How to Save this Scene (1/4) File
45. How to Save this Scene (2/4) SaveAs...
46. How to Save this Scene (3/4) ①Name as Sample1 ②Save
47. How to Save this Scene (4/4) Sample1 should be appear in Assets
48. You can override the scene with current info by Ctrl / Command + S
49. Letʼs use MRTK from now!
50. What Youʼll Do at First? Prepare moving around and using hand in UnityEditor
51. Using Camera of MRTK (1/3) MixedRealityToolkit
52. Using Camera of MRTK (2/3) Add to Scene and Configure...
53. Using Camera of MRTK (3/3) MixedRealityToolKit and MixedRealityPlayspace should appear
54. Play (1/4) Click Play
55. Play (2/4) [A][D] :Move to Left and Right [W][S] :Zoom In/Out [Q][E] :Move to Up and Down RightClick + Drag :Rotation
56. Play (3/4) [Shift] :Show left hand [Space] :Show right hand While hand is appearing+ [Click] :Pinch [Scroll] :Forward/Back
57. Play (4/4) Stop before going to next step
58. Letʼs manipulate Cube
59. Making Cube Manipulatable (1/3) ①Cube ②Add Component
60. Making Cube Manipulatable (2/3) Search [manipulation] Select ManipulationHandler
61. Making Cube Manipulatable (3/3) ①Cube ②ManipulationHandler should be added
62. Enable Near Interaction ①Cube ②Add Component
63. Enable Near Interaction ①Search [Near] ②NearInteractionGrabbable
64. Enable Near Interaction ①Cube ②NearInteractionGrabbable should be added
65. Play (1/4) Click Play
66. One Hand Manipulation
67. Play (2/4) Shift or Space
68. Play (3/4) Put the cursor on Cube
69. Play (4/4) Drag and move Cube
70. Two Hand Manipulation
71. Play (1/5) Show left hand by pushing Shift
72. Play (2/5) ①Click after putting cursor on Cube ② Hit [T] to fix left hand
73. Play (3/5) Show right hand by pushing Space
74. Play (4/5) Drag and move
75. Play (5/5) You can Rotate/Scale Cube
76. Letʼs Add Another Object
77. Adding Obj File(1/7) Assets
78. Adding Obj File (2/7) Sample Folder → Model
79. Adding Obj File (3/7) Drag & Drop Model into Assets
80. Adding Obj File (4/7) Model
81. Adding Obj File (5/7) model.obj
82. Adding Obj File (6/7) Drag & Drop model.obj into Hierarchy
83. Adding Obj File (7/7) ①model Set chair as following nbv Position 0.5 -0.3 1 Rotation 0 180 0
84. In order to enable chair to detect contact with hand, Collider should be added to model.obj
85. Adding Collider (1/5) ①model ②Add Component
86. Adding Collider (2/5) ①Search box ②BoxCollider
87. Adding Collider (3/5) Bounding box will appear
88. Adding Collider (4/5) ①model ②Edit Collider
89. Adding Collider (5/5) Adjust position of each face by moving▪
90. Make Chair Manipulatable (1/5) ①model ②Add Component
91. Make Chair Manipulatable (2/5) ① Searchmanipulation ②ManipulationHandler
92. Make Chair Manipulatable (3/5) ①model ②Add Component
93. Make Chair Manipulatable (4/5) ①Search near ②NearInteractionGrabbable
94. Make Chair Manipulatable (5/5) ①model ②NearInteractionGrabbable should be appear
95. Restrict the Direction of the Chair ①model ②Set One Hand Rotation Mode Near/Far to Gravity Aligned Maintain Rotation to User
96. Allow Only Rotation for 2 Hand Operation ①model ②Set Rotate for Two Hand Manipulation Type ③ Set Y Axis Only for Constraint On Rotation
97. Play Put cursor on chair Drag to move Rotate with to hands
98. Play Stop
99. Remove Performance Viewer Can be deleted
100. Remove Performance Viewer ①MixedRealityToolKit ②Copy & Customize
101. Remove Performance Viewer ①Change Profile Name to Sample1 MixedRealityToolkit... ②Clone
102. Remove Performance Viewer ①Diagnostics ②Turn Enable Diagnostics System into Off
103. Play
104. Play Stop
105. p HoloLens2 p ARFoundation(ARKit/ARCore) Running On AR Devices
106. p HoloLens2 p ARFoundation(ARKit/ARCore) Running On AR Devices
107. Build Setting for HoloLens2 File
108. Build Setting for HoloLens2 Build Setting
109. Build Setting for HoloLens2 ①Universal Windows Platform ②Switch Platform
110. Build Setting for HoloLens2 Apply
111. Build Setting for HoloLens2 Player Settings...
112. Build Setting for HoloLens2 ①XR Settings ②Select 16-bit depth ad Depth Format
113. Build for HoloLens2 Build
114. Build for HoloLens2 New Folder
115. Build for HoloLens2 Rename new folder to build
116. Build for HoloLens2 ①build ②Select Folder
117. Wait for a moment...
118. Build for HoloLens2 Open build
119. Build for HoloLens2 Open solution (*.sln) with VisualStudio2019
120. Build for HoloLens2 Release ARM
121. Build for HoloLens2 ▼
122. Build for HoloLens2 Device
123. Connect HoloLens2 and PC by using USB cable
124. Build for HoloLens2 ① Debug ②Start without debug
126. p HoloLens2 p ARFoundation(ARKit/ARCore) Running On AR Devices
127. p HoloLens2 p ARFoundation(ARKit/ARCore) Running On AR Devices
128. Instalation of ARFoundation Window
129. Instalation of ARFoundation Package Manager
130. Instalation of ARFoundation ①AR Foundation ②Open version list
131. Instalation of ARFoundation ①All Version ②1.5.0-preview.6
132. Instalation of ARFoundation Install
133. Setting procedure for iOS is from next page.
134. Installation of ARKit (for iOS) ①ARKit XR Plugin ②2.1.2
135. Installation of ARKit (for iOS) Install
136. Setting procedure for Android is from next page.
137. Installation of ARCore (for Android) ①ARCore XR Plugin ②2.1.2
138. Installation of ARCore (for Android) Install
139. Setting procedure for both ARCore/ARKi is from here.
140. Build Setting for ARFoundation ①MixedRealityToolkit ②DefaultHoloLens2Configuration
141. Build Setting for ARFoundation Copy & Customize
142. Build Setting for ARFoundation ①Set ProfileName as Mobile MixedRealityToolkit... ②Clone
143. Build Setting for ARFoundation ①Camera ②Clone
144. Build Setting for ARFoundation ①Set ProfileName as Mobile MixedRealityCameraProfile ②Clone
145. Build Setting for ARFoundation Camera Setting Providers
146. Build Setting for ARFoundation -
147. Build Setting for ARFoundation AddCameraSettingsProvider
148. Build Setting for ARFoundation Newdataprovider 0
149. Build Setting for ARFoundation Open Taypeを開く
150. Build Setting for ARFoundation ①MicrosoftMixedReality.Toolkit.Experimental.UnityAR ②UnityARCameraSetting
151. Remofe Performance Viewer Diagnostics
152. Remofe Performance Viewer Turn Off Enable Diagnostics System
153. Build for ARFoundation File
154. Build for ARFoundation Build Settings...
155. Build for ARKit(iOS) ①iOS ②Switch Platform
156. Build for ARCore(Android) ①Android ②Switch Platform
157. Build for ARFoundation Apply
158. Build for ARFoundation Player Settings...
159. Setting procedure for iOS is from next page.
160. Build for ARKit(iOS) Other Settings
161. Build for ARKit(iOS) Set Bundle Identifier ex) com.Yoshinaga.ARFukuoka
162. Build for ARKit(iOS) Strip Engine CodeをOff
163. Setting procedure for Android is from next page.
164. Build for ARCore(Android) Set Package Name ex)com.yourname.arfukuoka
165. Build and Run for ARCore/ARKit
166. 動作の様⼦
167. Ctrl / Command + S Save
168. Letʼs Use Spatial Awareness Next Step is...
169. Use room mesh model
170. Duplicate Current Scene File
171. Duplicate Current Scene Save As...
172. Duplicate Current Scene Save As...
173. Import of Spatial Mesh Model Assets
174. Import of Spatial Mesh Model Import New Asset
175. Import of Spatial Mesh Model ①room.obj ②import
176. Import of Spatial Mesh Model room should appear
177. Import of Spatial Mesh Model MixedRealityToolkit Clone
178. Import of Spatial Mesh Model ①Set ProfileName as Sample2 MixedRealityToolkit... ②Clone
179. Visualization of Spatial Mesh Model ①MixedRealityToolkit ②Spatial Awareness
180. Visualization of Spatial Mesh Model Clone
181. Visualization of Spatial Mesh Model ①Set ProfileName as Sample2MixedRealityCameraProfile ②Clone
182. Visualization of Spatial Mesh Model Windows Mixed Reality Spatial Mesh Observer
183. Visualization of Spatial Mesh Model Typeを開く
184. Visualization of Spatial Mesh Model ①Microsoft.MixedReality.Toolkit.SpatialObjectMeshObserver ②SpatialObjectMeshObserver
185. Visualization of Spatial Mesh Model Clone
186. Visualization of Spatial Mesh Model ①Set ProfileName as Sample2SpatialObjectMeshObserver... ②Clone
187. Visualization of Spatial Mesh Model See Spatial Mesh Object
188. Visualization of Spatial Mesh Model See room (Do NOT click)
189. Visualization of Spatial Mesh Model Drag&Drop into ExampleSpatialMesh
190. Visualization of Spatial Mesh Model Replaced to room
191. Replace Material of Spatial Mesh Click ◦ aligned on the right of MRTK_ Wireframe
192. Replace Material of Spatial Mesh Defaultで検索 Defaultを選択 ※他のMtaterialでもOK
193. Replace Material of Spatial Mesh Example Spatial Mesh Default Material
194. Using Gravity ①model ②Add Component
195. Using Gravity ①Search rigid ②Rigidbody
196. Using Gravity ①model ②Rigidbody ③Turn ON Use Gravity
197. Run
198. Depending on the development environment, the chair may fall off before the mesh appears on the scene. Keep the chair in the air at first and allow it to fall when you let go of your hand.
199. Turn Off Gravity ①model ② See Rigidbody
200. Turn Off Gravity ①Use GravityをOff ②Is KinematicをOn Note︓ Turn Is Kinematic On to prevent physical interference. Prevents the chair from hitting the mesh and flying away when the app is launched.
201. Turn On Gravity After Manipulation model The ManipulationHandler has OnManipulation Started/Ended, which makes it easy to get the operation started and finished.
202. Turn On Gravity After Manipulation ①model ②Add Component
203. Turn On Gravity After Manipulation ①Clear Search Area ③Set Script Name ex)PhysicsScript ②New script ④Create and Add
204. Letʼs Write Script ①model ②Physics Script should appear.
205. Letʼs Write Script ①model ②Open Physics Script
206. using System.Collections; using System.Collections.Generic; using UnityEngine; public class PhysicsScript : MonoBehaviour { // Start is called before the first frame update void Start() { } // Update is called once per frame void Update() { } } Editing Script
207. using System.Collections; using System.Collections.Generic; using UnityEngine; public class PhysicsScript : MonoBehaviour { Delete Start and Update function } Editing Script
208. using System.Collections; using System.Collections.Generic; using UnityEngine; public class PhysicsScript : MonoBehaviour { //SetGravityにtrue/falseを渡すことで重⼒および物理⼲渉のOn/Offを切り替える public void SetGravity(bool flag) { //重⼒のOn/Offを設定 gameObject.GetComponent<Rigidbody>().useGravity = flag; //物理的な⼲渉のOn/Offを設定 gameObject.GetComponent<Rigidbody>().isKinematic = !flag; } } Function to Switch On/Off Physics
209. Turn On Gravity when Manipulation is Ended ①model ②On Manipulation Ended ③Click +
210. Turn On Gravity when Manipulation is Ended ①model Drag & Drop into area of shown as None
211. Turn On Gravity when Manipulation is Ended ①model ②Click No Function
212. Turn On Gravity when Manipulation is Ended ②SetGravity(bool) ①PhysicsScript
213. Turn On Gravity when Manipulation is Ended ①model ③Turn On ②See Manipulation Ended
214. Setting Hand Gesture ①model
215. Setting Hand Gesture ①SetManipulation Type as One Handed Only ②Set Constraint on Rotation as None
216. The object (chair) will rebound and roll when spatial meshes are refreshed on the actual HoloLens. Turn off gravity as soon as collision with spatial mesh
217. using System.Collections; using System.Collections.Generic; using UnityEngine; public class HandScript : MonoBehaviour { //SetGravityにtrue/falseを渡すことで重⼒および物理⼲渉のOn/Offを切り替える public void SetGravity(bool flag) { gameObject.GetComponent<Rigidbody>().useGravity = flag; gameObject.GetComponent<Rigidbody>().isKinematic = !flag; } //Called when collision is occured private void OnCollisionEnter(Collision collision) { SetGravity(false); } }
218. Delete Unused Object Remove Cube
219. Delete Unused Object
220. Play on Unity Editor
221. Letʼs build and deploy this application into xR device.
222. p HoloLens2 p ARFoundation(ARKit/ARCore) Run On Each Device
223. p HoloLens2 p ARFoundation(ARKit/ARCore) Run On Each Device
224. Setting for HoloLens2 ① model ② Spatial Awareness
225. Setting for HoloLens2 See Windows Mixed Reality Spatial Mesh Observer
226. Setting for HoloLens2 Open dropdown menu aligned on right of Type
227. Setting for HoloLens2 ①MicrosoftMixedReality.Toolkit.WindowsMixedReality.SpatialAwareness ②MicrosoftMixedReality.Spatial.MeshObserver
228. Build -> Deploy -> Run Installation
230. p HoloLens2 p ARFoundation(ARKit/ARCore) Run On Each Device
231. p HoloLens2 p ARFoundation(ARKit/ARCore) Run On Each Device
232. Setting for AR Foundation ①MixedRealityToolkit ②See Mixed Reality Toolkit
233. Setting for AR Foundation MobileMixedRealityToolkitConfiguration (The setting made in the first half of tutorial)
234. Letʼs use plane detection instead of spatial awareness.
235. Using Plane Detection ①MixedRealityPlayspace ②Add Component
236. Using Plane Detection ①SearchAR Plane ②AR Plane Manager
237. Using Plane Detection ①MixedRealityPlayspace AR Session Origin & AR Plane Manager should be added
238. Using Plane Detection Right click on the blank of Hierarchy
239. Using Plane Detection ①XR ②AR Default Plane
240. Using Plane Detection ①See AR Default Plane ②Assets
241. Using Plane Detection Drag&Drop AR Default Plane into Assets
242. Using Plane Detection Delete AR Default Plane
243. Using Plane Detection ①MixedRealityPlayspace ②See Plane Prefab
244. Using Plane Detection ①SeeAR Default Plane (※ Do NOT click) ②Drag & Drop into None aligned right of Plane Prefab
245. Using Plane Detection Plane Prefab should be AR Default Plane
246. Build -> Deploy -> Run Installation
248. Fin.