Building VR experience with React VR

This presentation shows how you can use ReactVR to build Virtual Reality experience.

Transcript

1. Building VR experience with React VR
2. About me Mirek Ciastek Front-end Developer in SwingDev https://twitter.com/mciastek https://medium.com/@mciastek https://github.com/mciastek
3. About Swing SwingDev is a Polish-American company building software solutions for enterprise and startup clients in Silicon Valley. Our creative department, Swing3D, specialises in AR and VR technologies, building 3D modelling tools and interactive presentations of 3D projects. www.swingdev.io www.swing3d.io
4. State of VR in Web What is ReactVR? ReactVR basics Extending ReactVR Demo Docs and articles Agenda
5. State of VR in Web
6. Browser support https://github.com/googlevr/webvr-polyfill WebVR polyfill
7. What is ReactVR?
8. React + React Native + three.js Main Thread ReactVR Runtime React Native WebGL/WebVR OVRUI Three.js WebWorker ReactVR Application Code React Native React Web Browser Asynchronous Bridge React VR lets you build VR apps using only JavaScript. It uses the same design as React, letting you compose a rich VR world and UI from declarative components.
9. 1 2 3 <View> <Pano source={asset(‘chess-world.jpg')}/> <Text style={{ backgroundColor: '#777879', fontSize: 0.8, layoutOrigin: [0.5, 0.5], paddingLeft: 0.2, paddingRight: 0.2, textAlign: 'center', textAlignVertical: 'center', transform: [{translate: [0, 0, -3]}], }}> hello </Text </View> 1 2 3
10. 3D object primitives - Box, Sphere, Plane, Cylinder, etc. Panorama - both photo and video panoramas Lights - AmbientLight, DirectionalLight, PointLight, SpotLight User interactions through buttons and events - VrButton and e.g. onInput event Loading models in OBJ and GLTF format Animations CSS-like styles and flexbox (inherited from React Native) …and CLI ReactVR gives us..
12. Let’s build a room
13. Maybe something like this
14. First attempt
15. I prefer the real 3D experience
16. Final project
18. know basics of 3D know how to write your own shaders know very well API of three.js know how WebGL works have a dedicated team of 3D artists and 3D programmers You can do it yourself, if you…
19. Building this scene entirely in ReactVR is very difficult. You should use a 3D programme (like Blender or 3ds Max) to create a 3D model and then import it to ReactVR. ReactVR supports OBJ and GLTF formats by default, but you can add loaders for your own formats. You can’t build beautiful scene using only ReactVR primitives. You need to go further… How you can use ReactVR for such project?
20. By extending ReactVR, you can make pretty nice cursor navigation based on Navigation Mesh. Extending React VR
21. Navigation mesh made in any 3D programme, loaded in any format that three.js supports (I used JSON) Custom cursor component using Animated class Custom module for navigation mesh Raycaster to check if user can go to pointed direction RCTDeviceEventEmitter to communicate between UI and WebWorker What you need
22. Cursor component Add components <AnimatedPlane dimWidth={SIZE} dimHeight={SIZE} materialParameters={{ side: THREE.FrontSide, blending: THREE.AdditiveBlending, depthWrite: false, }} texture={asset('cursor-outline.png')} style={{ opacity: this.state.opacity, transform: [{ scale: this.state.scale, }], }} /> <Plane dimWidth={SIZE} dimHeight={SIZE} materialParameters={{ side: THREE.FrontSide, depthWrite: false, }} texture={asset('cursor-bg.png')} style={{ opacity: baseOpacity, }} />
23. Cursor component Animated.parallel([ Animated.timing(this.state.opacity, { duration: 300, toValue: 0, }), Animated.timing(this.state.scale, { duration: 300, toValue: 1.5, }), ]).start(this.handleAnimationEnd); Run the animation handleAnimationEnd = () => { Animated.timing(this.state.opacity, { duration: 200, toValue: OPACITY, }).start(); this.state.scale.setValue(1); if (this.props.onAnimationEnd) { this.props.onAnimationEnd(); } }; Reset scale on animation end
24. Nav mesh module Load navigation mesh load() { this.loader = new THREE.JSONLoader(); this.loader.load( ‘/static_assets/nav-mesh.json’, this.handleModelLoad ); } Emit event on camera move this.rtContext.callFunction( 'RCTDeviceEventEmitter', 'emit', ['teleport', this.targetPos.x, this.targetPos.z], );
25. Nav mesh module Check for intersections with navigation mesh // Set raycast from camera this.raycaster.setFromCamera({ x: this.lastX, y: this.lastY }, this.camera); // Get intersections const objectsIntersections = this.raycaster.intersectObjects(this.objects); // Check if there are any intersection with nav mesh if ( objectsIntersections[0] && objectsIntersections[0].object === this.mesh ) { // Move your camera }
26. Index.vr.js Add Cursor component <Cursor animate={this.state.cursorAnimate} onAnimationEnd={this.handleCursorAnimationEnd} visible={this.state.cursorVisibility} style={{ transform: [{ rotateX: -90, }, { translate: [this.state.cursorX, -this.state.cursorZ, -0.098], }], opacity: this.state.cursorOpacity, }} />
27. Index.vr.js Add event listener RCTDeviceEventEmitter.addListener('teleport', (x, z) => { this.setState({ cursorAnimate: true, }); Animated.timing(this.state.camPos, { toValue: { x, y: z }, duration: 500, easing: Easing.inOut(Easing.quad), }).start(this.handleCameraAnimationEnd); });
28. Demo Thanks to: Bartek Gadzała Tomek Wieczorkowski
29. Docs and articles Getting Started with ReactVR Oculus Developer Center WebVR Rocks Prototyping with React VR Getting Started with WebVR
30. Thank you