Building Blocks of WebVR

Virtual Reality is one of the hottest technologies lately. Learn how to use it in your browser and avoid user experience pitfalls.

Presented during SwingDev's WebVR meetup.

Transcript

1. www.swing3d.io Building Blocks of WebVR Introduction to VR and WebVR with Mirek Ciastek, SwingDev’s Front-end Developer
3. What’s virtual reality It’s a computer-generated scenario that simulates a realistic experience. The immersive environment can be similar to the real world in order to create a lifelike experience grounded in reality or sci-fi.
4. Back to the past
6. 90’s and recently
8. Today
9. Designing for VR
10. Key concpets of VR Head-mounted display (HMD) Position / Orientation / Velocity / Acceleration Field of View Stereoscopic vision
11. UX of VR • Comfortable using • In-world interface • Immersive sounds • Uninterrupted movement • Interaction with world
12. Problems of VR Hardware Health issues
13. Best experience = High price +
16. VR in browser
17. From VR to WebVR WebVR is an open specification that makes it possible to experience VR in your browser. The goal is to make it easier for everyone to get into VR experiences, no matter what device you have. All this thanks to browser environment!
18. APIs useful for WebVR • WebVR API - manages displays • WebGL - provides a way to run 3D in the browser • Gamepad API - helps with using controllers • Web Audio API - if you want to add some sound • Web Video API - for 360 videos • Web Workers - might give good performance boost
19. Libraries
20. Let’s make a VR in the browser
21. Make a VR scene
22. setScene () { this.camera = new THREE.PerspectiveCamera( 60, ASPECT_RATIO, 0.01, 10000 ) this.scene = new THREE.Scene() this.scene.add(this.camera) this.scene.background = new THREE.Color().setHSL(0.6, 0, 1) this.scene.fog = new THREE.Fog( this.scene.background, 1, 5000 ) } setRenderer () { const canvasEl = document.getElementById('scene') this.renderer = new THREE.WebGLRenderer({ canvas: canvasEl }) this.renderer.setPixelRatio(window.devicePixelRatio) this.renderer.setSize(window.innerWidth, window.innerHeight) this.renderer.gammaInput = true this.renderer.gammaOutput = true this.renderer.setClearColor(0x000000) this.renderer.shadowMap.enabled = true }
23. setComponents () { const sky = Sky() const ground = Ground() this.wall = new Wall(this.renderer) this.turret = new Turret(this.renderer) this.cursor = new Cursor() this.turret.init() .then(this.handleModelLoad) this.wall.init() .then(this.handleModelLoad) this.scene.add(this.cursor.mesh) this.scene.add(sky) this.scene.add(ground) } setLights () { const { hemiLight, dirLight } = makeLights() this.scene.add(hemiLight) this.scene.add(dirLight) }
24. Allow to move camera
25. MousePanControls handleMouseMove = (e) => { if (!this.tracking) { return } const width = this.target.innerWidth || this.target.clientWidth const height = this.target.innerHeight || this.target.clientHeight const deltaX = (typeof this.lastX === 'number') ? e.screenX - this.lastX : 0 const deltaY = (typeof this.lastY === 'number') ? e.screenY - this.lastY : 0 this.lastX = e.screenX this.lastY = e.screenY this.yaw += THREE.Math.degToRad( deltaX / width * this.camera.fov * this.camera.aspect ) this.pitch += THREE.Math.degToRad(deltaY / height * this.camera.fov) this.pitch = Math.max(-HALF_PI, Math.min(HALF_PI, this.pitch)) };
26. DeviceOrientationControls update () { if (!this.enabled) { return } const alpha = this.deviceOrientation.alpha || 0 const beta = this.deviceOrientation.beta || 0 const gamma = this.deviceOrientation.gamma || 0 const orientation = this.screenOrientation // Update the camera rotation quaternion const quaternion = this.camera.quaternion euler.set(beta, alpha, -gamma, 'YXZ') quaternion.setFromEuler(euler) if (this._initialAlpha !== null) { rotation.setFromAxisAngle(Y_UNIT, -this._initialAlpha) quaternion.premultiply(rotation) } quaternion.multiply(SCREEN_ROTATION) rotation.setFromAxisAngle(Z_UNIT, -orientation) quaternion.multiply(rotation) }
27. VRControls class VRControls { constructor (camera, vrDisplay) { this.camera = camera this._vrDisplay = vrDisplay } update (options) { const pose = options.frameData ? options.frameData.pose : null if (pose) { if (pose.position) { this.camera.position.fromArray(pose.position) } if (pose.orientation) { this.camera.quaternion.fromArray(pose.orientation) } } } }
28. Allow to interact with world
29. GamepadControls listenGamepadEvents () { const gamepads = getGamepads() // navigator.getGamepads for (let i = 0; i < gamepads.length; i += 1) { if (gamepads[i]) { const buttons = gamepads[i].buttons for (let j = 0; j < buttons.length; j += 1) { if (isPressed(buttons[j])) { this.batchedEvents.push({ type: 'GamepadEvent', eventType: 'keydown', button: buttons[j], gamepad: i }) } } } } }
30. Check for box hit hit (origin, direction) { raycaster.set(origin, direction) const intersects = raycaster.intersectObjects(this.root.children) for (let i = 0; i < intersects.length; i += 1) { const { object } = intersects[i] this.root.remove(object) } if (this.root.children.length === 0) { this.build() } }
31. Add some extra features
32. Crosshair const texture = new THREE.TextureLoader().load('images/crosshair.png') const geometry = new THREE.PlaneGeometry( DEFAULT_CURSOR_WIDTH, DEFAULT_CURSOR_WIDTH ) const material = new THREE.MeshBasicMaterial({ transparent: true, side: THREE.DoubleSide, depthTest: false, depthWrite: false, map: texture }) this.mesh = new THREE.Mesh(geometry, material) this.mesh.raycast = () => null this.mesh.renderOrder = 1
33. Explosions const particleSettings = { texture: { value: textureLoader.load(smokeImage) }, depthTest: true, depthWrite: false, blending: THREE.NormalBlending, maxParticleCount: 1000 } const emitters = [ { particleCount: 600, type: SPE.distributions.SPHERE, position: { radius: 0.1 }, maxAge: { value: 0.5 }, activeMultiplier: 20, velocity: { value: new THREE.Vector3(1.2) }, size: { value: 1.5 }, opacity: { value: [0.5, 0] } } ] // init particles initParticles () { this.particleGroup = new SPE.Group( particleSettings ) this.particleGroup.addPool(1, emitters, false) this.headGroup.add(this.particleGroup.mesh) } // play sound and show smoke triggerSmoke () { player.play('shot') this.particleGroup.triggerPoolEmitter( 1, smokePosition ) } I used Shader Particle Engine
34. Sounds class SoundPlayer { constructor () { this.context = new (window.AudioContext || window.webkitAudioContext)() this.bufferLoader = new BufferLoader( this.context, sounds, this.handleLoad ) this.loaded = false this.bufferLoader.load() } handleLoad = () => { this.loaded = true } play (soundName) { if (this.loaded) { const source = this.context.createBufferSource() source.buffer = this.bufferLoader.bufferList[soundName] source.connect(this.context.destination) source.start(0) } } }
35. …and allow to enter VR mode VRDisplay API provides all needed methods and properties to serve your scene in VR compatible format • eyes position - you need to slice your scene in half • isPresenting flag - check if you can serve VR frames • frameData - includes device position, etc. • submitFrame - remember to call it when your frame is prepared Check three.js VREffect interface for more details
36. Result See the demo Check out the code
38. How you can extend it • Use any WebVR framework (A-frame or ReactVR) - the farther you go the harder it gets to maintain all the things • Add any UI to allow user to track their progress • Add some physics - make the wall collapse • Add support for 3DoF and 6DoF controller - yes, you can do it in your browser!
39. Further reading 1/2 • The User Experience of Virtual Reality • Virtual Reality Basics • VR for UX Designers: What I Learned During My First Project • History Of Virtual Reality • The very real health dangers of virtual reality • The Story of VR - A Look at the History Behind Virtual Reality • Design Practices in Virtual Reality • The Future of Virtual Reality • Designing Screen Interfaces for VR (video)
40. Further reading 2/2 • WebVR concepts • Introduction to VR Web • Building Virtual Reality on the Web with WebVR (video) • Using the WebVR API • Using the Gamepad API • Using VR controllers with WebVR • Getting started with three.js • Web Audio API
41. Thank you