Share
Embed
Building Blocks of WebVR
Virtual Reality is one of the hottest technologies lately. Learn how to use it in your browser and avoid user experience pitfalls.
Presented during SwingDev's WebVR meetup.
mciastek
February 22, 2018
Transcript
41 slides
-
www.swing3d.io Building Blocks of WebVR Introduction to VR and WebVR
with Mirek Ciastek, SwingDev’s Front-end Developer -
What’s virtual reality It’s a computer-generated scenario that simulates a
realistic experience. The immersive environment can be similar to the real world in order to create a lifelike experience grounded in reality or sci-fi. -
Key concpets of VR Head-mounted display (HMD) Position / Orientation
/ Velocity / Acceleration Field of View Stereoscopic vision -
UX of VR • Comfortable using • In-world interface •
Immersive sounds • Uninterrupted movement • Interaction with world -
From VR to WebVR WebVR is an open specification that
makes it possible to experience VR in your browser. The goal is to make it easier for everyone to get into VR experiences, no matter what device you have. All this thanks to browser environment! -
APIs useful for WebVR • WebVR API - manages displays
• WebGL - provides a way to run 3D in the browser • Gamepad API - helps with using controllers • Web Audio API - if you want to add some sound • Web Video API - for 360 videos • Web Workers - might give good performance boost -
setScene () { this.camera = new THREE.PerspectiveCamera( 60, ASPECT_RATIO, 0.01,
10000 ) this.scene = new THREE.Scene() this.scene.add(this.camera) this.scene.background = new THREE.Color().setHSL(0.6, 0, 1) this.scene.fog = new THREE.Fog( this.scene.background, 1, 5000 ) } setRenderer () { const canvasEl = document.getElementById('scene') this.renderer = new THREE.WebGLRenderer({ canvas: canvasEl }) this.renderer.setPixelRatio(window.devicePixelRatio) this.renderer.setSize(window.innerWidth, window.innerHeight) this.renderer.gammaInput = true this.renderer.gammaOutput = true this.renderer.setClearColor(0x000000) this.renderer.shadowMap.enabled = true } -
setComponents () { const sky = Sky() const ground =
Ground() this.wall = new Wall(this.renderer) this.turret = new Turret(this.renderer) this.cursor = new Cursor() this.turret.init() .then(this.handleModelLoad) this.wall.init() .then(this.handleModelLoad) this.scene.add(this.cursor.mesh) this.scene.add(sky) this.scene.add(ground) } setLights () { const { hemiLight, dirLight } = makeLights() this.scene.add(hemiLight) this.scene.add(dirLight) } -
MousePanControls handleMouseMove = (e) => { if (!this.tracking) { return
} const width = this.target.innerWidth || this.target.clientWidth const height = this.target.innerHeight || this.target.clientHeight const deltaX = (typeof this.lastX === 'number') ? e.screenX - this.lastX : 0 const deltaY = (typeof this.lastY === 'number') ? e.screenY - this.lastY : 0 this.lastX = e.screenX this.lastY = e.screenY this.yaw += THREE.Math.degToRad( deltaX / width * this.camera.fov * this.camera.aspect ) this.pitch += THREE.Math.degToRad(deltaY / height * this.camera.fov) this.pitch = Math.max(-HALF_PI, Math.min(HALF_PI, this.pitch)) }; -
DeviceOrientationControls update () { if (!this.enabled) { return } const
alpha = this.deviceOrientation.alpha || 0 const beta = this.deviceOrientation.beta || 0 const gamma = this.deviceOrientation.gamma || 0 const orientation = this.screenOrientation // Update the camera rotation quaternion const quaternion = this.camera.quaternion euler.set(beta, alpha, -gamma, 'YXZ') quaternion.setFromEuler(euler) if (this._initialAlpha !== null) { rotation.setFromAxisAngle(Y_UNIT, -this._initialAlpha) quaternion.premultiply(rotation) } quaternion.multiply(SCREEN_ROTATION) rotation.setFromAxisAngle(Z_UNIT, -orientation) quaternion.multiply(rotation) } -
VRControls class VRControls { constructor (camera, vrDisplay) { this.camera =
camera this._vrDisplay = vrDisplay } update (options) { const pose = options.frameData ? options.frameData.pose : null if (pose) { if (pose.position) { this.camera.position.fromArray(pose.position) } if (pose.orientation) { this.camera.quaternion.fromArray(pose.orientation) } } } } -
GamepadControls listenGamepadEvents () { const gamepads = getGamepads() // navigator.getGamepads
for (let i = 0; i < gamepads.length; i += 1) { if (gamepads[i]) { const buttons = gamepads[i].buttons for (let j = 0; j < buttons.length; j += 1) { if (isPressed(buttons[j])) { this.batchedEvents.push({ type: 'GamepadEvent', eventType: 'keydown', button: buttons[j], gamepad: i }) } } } } } -
Check for box hit hit (origin, direction) { raycaster.set(origin, direction)
const intersects = raycaster.intersectObjects(this.root.children) for (let i = 0; i < intersects.length; i += 1) { const { object } = intersects[i] this.root.remove(object) } if (this.root.children.length === 0) { this.build() } } -
Crosshair const texture = new THREE.TextureLoader().load('images/crosshair.png') const geometry = new
THREE.PlaneGeometry( DEFAULT_CURSOR_WIDTH, DEFAULT_CURSOR_WIDTH ) const material = new THREE.MeshBasicMaterial({ transparent: true, side: THREE.DoubleSide, depthTest: false, depthWrite: false, map: texture }) this.mesh = new THREE.Mesh(geometry, material) this.mesh.raycast = () => null this.mesh.renderOrder = 1 -
Explosions const particleSettings = { texture: { value: textureLoader.load(smokeImage) },
depthTest: true, depthWrite: false, blending: THREE.NormalBlending, maxParticleCount: 1000 } const emitters = [ { particleCount: 600, type: SPE.distributions.SPHERE, position: { radius: 0.1 }, maxAge: { value: 0.5 }, activeMultiplier: 20, velocity: { value: new THREE.Vector3(1.2) }, size: { value: 1.5 }, opacity: { value: [0.5, 0] } } ] // init particles initParticles () { this.particleGroup = new SPE.Group( particleSettings ) this.particleGroup.addPool(1, emitters, false) this.headGroup.add(this.particleGroup.mesh) } // play sound and show smoke triggerSmoke () { player.play('shot') this.particleGroup.triggerPoolEmitter( 1, smokePosition ) } I used Shader Particle Engine -
Sounds class SoundPlayer { constructor () { this.context = new
(window.AudioContext || window.webkitAudioContext)() this.bufferLoader = new BufferLoader( this.context, sounds, this.handleLoad ) this.loaded = false this.bufferLoader.load() } handleLoad = () => { this.loaded = true } play (soundName) { if (this.loaded) { const source = this.context.createBufferSource() source.buffer = this.bufferLoader.bufferList[soundName] source.connect(this.context.destination) source.start(0) } } } -
…and allow to enter VR mode VRDisplay API provides all
needed methods and properties to serve your scene in VR compatible format • eyes position - you need to slice your scene in half • isPresenting flag - check if you can serve VR frames • frameData - includes device position, etc. • submitFrame - remember to call it when your frame is prepared Check three.js VREffect interface for more details -
How you can extend it • Use any WebVR framework
(A-frame or ReactVR) - the farther you go the harder it gets to maintain all the things • Add any UI to allow user to track their progress • Add some physics - make the wall collapse • Add support for 3DoF and 6DoF controller - yes, you can do it in your browser! -
Further reading 1/2 • The User Experience of Virtual Reality
• Virtual Reality Basics • VR for UX Designers: What I Learned During My First Project • History Of Virtual Reality • The very real health dangers of virtual reality • The Story of VR - A Look at the History Behind Virtual Reality • Design Practices in Virtual Reality • The Future of Virtual Reality • Designing Screen Interfaces for VR (video) -
Further reading 2/2 • WebVR concepts • Introduction to VR
Web • Building Virtual Reality on the Web with WebVR (video) • Using the WebVR API • Using the Gamepad API • Using VR controllers with WebVR • Getting started with three.js • Web Audio API