Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Getting Started with WebAR for HoloLens2 and Me...

Getting Started with WebAR for HoloLens2 and Meta Quest

TakashiYoshinaga

August 13, 2021
Tweet

More Decks by TakashiYoshinaga

Other Decks in Technology

Transcript

  1. Goal Let's try hand tracking using WebXR on HoloLens2 and

    Oculus Quest. https://youtu.be/y4WD2wgzKYQ
  2. Letʼs Try You can try demo if setting to use

    WebXR is finished on your device. https://webxr-hand-drawing.glitch.me
  3. Change the Content Name (URL) Click on the content name

    on the upper left corner of the screen. Click here
  4. Change the Content Name (URL) Change each string to a

    name that is easier to understand. Change to a simple name
  5. Check the URL Confirm the URL by clicking [Change URL]

    on the top of the preview screen. Change URL URL
  6. Check on the Device Enter the URL AR Button Enter

    the URL VR Button HoloLens2 Quest
  7. Tutorial Procedures 1.Explain the overview of the sample. 2.Add hand

    tracking feature. 3.Prepare work with hand tracking by javascript. 4.Describe the action of the right hand. a) Spawn an object when pinching starts. b) Spawn objects to the trajectory of the hand. c) Set the color of the next object to be added. when the pinch ends. 5.Describe the action of the left hand. a) Delete all objects at the start of pinch 6.Extra a) Always track the position of the index finger
  8. Tutorial Procedures 1.Explain the overview of the sample. 2.Add hand

    tracking feature. 3.Prepare work with hand tracking by javascript. 4.Describe the action of the right hand. a) Spawn an object when pinching starts. b) Spawn objects to the trajectory of the hand. c) Set the color of the next object to be added. when the pinch ends. 5.Describe the action of the left hand. a) Delete all objects at the start of pinch 6.Extra a) Always track the position of the index finger
  9. Overview of the Sample <a-scene background="color: #ECECEC"> <a-box position="0 1.5

    -2" scale="0.5 0.5 0.5" rotation="0 45 0" color="#4CC3D9" shadow></a-box> </a-scene> a-box position 0 1.5 -2
  10. Overview of the Sample <script> window.onload = function() { initRightHand();

    //右⼿の初期化 initLeftHand(); //左⼿の初期化 }; //右⼿の初期化を⾏う関数 function initRightHand(){ }; //左⼿の初期化を⾏う関数 function initLeftHand(){ }; </script> Automatically called once the page has loaded. Function prepared to describe the behavior of the right hand. Function prepared to describe the behavior of the left hand.
  11. Tutorial Procedures 1.Explain the overview of the sample. 2.Add hand

    tracking feature. 3.Prepare work with hand tracking by javascript. 4.Describe the action of the right hand. a) Spawn an object when pinching starts. b) Spawn objects to the trajectory of the hand. c) Set the color of the next object to be added. when the pinch ends. 5.Describe the action of the left hand. a) Delete all objects at the start of pinch 6.Extra a) Always track the position of the index finger
  12. Show Hand Objects <a-scene background="color: #ECECEC"> <a-box position="0 1.5 -2"

    scale="0.5 0.5 0.5" rotation="0 45 0" color="#4CC3D9" shadow></a-box> <!-- Drawing the left hand --> <a-entity id="leftHand" hand-tracking-controls="hand: left; modelStyle: dots;"></a-entity> <!-- Drawing the rihgt hand --> <a-entity id="rightHand" hand-tracking-controls="hand: right; modelStyle: dots;"></a-entity> </a-scene> 【hand-tracking-controls】 hand: left or right modelStyle: mesh pr dots (mesh doesnʼt work) modelColor: color of hand (Default is white) Lesson02
  13. Tutorial Procedures 1.Explain the overview of the sample. 2.Add hand

    tracking feature. 3.Prepare work with hand tracking by javascript. 4.Describe the action of the right hand. a) Spawn an object when pinching starts. b) Spawn objects to the trajectory of the hand. c) Set the color of the next object to be added. when the pinch ends. 5.Describe the action of the left hand. a) Delete all objects at the start of pinch 6.Extra a) Always track the position of the index finger
  14. Prepare Work with Hand Tracking <script> window.onload = function() {

    initRightHand(); //右⼿の初期化 initLeftHand(); //左⼿の初期化 }; //右⼿の初期化を⾏う関数 function initRightHand(){ }; //左⼿の初期化を⾏う関数 function initLeftHand(){ }; </script> Add variables here
  15. Prepare Work with Hand Tracking var scene; //3D space to

    add 3D objects var rightHand; //Variable to access the right hand. var leftHand; //Variable to access the left hand. window.onload = function() { initRightHand(); //右⼿の初期化 initLeftHand(); //左⼿の初期化 }; //右⼿の初期化を⾏う関数 function initRightHand(){ }; //左⼿の初期化を⾏う関数 function initLeftHand(){ }; Lesson03
  16. Prepare Work with Hand Tracking var scene; //3D space to

    add 3D objects var rightHand; //Variable to access the right hand var leftHand; //Variable to access the left hand window.onload = function() { // Keeps information in 3D space as variables. scene = document.querySelector('a-scene'); initRightHand(); //右⼿の初期化 initLeftHand(); //左⼿の初期化 }; //右⼿の初期化を⾏う関数 function initRightHand(){ }; //左⼿の初期化を⾏う関数 function initLeftHand(){ }; Lesson04
  17. Tutorial Procedures 1.Explain the overview of the sample. 2.Add hand

    tracking feature. 3.Prepare work with hand tracking by javascript. 4.Describe the action of the right hand. a) Spawn an object when pinching starts. b) Spawn objects to the trajectory of the hand. c) Set the color of the next object to be added. when the pinch ends. 5.Describe the action of the left hand. a) Delete all objects at the start of pinch 6.Extra a) Always track the position of the index finger
  18. Create Object by Right Hand Operation var scene; //描画する3D空間 var

    rightHand; //右⼿ var leftHand; //左⼿ window.onload = function() { //3D空間を変数として保持 scene = document.querySelector('a-scene'); initRightHand(); //右⼿の初期化 initLeftHand(); //左⼿の初期化 }; //右⼿の初期化を⾏う関数 function initRightHand(){ }; //左⼿の初期化を⾏う関数 function initLeftHand(){ }; Edit here
  19. Create Object by Right Hand Operation function initRightHand(){ //Get the

    right hand object (id=rightHand) rightHand = document.getElementById("rightHand"); //Begin pinch rightHand.addEventListener('pinchstarted', function (e) { }); //End pinch rightHand.addEventListener('pinchended', function (e) { }); //Move hand while pinching rightHand.addEventListener('pinchmoved', function (e) { }); }; Lesson05
  20. Getting the Pinch Start function initRightHand(){ //Get the right hand

    object (id=rightHand) rightHand = document.getElementById("rightHand"); //Begin pinch rightHand.addEventListener('pinchstarted', function (e) { }); //End pinch rightHand.addEventListener('pinchended', function (e) { }); // Move hand while pinching rightHand.addEventListener('pinchmoved', function (e) { }); }; Edit here
  21. Getting the Pinch Start function initRightHand(){ // Get the right

    hand object (id=rightHand) rightHand = document.getElementById("rightHand"); // Begin pinch rightHand.addEventListener('pinchstarted', function (e) { var position = e.detail.position; //Calling the addBOX function (explained in the next page) addBox(position); }); // End pinch rightHand.addEventListener('pinchended', function (e) { }); // Move hand while pinching rightHand.addEventListener('pinchmoved', function (e) { }); }; Lesson06
  22. Generating Box Object //左⼿の初期化を⾏う関数 function initLeftHand(){ }; function addBox(position){ //Create

    an element named a-box var box = document.createElement('a-box'); // Set the position, size, and color. box.setAttribute('position', position); box.setAttribute('scale', '0.02 0.02 0.02'); box.setAttribute('color', '#0062C6'); // Set class names for easy deletion later. box.setAttribute('class', 'box'); // Add a box in 3D space. scene.appendChild(box); } </script> Lesson07
  23. Tutorial Procedures 1.Explain the overview of the sample. 2.Add hand

    tracking feature. 3.Prepare work with hand tracking by javascript. 4.Describe the action of the right hand. a) Spawn an object when pinching starts. b) Spawn objects to the trajectory of the hand. c) Set the color of the next object to be added. when the pinch ends. 5.Describe the action of the left hand. a) Delete all objects at the start of pinch 6.Extra a) Always track the position of the index finger
  24. Follows the Movement of the Hand function initRightHand(){ // Get

    the right hand object (id=rightHand) rightHand = document.getElementById("rightHand"); // Begin pinch rightHand.addEventListener('pinchstarted', function (e) { /*omitted*/ }); // End pinch rightHand.addEventListener('pinchended', function (e) { }); // Move hand while pinching rightHand.addEventListener('pinchmoved', function (e) { }); }; Edit here
  25. Follows the Movement of the Hand // Get the right

    hand object (id=rightHand) rightHand = document.getElementById("rightHand"); // Begin pinch rightHand.addEventListener(ʻpinchstartedʼ, function (e) { /*omitted*/ }); // End pinch rightHand.addEventListener('pinchended', function (e) { }); // Move hand while pinching rightHand.addEventListener('pinchmoved', function (e) { var position = e.detail.position; //generate a box addBox(position); }); Lesson08
  26. Reduce the Density of Boxes <script> var scene; var rightHand;

    var leftHand; var prev;//To store the position where the last box was placed. window.onload = function() { //3D空間を変数として保持 scene = document.querySelector('a-scene'); initRightHand(); initLeftHand(); }; //右⼿の初期化を⾏う関数 function initRightHand(){ /*以下省略*/ Lesson09
  27. Reduce the Density of Boxes var prev;//最後にBox置いた位置を記憶 window.onload = function()

    { /*omitted*/ }; //右⼿の初期化を⾏う関数 function initRightHand(){ /*omitted*/ }; //左⼿の初期化を⾏う関数 function initLeftHand(){ }; //Function to add box function addBox(position){ } Edit here
  28. Reduce the Density of Boxes function addBox(position){ // Create an

    element named a-box var box = document.createElement('a-box'); // Set the position, size, and color box.setAttribute('position', position); box.setAttribute('scale', '0.02 0.02 0.02'); box.setAttribute('color', '#0062C6'); // Set class names for easy deletion later box.setAttribute('class', 'box'); //Add a box in 3D space. scene.appendChild(box); // Memorize the position where the box was placed. prev=box.object3D.position; } Lesson09
  29. Reduce the Density of Boxes var prev;//最後にBox置いた位置を記憶 window.onload = function()

    { /*omitted*/ }; //右⼿の初期化を⾏う関数 function initRightHand(){ /*omitted*/ }; //左⼿の初期化を⾏う関数 function initLeftHand(){ }; //Boxの追加を⾏う関数 function addBox(position){ } Edit here
  30. Reduce the Density of Boxes function initRightHand(){ //Get the right

    hand object (id=rightHand) rightHand = document.getElementById("rightHand"); // Begin pinch rightHand.addEventListener('pinchstarted', function (e) { /*omitted*/ }); // End pinch rightHand.addEventListener('pinchended', function (e) { }); // Move hand while pinching rightHand.addEventListener('pinchmoved', function (e) { }); }; Edit here
  31. Reduce the Density of Boxes rightHand.addEventListener('pinchmoved', function (e) { var

    position = e.detail.position; // Distance between the last position of the box and // the current position of the hand var length= prev.distanceTo(position); if(length>=0.02){ //Generate a box addBox(position); } }); Lesson10 prev position 一定距離以上
  32. Tutorial Procedures 1.Explain the overview of the sample. 2.Add hand

    tracking feature. 3.Prepare work with hand tracking by javascript. 4.Describe the action of the right hand. a) Spawn an object when pinching starts. b) Spawn objects to the trajectory of the hand. c) Set the color of the next object to be added. when the pinch ends. 5.Describe the action of the left hand. a) Delete all objects at the start of pinch 6.Extra a) Always track the position of the index finger
  33. Switch Color of Box var scene; var rightHand; var leftHand;

    var prev; // Array of colors to assign to a cube var colors = ['#0062C6', '#00AB00', '#FFCF00', '#7F00FF', '#FFFFFF', '#FF2000', '#F52394', '#593110']; // Index of color array to use var index=0; window.onload = function() { //3D空間を変数として保持 scene = document.querySelector('a-scene'); initRightHand(); //右⼿の初期化 initLeftHand(); //左⼿の初期化 }; Lesson11
  34. Switch Color of Box function initRightHand(){ // Get the right

    hand object (id=rightHand) rightHand = document.getElementById("rightHand"); // Begin pinch rightHand.addEventListener('pinchstarted', function (e) { /*omitted*/ }); // End pinch rightHand.addEventListener('pinchended', function (e) { }); // Move hand while pinching rightHand.addEventListener('pinchmoved', function (e) { /*omitted*/ }); }; Edit here
  35. Switch Color of Box function initRightHand(){ // Get the right

    hand object (id=rightHand) rightHand = document.getElementById("rightHand"); // Begin pinch rightHand.addEventListener('pinchstarted', function (e) { var position = e.detail.position; //Generate a box addBox(position); }); // End pinch rightHand.addEventListener('pinchended', function (e) { // Add up the number of colors. index++; index%=colors.length; }); Lesson12
  36. Switch Color of Box window.onload = function() { /*省略*/ };

    //右⼿の初期化を⾏う関数 function initRightHand(){ /*省略*/ }; //左⼿の初期化を⾏う関数 function initLeftHand(){ }; //Function to generate a box function addBox(position){ } Edit here
  37. Switch Color of Box function addBox(position){ // Create an element

    named a-box var box = document.createElement('a-box'); // Set the position, size, and color box.setAttribute('position', position); box.setAttribute('scale', '0.02 0.02 0.02ʼ); box.setAttribute('color', '#0062C6'); box.setAttribute('color', colors[index]); // Set class names for easy deletion later box.setAttribute('class', 'box'); // Add a box in 3D space scene.appendChild(box); // Memorize the position where the box was placed prev=box.object3D.position; } Lesson13 box.setAttribute('color', '#0062C6');
  38. Tutorial Procedures 1.Explain the overview of the sample. 2.Add hand

    tracking feature. 3.Prepare work with hand tracking by javascript. 4.Describe the action of the right hand. a) Spawn an object when pinching starts. b) Spawn objects to the trajectory of the hand. c) Set the color of the next object to be added. when the pinch ends. 5.Describe the action of the left hand. a) Delete all objects at the start of pinch 6.Extra a) Always track the position of the index finger
  39. Delete All Boxes var prev;//最後にBox置いた位置を記憶 window.onload = function() { /*omitted*/

    }; //右⼿の初期化を⾏う関数 function initRightHand(){ /* omitted */ }; //左⼿の初期化を⾏う関数 function initLeftHand(){ }; //Function to generate a box function addBox(position){ /*omitted */ } Edit here
  40. Delete All Boxes function initLeftHand(){ // Get the left hand

    object. leftHand = document.getElementById("leftHand"); // Behavior at the start of a pinch leftHand.addEventListener('pinchstarted', function (e) { // Get all objects to which the box class is assigned. var els = document.querySelectorAll('.boxʼ); // Delete the boxes one by one. for (var i = 0; i < els.length; i++) { els[i].parentNode.removeChild(els[i]); } }); }; Lesson14
  41. Tutorial Procedures 1.Explain the overview of the sample. 2.Add hand

    tracking feature. 3.Prepare work with hand tracking by javascript. 4.Describe the action of the right hand. a) Spawn an object when pinching starts. b) Spawn objects to the trajectory of the hand. c) Set the color of the next object to be added. when the pinch ends. 5.Describe the action of the left hand. a) Delete all objects at the start of pinch 6.Extra a) Always track the position of the index finger
  42. Getting the Position of Finger Tip <a-scene background="color: #ECECEC"> <a-box

    position="0 1.5 -2" scale="0.5 0.5 0.5" rotation="0 45 0" color="#4CC3D9" shadow></a-box> <a-entity id="leftHand" hand-tracking-controls="hand: left; modelStyle: dots;"></a-entity> <a-entity id="rightHand" hand-tracking-controls="hand: right; modelStyle: dots;"></a-entity> <!-- Sphere to be displayed at fingertip --> <a-sphere id="tip" position="0 0 0" radius="0.015" color="#0062C6"></a-sphere> </a-scene> Lesson15
  43. Getting the Position of Finger Tip // Array of colors

    to assign to a cube var colors = ['#0062C6', '#00AB00', '#FFCF00', '#7F00FF', '#FFFFFF', '#FF2000', '#F52394', '#593110']; // Index of color array to use var index=0; //Sphere representing the tip of the finger var tip; window.onload = function() { //3D空間を変数として保持 scene = document.querySelector('a-scene'); //Get the sphere representing the fingertip position tip = document.getElementById("tip"); initRightHand(); //右⼿の初期化 initLeftHand(); //左⼿の初期化 }; Lesson16
  44. Getting the Position of Finger Tip var prev;//最後にBox置いた位置を記憶 window.onload =

    function() { /*省略*/ }; //右⼿の初期化を⾏う関数 function initRightHand(){ /*omitted*/ }; //左⼿の初期化を⾏う関数 function initLeftHand(){ /*省略*/ }; //Boxの追加を⾏う関数 function addBox(position){ /*省略*/ } Edit here
  45. Getting the Position of Finger Tip function initRightHand(){ // Get

    the right hand object (id=rightHand) rightHand = document.getElementById("rightHand"); // Track finger tips in real time (to be implemented later) tipTracking(rightHand.components['hand-tracking-controls']); // Begin pinch rightHand.addEventListener('pinchstarted', function (e) { /*omitted*/ }); // End pinch rightHand.addEventListener('pinchended', function (e) { /*omitted*/ }); Lesson17
  46. Getting the Position of Finger Tip window.onload = function() {

    /*omitted*/ }; function initRightHand(){ /* omitted(右⼿の初期化を⾏う関数)*/ }; function initLeftHand(){ /* omitted(左⼿の初期化を⾏う関数)*/ }; function addBox(position){ /* omitted(Boxの追加を⾏う関数)*/ } Add a function for tracking finger position here
  47. Getting the Position of Finger Tip function initLeftHand(){ /*omitted(左⼿の初期化を⾏う関数)*/ };

    function addBox(position){ /*omitted(Boxの追加を⾏う関数)*/ } function tipTracking(ctrl){ // Finger tip position var position = ctrl.indexTipPosition; // Assign finger position to the ball tip.setAttribute('position', position); // Call tipTracking again after 30ms setTimeout(function(){tipTracking(ctrl);}, 30); }; Lesson18
  48. Assign the Box Color to the Fingertip function initRightHand(){ //右⼿のオブジェクト(id=rightHand)を取得

    rightHand = document.getElementById("rightHand"); //ピンチ開始 rightHand.addEventListener('pinchstarted', function (e) { /*中略*/ }); //End pinch rightHand.addEventListener('pinchended', function (e) { }); //ピンチしながら⼿を移動 rightHand.addEventListener('pinchmoved', function (e) { /*中略*/ }); }; Edit here
  49. 次のBoxの⾊を指先の球に反映 // Begin pinch rightHand.addEventListener('pinchstarted', function (e) { var position

    = e.detail.position; //BOX⽣成関数を呼び出す addBox(position); }); // End pinch rightHand.addEventListener('pinchended', function (e) { //⾊の番号を加算してboxの⾊に反映 index++; index%=colors.length; // Assing the color to the ball at the tip of the finger tip.setAttribute('color', colors[index]); }); Lesson19