Upgrade to Pro — share decks privately, control downloads, hide ads and more …

IVILab. Research Introduction

IVILab. Research Introduction

Y. Sugano

May 17, 2024

Other Decks in Research


  1. • Computer vision and human-computer interaction ‣ How can we

    achieve intelligent interaction between humans and computers through videos/images? Our Research Interests 2 Machine Human HCI CV Vision-based Human Understanding Understanding people from images Human-in-the-loop Interactive ML Understanding media together with people 3D models Latent appearance vector 𝒛 𝒛1 𝒛2 𝒛3 𝒛𝟎 Input Normal map rendering Images generated by SCGAN User-Adaptive Image Generation Generating images that match the user's intentions Distractiveness Noticeability Gaze distance Size Blink Opacity Movement User-Adaptive UI/Visualization Presenting information with the user's intentions
  2. • Technique to measure where a person is looking at

    • Many applications ‣ As Measurement: advertisement and marketing ‣ As Input: gaze interaction, assistive technologies, attentive user interfaces, VR and e-sports ‣ As Feature: activity and intention recognition, medical diagnosis Eye Tracking and Gaze Estimation 3 https://www.youtube.com/watch?v=RpQVSmGvbMo
  3. • Estimating gaze directions from ordinary cameras (webcams, wearable cameras,

    … ) ‣ Calibration-free estimation from low-resolution images • Technically more challenging than other existing approaches Camera-based Gaze Estimation 4
  4. • Building techniques to adapt gaze estimation models to novel

    environments ‣ Use 3 D face reconstruction to synthesize training data for unseen head poses ‣ Train multi-view gaze estimator that can handle different camera configurations Adaptive Training for Gaze Estimation 5 Synthetic Training Data Multi-view Gaze Estimation {"!"# } {"$%$ } Training Test Multi-View Gaze Estimator {$} … Multi-View Gaze Estimation {"} … … … Single-View Gaze Estimation From novel camera pairs Single-View Gaze Estimator
  5. • Gamified workshop for democratizing gaze estimation research ‣ Provide

    game-like experience where users participate in and understand the data collection process ‣ Joint project with DLX design lab, UTokyo-IIS AICOM Project 6 https://vimeo.com/ 4 29 881 90 7
  6. • Real-world computer vision and machine learning applications with end-users

    in the loop CV Applications & Interactive Machine Learning 7 human: 27% natural: 24% ML workshop with novice users with disabilities Interactive Machine Learning
  7. • Public workshops as social implementation of research ‣ Analyze

    how the perception of technology and sound change through interactions with ML technologies Workshop Events using Interactive ML 8 ML Workshop at Science Museum