Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Machine Learning for HCI @ NTU CSIE, 2013/7/21

Machine Learning for HCI @ NTU CSIE, 2013/7/21

Johnson Liang

July 21, 2013
Tweet

More Decks by Johnson Liang

Other Decks in Education

Transcript

  1. ML in Projects iRotateGrasp by Xman 龍哥 SenSleep by Jimmyken

    RingTune (Final project for the course Data Mining)
  2. What’s Common in Those Projects? They collects input data. iRotateGrasp:

    44 capacitive sensor values. SenSleep: mobile & PC activities. Ringtune: Ambient sound, accelerations, light.
  3. What’s Common in Those Projects? (2) The data is used

    to determine an output. iRotateGrasp: screen orientation. SenSleep: if the user is sleeping in a time slot. Ringtune: the desired ringer volume (0~7)
  4. What’s Common in Those Projects? (2) The data is used

    to determine an output. iRotateGrasp: screen orientation. SenSleep: if the user is sleeping in a time slot. Ringtune: the desired ringer volume (0~7)
  5. The Core of Decision Making Classifier 分類器 Given: <input, output>

    pairs Goal: Given any inputs, predict the outputs.
  6. Training & Testing a Classifier Training Learns from data <input,

    output> <input, output> <input, output> <input, output> <input, output> <input, output> <input, output> <input, output> <input, output> <input, output> ...... Testing Ask for output <input, ?>
  7. 44 sensors < s1, s2, 14, ......, s44 > &

    output 44-value input +output 
  8. 44 sensors < s1, s2, 14, ......, s44 > &

    output 44-value input +output LIBSVM Classifier   Chih-Chung Chang and Chih-Jen Lin, LIBSVM : a library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2:27:1--27:27, 2011
  9. 44 sensors < s1, s2, 14, ......, s44 > &

    output 44-value input +output LIBSVM Classifier   Chih-Chung Chang and Chih-Jen Lin, LIBSVM : a library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2:27:1--27:27, 2011
  10. LIBSVM Data Collection Session Stand Sit Lie down Lie down

    (side) Left hand Right hand Both hands ❌ ❌ 
  11. LIBSVM Data Collection Session Stand Sit Lie down Lie down

    (side) Left hand Right hand Both hands ❌ ❌ ❌ 
  12. LIBSVM Data Collection Session Stand Sit Lie down Lie down

    (side) Left hand Right hand Both hands ❌ ❌ ❌  162,000 samples
  13. SenSleep Data Collection Mobile Screen Lock/Unlock events Accelerometer values Battery

    charging events Light sensor values Current location change events System-defined broadcast events Desktop PC Idle Intervals 12 participants, 7 days
  14. That’s A Lot to Learn From! Too much raw data

    Requires lots of training data to conclude!
  15. That’s A Lot to Learn From! Too much raw data

    Requires lots of training data to conclude! What really matters? “Features” Features should describe our data better.
  16. SenSleep Features Screen on/off (0 or 1) Elapsed time since

    screen on/off Battery charging on/off Elapsed time since last battery event Current coordinate (location) Offset in location, compared to 15 min before Accelerometer average values Accelerometer median values Elapsed time since last PC keyboard / mouse activity <f1, ..., f9>
  17. SenSleep Features Screen on/off (0 or 1) Elapsed time since

    screen on/off Battery charging on/off Elapsed time since last battery event Current coordinate (location) Offset in location, compared to 15 min before Accelerometer average values Accelerometer median values Elapsed time since last PC keyboard / mouse activity <f1, ..., f9> 9-dimensional feature vector
  18. History-Related Feature < f1, ..., f9 > < last_is_sleeping, f1,

    ..., f9 > < last_f1, ..., last_f9, f1, ..., f9 >
  19. ‹ › Classifier Implementation » avg_x, avg_y, avg_z, var_x, var_y,

    var_z, avg_dx, avg_dy, avg_dz, light, close 11D feature vector & Volume
  20. Phone Localization Martin et al. MobiCom '09, Duke University SurroundSense:

    mobile phone localization via ambience fingerprinting
  21. Phone Localization Martin et al. MobiCom '09, Duke University SurroundSense:

    mobile phone localization via ambience fingerprinting Sound
  22. Phone Localization Martin et al. MobiCom '09, Duke University SurroundSense:

    mobile phone localization via ambience fingerprinting Sound Color of Light
  23. Phone Localization Martin et al. MobiCom '09, Duke University SurroundSense:

    mobile phone localization via ambience fingerprinting Sound Color of Light Motion
  24. Sound Feature Waveform Waveform (Zoomed to samples) 1 0 -1

    Martin et al. MobiCom '09, Duke University SurroundSense: mobile phone localization via ambience fingerprinting
  25. Color Feature Martin et al. MobiCom '09, Duke University SurroundSense:

    mobile phone localization via ambience fingerprinting
  26. Motion Feature Moving Static Feature: moving average & variance of

    instaneous acceleration Martin et al. MobiCom '09, Duke University SurroundSense: mobile phone localization via ambience fingerprinting
  27. Sensing Grip Pattern Determine On Table / In Hand Thumb

    / Index Finger Left / Right Thumb Pressure Mayank et al. UIST '12, University of Washington GripSense: using built-in sensors to detect hand posture and pressure on commodity mobile phones
  28. Actual Application in Use Hailpern et al. DIS '10, University

    of Illinois at Urbana Champaign The CLOTHO Project: Predicting Application Utility 程式啟動 / 結束 視窗切換 登入登出 開機關機 開機關機 某程式 CPU 用量 某程式 RAM 用量 視窗 z-buffer 桌面大小 視窗大小 視窗座標 視窗可見範圍 Focused App 滑鼠位置 Timestamp 真正在使用的 「重要的程式」 High-utilization Application 當前系統快照 System Snapshot
  29. Document Classification Bag of words John likes to watch movies.

    Mary likes too. John also likes to watch football games.
  30. Document Classification Bag of words John likes to watch movies.

    Mary likes too. John also likes to watch football games. Dictionary John, likes, tp, watch, movies, also, football, games, Mary, too
  31. Document Classification Bag of words John likes to watch movies.

    Mary likes too. John also likes to watch football games. Dictionary John, likes, tp, watch, movies, also, football, games, Mary, too <1, 2, 1, 1, 1, 0, 0, 0, 1, 1> <1, 1, 1, 1, 0, 1, 1, 1, 0, 0> (Multinominal, counts occurrence)
  32. Document Classification Bag of words John likes to watch movies.

    Mary likes too. John also likes to watch football games. Dictionary John, likes, to, watch, movies, also, football, games, Mary, too <1, 1, 1, 1, 1, 0, 0, 0, 1, 1> <1, 1, 1, 1, 0, 1, 1, 1, 0, 0> (Bernoulli, present or not)
  33. Learning Curve Mayank et al. UIST '12, University of Washington

    GripSense: using built-in sensors to detect hand posture and pressure on commodity mobile phones
  34. Numerical Function Numeric input & numeric output. Categorical Output -

    Discrete “type” labels Continuous Output - Real values
  35. PJ Cheng, Text Categorization, 2013 Web IR Slides Choose a

    distance metric to calculate the distances between feature vectors
  36. ⼀一棵樹分兩邊:supervised, unsupervised 「classifier」/「cluster」 SD Lin, Final Mark on Machine Leaerning,

    2013 PGM Slides 剛剛教的 幾乎在這 HMM 在這 AI 會教 Clustering 分群
  37. 技能樹 Starter Class, 必修 (?) 鄭卜壬 網路資訊檢索與探勘 下學期 修完必修之後會偏涼: 陳銘憲(@EE)

    資訊勘測 上學期 陳信希 自然語言處理 上學期 李琳山 數位語音處理概論 下學期 會用到但沒太大關係的課 于天立(EE) / 許永真 人工智慧 上學期 徐宏民 多媒體資訊分析與檢 索 上學期 前往真理前要打倒的大魔王 林軒田 機器學習 上學期 林守德 機率圖形學習模型 上學期 我很猛想比賽 林智仁 機器學習理論與實務 下學期
  38. Weka The University of Waikato The WEKA Data Mining Software:

    An Update, 2009 In Java Can be put in Android GUI ! Multiple algorithms implemented Unified input / output format
  39. LIBSVM 林智仁老師 LIBSVM: A library for support vector machines, 2011

    In multiple Languages Can be put in Android & iOS ! Simple install (just make!) Simple input / output format Tutorial: http://www.csie.ntu.edu.tw/~piaip/docs/svm/
  40. LIBSVM Output format Model file Prediction file One label per

    line Confidence attached if a flag is set.
  41. 217 Train Stations Usually running grid.py Finding optimal c (cost)

    and g (gamma) SSH authorized_keys setup (google SSH 免密碼) ssh_workers & nr_local_worker 讀 svm 資料夾/tools/README