Familiarisation (IUI 2018, Tokyo, Japan)

Slides from the talk on "Familiarisation: Restructuring Layouts with Visual Learning Models" given at ACM IUI 2018, in Tokyo, Japan.

Transcript

1. Restructuring Visual Layouts with Learning Kashyap Todi Jussi Jokinen Kris Luyten Antti Oulasvirta Familiarisation Models
2. Familiariser New/Unvisited Page Familiarised Page
3. page designs can be quite diverse
4. Frequently visited pages
5. New/Unvisited Frequently visited pages
7. increased visual search time reduces performace/usability
8. New/Unvisited Page Familiarised Page Familiariser
9. Familiariser User History History Logging
10. Visit Page User History History Logging Familiariser
11. User History Familiariser History Logging
12. User History Visit Page Familiariser History Logging
13. User History Familiariser History Logging
14. User History Visit Page Familiariser History Logging
15. Familiarity Models x y Time Frequency Page Frequency I. Frequency II. Serial-Position Curve III. Visual Statistical Learning IV. Cognitive Model History Logging Familiarity Modelling Familiariser
16. History Logging Familiarity Modelling FAMILIAR DESIGN Familiariser
17. New/Unvisited Page History Logging Familiarity Modelling FAMILIAR DESIGN Familiariser
18. New/Unvisited Page Familiarised Page History Logging Familiarity Modelling Page Restructuring FAMILIAR DESIGN Familiariser
19. Layout Generation Run-time Adaptations
20. Layout Generation DesignScape [CHI ’15] ….. Sketchplore [DIS ’17] and many more generate/optimise designs for the general population
21. Run-time Adaptations Split / Moving / Visual Popout Interface [CHI ’11] Walking User Interface [TACCESS ’11] ….. Actively adapt known interfaces as they are being used
22. Familiarisation models per-user familiarity to generate a familiar design, and uses this to adapt unvisited designs by restructuring
23. Familiarisation History Logging Familiarity Modelling Page Restructuring
24. History Logging Parse for Features
25. History Logging User History • Order of page visits • Parsed page layouts • Visit frequency, duration, etc
26. History Logging Familiarity Modelling Page Restructuring Familiarisation
27. Familiarity Modelling I. Frequency-Based II. Serial-Position Curve III. Visual Statistical Learning IV. Cognitive Model
28. I. FREQUENCY-BASED Page Frequency
29. I. FREQUENCY-BASED Page with highest visit frequency selected fpage = nvisits ∗ taverage
30. II. SERIAL-POSITION CURVE Time Frequency
31. II. SERIAL-POSITION CURVE Page selected based on function of long-term recall Frequency Recency Primacy npage / ntotal 1 – rpage / rtotal 1−((ppage −1)/npages)
32. III. VISUAL-STATISTICAL LEARNIng x y
33. III. VISUAL-STATISTICAL LEARNIng Page generated based on feature distributions Logo Search Social Menu
34. IV. COGNITIVE MODEL
35. IV. COGNITIVE MODEL Page generated based on cognitive model of visual learning
36. Familiarity Modelling FAMILIAR DESIGN As a result, a familiar design is selected/generated
37. Familiarity Modelling Logo As a result, a familiar design is selected/generated Login Cart Search Newsletter Social Content
38. History Logging Familiarity Modelling Page Restructuring Familiarisation
39. Page Restructuring Bricolage [CHI ’11] — Requires element size information — Hard to deal with feature mismatches — Content might be cropped due to mismatch
40. Page Restructuring FAMILIAR DESIGN ORIGINAL PAGE + FAMILIARISED PAGE —>
41. Page Restructuring FAMILIAR DESIGN ORIGINAL PAGE Logo Login Cart Search Newsletter Social Content ORIGINAL PAGE FAMILIAR DESIGN
42. Page Restructuring FAMILIAR DESIGN ORIGINAL PAGE ORIGINAL PAGE RESTRUCTURED FAMILIAR DESIGN Page restructured but has overlaps!
43. Page Restructuring FAMILIAR DESIGN ORIGINAL PAGE ORIGINAL PAGE RESTRUCTURED FAMILIAR DESIGN Page restructured but has overlaps! Overlap Resolution LP solver with constraints to: • Eliminate overlaps • Align elements
44. Page Restructuring FAMILIAR DESIGN ORIGINAL PAGE ORIGINAL PAGE FAMILIARISED FAMILIAR DESIGN
45. An Example
46. User History
47. User History Unvisited Page
48. User History Unvisited Page Model Results Frequency-Based Serial-Position Curve Visual Statistical Learning Cognitive Model
49. User Study Evaluating how familiarisation impacts visual search time during page browsing tasks
50. 1. Learning Phase 5 page designs 25 visits Original design displayed
51. 2. Test Phase 25 page designs 100 visits Original or familiarised design displayed
52. Visual search time reduced by 10% (2.8 vs 2.5 seconds) 23% fewer eye-gaze fixations (3.4 vs 2.6 fixations) Results =16
53. A principled approach to restructure new layouts using visual learning models
54. Designer Global Consistency Population
55. Designer Global Consistency Familiariser History Logging Familiarity Modelling Page Restructuring Local Consistency Population Individual
56. Familiarisation Kashyap Todi Jussi Jokinen Kris Luyten Antti Oulasvirta Unvisited Familiarised History Logging Familiarity Modelling Page Restructuring II. Serial-Position Curve I. Frequency-Based III. Visual Statistical Learning IV. Cognitive Model www.kashyaptodi.com/familiarisation