Use Cases and Course Review - Lecture 8 - Human-Computer Interaction (1023841ANR)

This lecture forms part of the course Human-Computer Interaction given at the Vrije Universiteit Brussel.

Transcript

1. 2 December 2005 Human-Computer Interaction Use Cases and Course Review Prof. Beat Signer Department of Computer Science Vrije Universiteit Brussel beatsigner.com Department of Computer Science Vrije Universiteit Brussel beatsigner.com
2. Beat Signer - Department of Computer Science - bsigner@vub.be 2 December 8, 2025 HCI Research Contributions Revisited ▪ Empirical contributions ▪ Artefact contributions (systems research) ▪ Methodological contributions ▪ Theoretical contributions ▪ Dataset contributions ▪ Literature survey contributions ▪ Opinion contributions
3. Beat Signer - Department of Computer Science - bsigner@vub.be 3 December 8, 2025 A screenshot of a computer AI-generated content may be incorrect.
4. Beat Signer - Department of Computer Science - bsigner@vub.be 4 December 8, 2025 Prof. Dr. Beat Signer Cross-Media Technology, Interac- tive Paper, Data Physicalisation Dr. Audrey Sanctorum User-defined XDI and IoT Inter- action, Human-AI Interaction CISA Human-Machine & Human-Information Interaction Information Systems & Management Information Visualisation & Navigation logo.wmf WEB INFORMATION SYSTEMS ENGINEERING WEB & INFORMATION SYSTEMS ENGINEERING Dr. Maxim Van de Wynckel Interoperable Positioning Systems, Implicit HCI Ingela Rossing Dynamic Data Physicalisation Framework and Guidelines CROSS-MEDIA INFORMATION SPACES AND ARCHITECTURES (CISA) CROSS-MEDIA INFORMATION SPACES AND ARCHITECTURES (CISA) A close-up of a person smiling Description automatically generated
5. Beat Signer - Department of Computer Science - bsigner@vub.be 5 December 8, 2025 logo.wmf WEB INFORMATION SYSTEMS ENGINEERING WEB & INFORMATION SYSTEMS ENGINEERING CISA Human-Machine & Human-Information Interaction Information Systems & Management Information Visualisation & Navigation CROSS-MEDIA INFORMATION SPACES AND ARCHITECTURES (CISA) CROSS-MEDIA INFORMATION SPACES AND ARCHITECTURES (CISA) Jan Boonants Mixed Reality Information Archi- tectures, Generic IoT Interfaces Bingyan Lee Next Generation User Interfaces for Human-Information Interaction Yoshi Malaise Technology-enhanced Learning, Content-driven Presentations Qi Xu Next Generation Scholarly Workflows, Cross-Media PIM A person with glasses and a white background Description automatically generated A person with long hair Description automatically generated with low confidence A person with a mustache and beard AI-generated content may be incorrect.
6. Beat Signer - Department of Computer Science - bsigner@vub.be 6 December 8, 2025 logo.wmf WEB INFORMATION SYSTEMS ENGINEERING WEB & INFORMATION SYSTEMS ENGINEERING CISA Human-Machine & Human-Information Interaction Information Systems & Management Information Visualisation & Navigation CROSS-MEDIA INFORMATION SPACES AND ARCHITECTURES (CISA) CROSS-MEDIA INFORMATION SPACES AND ARCHITECTURES (CISA) Piet Van Der Paelt Julia-based Framework for Simulation and Optimisation Ekene Attoh IoT Middleware, Context-aware Computing, Implicit HCI Xuyao Zhang Extensible Platform for Dynamic Data Physicalisation A person with a beard and glasses AI-generated content may be incorrect. Isaac Valadez Knowledge Physicalisation and Augmentation, Tangible UIs
7. Beat Signer - Department of Computer Science - bsigner@vub.be 7 December 8, 2025 logo.wmf WEB INFORMATION SYSTEMS ENGINEERING WEB & INFORMATION SYSTEMS ENGINEERING CISA Human-Machine & Human-Information Interaction Information Systems & Management Information Visualisation & Navigation CROSS-MEDIA INFORMATION SPACES AND ARCHITECTURES (CISA) CROSS-MEDIA INFORMATION SPACES AND ARCHITECTURES (CISA) Arun Sojan Physicalisation for Digital Twins Evan Cole Technology-enhanced Learning, Study Lenses A person wearing glasses Description automatically generated with medium confidence Patrick Sello Digital Twins for Healthcare, Federated Learning
8. Beat Signer - Department of Computer Science - bsigner@vub.be 8 December 8, 2025 Interactive Paper edfestComponents Global Information Systems Group ETH Zurich
9. Beat Signer - Department of Computer Science - bsigner@vub.be 9 December 8, 2025 Visual and Interaction Design ▪ Visualisation of interactive areas and functionality ▪ design patterns ▪ Interaction design ▪ online versus offline input processing ▪ lack of modal dialogues ▪ multimodal interaction ▪ Findings might be appli- cable to traditional GUIs ▪ e.g.cross-application widget interaction booklet 2004 booklet 2005 Global Information Systems Group, ETH Zurich
10. Beat Signer - Department of Computer Science - bsigner@vub.be 10 December 8, 2025 Pen-based Interaction A person drawing on a computer AI-generated content may be incorrect. ▪ Survey on pen-based interaction ▪ history ▪ pen-and-device technologies ▪ pen-and-paper technologies ▪ interface and interaction styles ▪ application domains ▪ challenges and future directions A person wearing a virtual reality headset AI-generated content may be incorrect. Microsoft Surface Studio 2 Massless Pen
11. Beat Signer - Department of Computer Science - bsigner@vub.be 11 December 8, 2025 iGesture Framework ▪ iGesture Workbench ▪ create/test gesture sets and algorithms ▪ Different modalities ▪ digital pen, tablet PC, mouse, Wii remote, … ▪ multimodal gestures ▪ Open source ▪ http://www.igesture.org gestureSets.jpg captureFormNew.jpg
12. Beat Signer - Department of Computer Science - bsigner@vub.be 12 December 8, 2025 Multimodal Interaction ▪ Midas/Mudra Framework ▪ declarative definition of multimodal and multi-touch interactions ▪ rule-based language approach ▪ rapid prototyping and application development Hoste et al., 2011 A screenshot of a computer AI-generated content may be incorrect.
13. Beat Signer - Department of Computer Science - bsigner@vub.be 13 December 8, 2025 edfestComponents 01001000100 0101001100010010100010010 0100100010010001000100001001000 100010001100001001000110010101010110 1100010010100010010010010100010101011001 101001100001100 101010100 0010100 100 110101010101 00100 01010101000 1001 1111111111 111 0000000010 0 11111111 0101001011 010101010 111010101 001010010110 splash.gif Fluid Cross-Media Information Spaces
14. Beat Signer - Department of Computer Science - bsigner@vub.be 14 December 8, 2025 RSL Hypermedia Metamodel iwebfirefox movieplugin ipaper Global Information Systems Group, ETH Zurich
15. Beat Signer - Department of Computer Science - bsigner@vub.be 15 December 8, 2025 What is Wrong with Slideware? ▪ Simulation of physical slides ▪ limited space due to the slide concept ▪ linear navigation from slide to slide ▪ difficult to reuse content and embed rich media types ▪ MindXpres addresses these issues ▪ unlimited canvas with zoomable user interface ▪ non-linear navigation and associative linking ▪ content-based approach with automatic visualisation
16. Beat Signer - Department of Computer Science - bsigner@vub.be 16 December 8, 2025 Presentation Solutions ▪ PaperPoint ▪ interactive PowerPoint paper handouts ▪ non-linear navigation and annotations ▪ MindXpres ▪ extensible presentation platform ▪ content-driven approach ▪ non-linear navigation via zoomable user interface ▪ associative linking ▪ rich media plug-ins
17. Beat Signer - Department of Computer Science - bsigner@vub.be 17 December 8, 2025 Interactive Source Code Plug-in Paul Mestereaga
18. Beat Signer - Department of Computer Science - bsigner@vub.be 18 December 8, 2025 MindXpres Video
19. Beat Signer - Department of Computer Science - bsigner@vub.be 19 December 8, 2025 Personal Information Management (PIM) ▪ Keeping, organising and re-finding information ▪ digital and physical ▪ Study of human- information interaction ▪ files, piles, mixtures, … ▪ OC2 PIM model ▪ based on RSL hypermedia metamodel ▪ Cross-Media PIM system ▪ explicit as well as implicit associations between entities
20. Beat Signer - Department of Computer Science - bsigner@vub.be 20 December 8, 2025 PimVis Document View Audrey Sanctorum
21. Beat Signer - Department of Computer Science - bsigner@vub.be 21 December 8, 2025 PimVis Setup
22. Beat Signer - Department of Computer Science - bsigner@vub.be 22 December 8, 2025 EUD of Cross-Device and IoT Applications ▪ Rapid prototyping platform for cross-device and IoT applications (eSPACE) ▪ End-user authoring ▪ customised distribution of user interface components ▪ mashup tool for digital and physical (IoT) components
23. Beat Signer - Department of Computer Science - bsigner@vub.be 23 December 8, 2025 Mixed Reality in IoT Environments ▪ Mixed-Reality-based authoring of IoT inter- actions ▪ Context Modelling Toolkit (CMT) ▪ OpenHPS hybrid positioning system ▪ stream-based processing ▪ interoperability ▪ discoverability Elena Zambon
24. Beat Signer - Department of Computer Science - bsigner@vub.be 24 December 8, 2025 Hypertext principles can play a major role in realising a general infrastructure for human- information interaction in mixed reality environments
25. Beat Signer - Department of Computer Science - bsigner@vub.be 25 December 8, 2025 General Infrastructure for Mixed Reality ▪ W3C WebXR Device API ▪ cross-platform compatibility ▪ web-based applications running in a browser ▪ W3C Web of Things (WoT) Thing Description (TD) ▪ metadata and interfaces of IoT devices & passive things ▪ Solid Project ▪ decentralised Solid Pods in addition to the Web ▪ collaborative authoring and sharing of information Solid
26. Beat Signer - Department of Computer Science - bsigner@vub.be 26 December 8, 2025 Tangible Holograms (TangHo) Timothy J. Curtin
27. Beat Signer - Department of Computer Science - bsigner@vub.be 27 December 8, 2025 TangHo Prototype ▪ Lego Mindstorms-based 6DOF arm prototype ▪ motors can be locked or set to float mode - useful for bidirectional I/O ▪ replaceable end effectors for different types of non-visual feedback ▪ Challenges ▪ inverse kinematics (final system is body mounted) - use robotics toolbox (MATLAB) ▪ limited HoloLens hand tracking ▪ 3D printing of final arm Timothy J. Curtin
28. Beat Signer - Department of Computer Science - bsigner@vub.be 28 December 8, 2025 TangHo Interactions
29. Beat Signer - Department of Computer Science - bsigner@vub.be 29 December 8, 2025 Dynamic Data Physicalisation ▪ Understanding the design space ▪ how do users map data to physicalisations ▪ Data physicalisation design guidelines ▪ what are the just-perceptual differences that a physical variable can convey? ▪ Dynamic data physicalisation framework ▪ data physicalisation grammar ▪ data processing with device-independent output ▪ software drivers for different physicalisations (e.g. TangHo) ▪ New application domains Jacques Bertin, 1967
30. Beat Signer - Department of Computer Science - bsigner@vub.be 30 December 8, 2025 Dynamic Data Physicalisation Framework ...
31. Beat Signer - Department of Computer Science - bsigner@vub.be 31 December 8, 2025 Course Summary 1. Introduction ▪ HCI and Interaction Design ▪ history of human-computer interaction - analogue computers, oNLine System (NLS) and desktop computers ▪ “evolution” of interfaces ▪ interface types - command-based, WIMP, pen, touch, speech, gesture, VR, AR, multimodal, … 2. HCI and Interaction Design ▪ what to design - who (user), how (task) and where (context) ▪ usability goals - effectiveness, efficiency, safety, utility, learnability, memorability ▪ user experience (UX) goals
32. Beat Signer - Department of Computer Science - bsigner@vub.be 32 December 8, 2025 Course Summary … HCI and Interaction Design … ▪ Double Diamond Design Process ▪ user-centred design principles ▪ Interaction Design Lifecycle Model - establishing requirements, designing alternatives, prototyping and evaluating 3. Requirements Analysis and Prototyping ▪ types of requirements - functional, data, environmental, user characteristics, usability, user experience ▪ requirements gathering techniques - interviews, focus groups, surveys, direct and indirect observation, … ▪ task description - scenarios and use cases ▪ prototyping - horizontal vs. vertical, throwaway vs. evolutionary
33. Beat Signer - Department of Computer Science - bsigner@vub.be 33 December 8, 2025 Course Summary … 4. Human Perception and Cognition ▪ human senses and modalities ▪ model of visual perception processing ▪ anatomy of the human eye - lens, pupil, retina (rods and cones), … ▪ brightness perception and illusions ▪ visual field and peripheral vision ▪ perception - biased by past, present and future - designing for biased perception ▪ Gestalt principles - law of proximity, law of similarity, law of continuity, law of closure, … ▪ long-term memory and immediate memory
34. Beat Signer - Department of Computer Science - bsigner@vub.be 34 December 8, 2025 Course Summary … 5. Design Guidelines and Models ▪ Shneiderman’s eight golden rules of interfaces design ▪ Nielsen’s ten usability heuristics for UI design ▪ visibility, simplicity and mapping ▪ guidelines for using colours ▪ inattentional /change blindness ▪ affordances ▪ Fitts’s law
35. Beat Signer - Department of Computer Science - bsigner@vub.be 35 December 8, 2025 Course Summary … 6. Evaluation Methods ▪ why, what, where and when? ▪ DECIDE evaluation framework - goals, questions, methods, practical issues, ethical issues, evaluate/present ▪ usability testing ▪ experimental design - between subjects, within-subjects and pair-wise design ▪ UEQ+ user experience questionnaire ▪ inspection - heuristic evaluation and walkthroughs ▪ analytics - Google analytics, A/B testing, … ▪ predictive models - GOMS model, keystroke level model and Fitts’s law
36. Beat Signer - Department of Computer Science - bsigner@vub.be 36 December 8, 2025 Course Summary … 7. HCI Research Methods ▪ seven types of HCI research contributions - empirical contributions, artefact contributions (systems research), methodological contributions, theoretical contributions, dataset contributions, literature survey contributions, opinion contributions ▪ descriptive, relational and experimental research - statistical significance (e.g. t tests) - correlation does not imply a causal relationship ▪ surveys, diaries and interviews ▪ grounded theory (inductive research) ▪ ethnography ▪ automatic data collection - measuring the human and online data ▪ human computation
37. Beat Signer - Department of Computer Science - bsigner@vub.be 37 December 8, 2025 Course Summary … 8. Use Cases and Course Review ▪ use cases for different HCI research contributions
38. Beat Signer - Department of Computer Science - bsigner@vub.be 38 December 8, 2025 Exam ▪ Written closed book exam ▪ covers content of lectures and exercises ▪ Date ▪ January 19, E.0.04 and E.0.05, 09:00–12:00 ▪ Overall grade = written exam (50%) + assignment (50%) ▪ for the assignment students have some flexibility in distributing the grades (±2 points) - send us an email by January 12, 2026 with all group members in CC if you want to distribute some points ▪ note that the grade for the written exam as well as for the assign- ment have to be 8/20 or higher in order to pass the exam! ▪ Submission of the assignment via Canvas ▪ deadline: December 23, 24:00 (Brussels timezone)
39. Beat Signer - Department of Computer Science - bsigner@vub.be 39 December 8, 2025 Exam … ▪ The exam will cover all the content presented in the lectures as well as any additional information from the exercise sessions ▪ includes the videos shown in some of the lectures ▪ Make sure that you understand the basic concepts first ▪ however, we might ask questions at any level of detail to evaluate your knowledge ▪ Applying the theoretical knowledge to concrete use cases ▪ e.g.given a description of a specific problem in a given domain - decide about requirements gathering techniques - define different types of requirements
40. Beat Signer - Department of Computer Science - bsigner@vub.be 40 December 8, 2025 Exam … ▪ Applying the theoretical knowledge to concrete use cases … ▪ e.g.given a description /pictures of a specific UI - identify potential issues based on heuristics, Gestalt principles, Fitts’s law and other guidelines ▪ Describing (and potentially applying) certain concepts ▪ Interaction Design Lifecyle Model ▪ Double Diamond Design Process ▪ steps of the DECIDE evaluation framework ▪ usability goals ▪ …
41. Beat Signer - Department of Computer Science - bsigner@vub.be 41 December 8, 2025 Other Related Courses ▪ Specialisation: Data Management and Analytics (DAMA) ▪ Information Visualisation (MA) ▪ representation of data (encoding of value /relation) ▪ presentation of data (visualisation techniques) ▪ interaction ▪ dashboards and case studies ▪ Next Generation User Interfaces (MA) ▪ multimodal interaction - theoretical concepts, fusion and fission, ... ▪ interactive tabletops and surfaces ▪ pen-based interaction ▪ gesture-based interaction ▪ tangible interaction and mixed reality
42. Beat Signer - Department of Computer Science - bsigner@vub.be 42 December 8, 2025 Other Related Courses ... ▪ Advanced Topics in Big Data (MA) ▪ seminar about recent developments in Big Data ▪ scalable data management and big data analytics ▪ graph processing and distributed query processing ▪ human-data interaction and human-in-the loop data processing ▪ information visualisation and data physicalisation ▪ Bachelor's and a Master's thesis
43. Beat Signer - Department of Computer Science - bsigner@vub.be 43 December 8, 2025 References ▪ B. Signer, Fundamental Concepts for Interactive Paper and Cross-Media Information Spaces, Second Edition, ISBN 978-3-8370-2713-6, August 2017 cover2.jpg B. Signer, Pen-based Interaction, Handbook of Human Computer Interaction, Major Reference Work, Springer Nature, 2024  https://beatsigner.com/publications/pen-based-interaction.pdf ▪ J.O. Wobbrock and J.A. Kientz, Research Contributions in Human-Computer Interaction, Interactions 23(3), April 2016 ▪ https://doi.org/10.1145/2907069 A blue and white sign with text Description automatically generated
44. Beat Signer - Department of Computer Science - bsigner@vub.be 44 December 8, 2025 References ... ▪ B. Signer and T.J. Curtin, Tangible Holograms: Towards Mobile Physical Augmentation of Virtual Objects, Technical Report WISE Lab, WISE-2017-01, March 2017 ▪ https://beatsigner.com/publications/signer_arXiv2017.pdf ▪ B. Signer, P. Ebrahimi, T.J. Curtin and Ahmed K.A. Abdullah, Towards a Framework for Dynamic Data Physicalisation, International Workshop Toward a Design Language for Data Physicalisation, Berlin, Germany, October 2018 ▪ https://beatsigner.com/publications/signer_DataPhys2018.pdf
45. Beat Signer - Department of Computer Science - bsigner@vub.be 45 December 8, 2025 References ... ▪ E.I. Reuss, B. Signer and M.C. Norrie, PowerPoint Multimedia Presentations in Computer Science Education: What do Users Need?, Proceedings of USAB 2008, 4th Symposium on Usability & HCI for Education and Work, Graz, Austria, November 2008 ▪ https://beatsigner.com/publications/reuss_USAB2008.pdf ▪ A. Sanctorum and B. Signer, eSPACE: Leveraging Theoretical Foundations for the End-User Development of Cross-Device and IoT Applications, ACM Transactions on Computer-Human Interaction (TOCHI), 2025 ▪ https://wise.vub.ac.be/index.php/publication/espace-leveraging- theoretical-foundations-end-user-development-cross-device-and-iot
46. Beat Signer - Department of Computer Science - bsigner@vub.be 46 December 8, 2025 References ... ▪ R. Roels and B. Signer, MindXpres: An Extensible Content-driven Cross-Media Presentation Platform, Proceedings of WISE 2014, 15th International Conference on Web Information System Engineering, Thessaloniki, Greece, October, 2014 ▪ https://beatsigner.com/publications/roels_WISE2014.pdf ▪ R. Roels, P. Mestereaga and B. Signer, An Interactive Source Code Visualisation Plug-in for the MindXpres Presentation Platform, Communications in Computer and Information Science (CCIS), 583, 2016 ▪ https://beatsigner.com/publications/roels_CCIS2016.pdf
47. Beat Signer - Department of Computer Science - bsigner@vub.be 47 December 8, 2025 References ... ▪ M.C. Norrie, B. Signer, M. Grossniklaus, R. Belotti, C. Decurtins and N. Weibel, Context-aware Platform for Mobile Data Management, Wireless Networks (WINET), 13(6), Springer, December 2007 ▪ https://beatsigner.com/publications/norrie_WINET2007.pdf ▪ B. Signer, Towards Cross-Media Information Spaces and Architectures, Proceedings of RCIS 2019, Brussels, Belgium, May 2019 ▪ https://beatsigner.com/publications/signer_RCIS2019.pdf
48. Beat Signer - Department of Computer Science - bsigner@vub.be 48 December 8, 2025 References ... ▪ B. Signer, U. Kurmann and M.C. Norrie, iGesture: A General Gesture Recognition Framework, Proceedings of ICDAR 2007, 9th International Conference on Document Analysis and Recognition, Curitiba, Brazil, September 2007 ▪ https://beatsigner.com/publications/signer_ICDAR2007.pdf ▪ S. Trullemans, A. Sanctorum and B. Signer, PimVis: Exploring and Re-finding Documents in Cross-Media Information Spaces, Proceedings of AVI 2016, International Working Conference on Advanced Visual Interfaces, Bari, Italy, June 2016 ▪ https://beatsigner.com/publications/trullemans_AVI2016.pdf
49. Beat Signer - Department of Computer Science - bsigner@vub.be 49 December 8, 2025 References ... ▪ B. Signer and M.C. Norrie, As We May Link: A General Metamodel for Hypermedia Systems, Proceedings of ER 2007, International Conference on Conceptual Modeling, Auckland, New Zealand, November 2007 ▪ https://beatsigner.com/publications/signer_ER2007.pdf ▪ B. Signer, What is Wrong with Digital Documents? A Conceptual Model for Structural Cross-Media Content Composition and Reuse, Proceedings of ER 2010, International Conference on Conceptual Modeling, Vancouver, Canada, November 2010 ▪ https://beatsigner.com/publications/signer_ER2010.pdf
50. Beat Signer - Department of Computer Science - bsigner@vub.be 50 December 8, 2025 References ... ▪ S. Trullemans, L. Van Holsbeeke and B. Signer, The Context Modelling Toolkit: A Unified Multi-layered Context Modelling Approach, Proceedings of PACMHCI 1(1), June 2017 ▪ https://beatsigner.com/publications/trullemans_EICS2017.pdf ▪ M. Van de Wynckel and B. Signer, Indoor Positioning Using the OpenHPS Framework, Proceedings of IPIN 2021, International Conference on Indoor Positioning and Indoor Navigation, Lloret de Mar, Spain, November 2021 ▪ https://beatsigner.com/publications/trullemans_EICS2017.pdf
51. Beat Signer - Department of Computer Science - bsigner@vub.be 51 December 8, 2025 References ... ▪ L. Hoste, B. Dumas and B. Signer, Mudra: A Unified Multimodal Interaction Framework, Proceedings of ICMI 2011, International Conference on Multimodal Interaction, Alicante, Spain, November 2011 ▪ https://beatsigner.com/publications/hoste_ICMI2011.pdf ▪ B. Signer, As We May Interact: Challenges and Opportunities for Next-Generation Human-Information Interaction, Proceedings of HUMAN 2024, Poznań, Poland, September 2024 ▪ https://beatsigner.com/publications/signer_HUMAN2024.pdf ▪
52. Beat Signer - Department of Computer Science - bsigner@vub.be 52 December 8, 2025 References ... ▪ MindXpres Project ▪ https://mindxpres.com ▪ Dynamic Data Physicalisation and Tangible Holograms ▪ https://wise.vub.ac.be/topic/dynamic-data-physicalisation ▪ https://wise.vub.ac.be/topic/tangible-holograms-tangho ▪ iGesture ▪ http://igesture.org
53. 2 December 2005 The End Good Luck with the Exam!