The Case for Multimodal Virtual Environments in Multi-Agent Simulations
Slides from a talk presented at the Workshop on Multimodal Human-Agent Interfaces for Virtual Environments at Macquarie University in Sydney, Australia on 20 November 2014.
Michael Papasimeon Workshop on Multimodal Human-Agent Interfaces for Virtual Environments Macquarie University Sydney 20 November 2014 Michael Papasimeon Multimodal Virtual Environments 20 November 2014 1 / 9
Multi-Agent Simulations Computational Science Computational Operations Research Aim Authentic, credible and situated computational models of human decision making in multi-agent simulations used in computational operations research. Michael Papasimeon Multimodal Virtual Environments 20 November 2014 2 / 9
Virtual Environment a dynamic computational representation of a world 1 populated by entities and intelligent agents. Intelligent Agents an autonomous decision making entity situated in virtual environment that can perceive the environment, reason and make decisions and take action in the environment. 1Real, fictitious or abstract world Michael Papasimeon Multimodal Virtual Environments 20 November 2014 3 / 9
Environment K RF IR Intelligent Agent Engine Graphics Engine Audio Engine Physics Engine Virtual Environment See: The Human-Agent Virtual Environment Papasimeon, Pearce and Goss. 6th International Conference on Autonomous Agents and Multi-Agent Systems, 2007 Michael Papasimeon Multimodal Virtual Environments 20 November 2014 5 / 9
with the environment they are situated in. Ecology Ecology = Environment + Humans/Animals + Interaction Michael Papasimeon Multimodal Virtual Environments 20 November 2014 6 / 9
with the environment they are situated in. Ecology Ecology = Environment + Humans/Animals + Interaction Virtual Ecology Virtual Ecology = Virtual Environment + Intelligent Agents + Interaction A Multi-Agent Simulation or a Video Game is an example of a virtual ecology. Designing one of these systems involves designing a virtual ecology. Michael Papasimeon Multimodal Virtual Environments 20 November 2014 6 / 9
is the Theory of Affordances. Affordances Affordances are the action possibilities an environment provides an agent. Michael Papasimeon Multimodal Virtual Environments 20 November 2014 7 / 9
is the Theory of Affordances. Affordances Affordances are the action possibilities an environment provides an agent. J.J. Gibson’s Definition The affordances of the environment are what it offers the animal, what it provides or furnishes, either for good or ill. Michael Papasimeon Multimodal Virtual Environments 20 November 2014 7 / 9
is the Theory of Affordances. Affordances Affordances are the action possibilities an environment provides an agent. J.J. Gibson’s Definition The affordances of the environment are what it offers the animal, what it provides or furnishes, either for good or ill. Direct Perception Gibson claimed that affordances (action possibilities) are directly perceivable by humans and animals in the environment. Michael Papasimeon Multimodal Virtual Environments 20 November 2014 7 / 9
on many different things... not just the physical environment. Action possibilities maybe influenced by the social, command, electromagnetic, team, network or other abstract environments. We want action possibilities that are dynamic, observer tailored, relational, intentional/goal oriented, introspective, meaningful and context sensitive. 2 In order to compute affordances (or find the action possibilities), our virtual environments have to be inherently multimodal 2Modelling Agent-Environment Interaction in Multi-Agent Simulations with Affordances. M. Papasimeon. 2009 Michael Papasimeon Multimodal Virtual Environments 20 November 2014 8 / 9
to consider the agent-environment interaction as an ecology. That is we should be talking about... Designing Virtual Ecologies Computational models of affordance (action possibilities) give us a closer (more situated link) between agent and environment. However, in order to determine what actions are possible (what the environment affords the agent) the environment needs to be multimodal Challenges Non-situated models of agency Design of Agent-Friendly Virtual Environments Software Engineering Adoption of Ecological Models Michael Papasimeon Multimodal Virtual Environments 20 November 2014 9 / 9