This project is concerned with the question of how shared situation models arise in communication and how the formation of these situation models interacts with the interlocutors' visual perception of the environment.

 

Language processing involves the formation of a representation of the matters talked about – besides a representation of the language itself. For a dialogue to be successful, it is essential that both interlocutors develop partially aligned models of the situation under discussion. We assume that this holds also for communication about complex, visually perceived environments and propose that they are mentally represented by selective situation models based on salience.

 

The main goal of the project is to investigate the process of building a shared perceptually grounded mental model of the situation under discussion through conversation and to exploit the principles of human-human situation-model aligning for human-robot communication. This will enable a mobile robot to more effectively understand descriptive language, to deal with partially unknown situations, to ignore irrelevant information and to act in cluttered environments. We intend to examine factors that lead to an alignment of such representations in situational models. We focus on relational and perceptual components of such situation models that are grounded in a common visually perceived environment. Several aspects of the question how shared situation models arise will be addressed in the project by combining psycholinguistic experiments with theoretical and conceptual work in cognitive science, and computational modelling. These include the interplay between automatic alignment processes and resource-limited elaborate construction, the essentially resource-free use of these integrated situation models once they are formed, and the role of pan-situational world knowledge (schemata, categorisation levels, typicality) invoked by language use.