Project A1 investigates alignment phenomena relying on cognitive and affective partner modelling. Work in the second period will focus on how attentive processes can be linked to affective parameters in human-agent interaction, complemented by psycholinguistic studies investigating emotion-language priming, i.e., how human listeners align with a speaker’s emotional states and how that emotional alignment can facilitate language processing in the listener.
The AI part will further investigate cooperative communication (with shared goals) in human-agent settings by implementing and extending operational partner models that allow for intentional and emotional alignment through the empathy mechanisms mimicry and role-taking. A focus will be on how attentive processes are interrelated with affective parameters, based on a reward system that can be conceived of as generating a new, non-symbolic kind of representational content, namely the internal value assigned to a certain state. The reward system will be based on dynamic emotion simulation (previously developed). This represents a core element of an automatic and pre-reflexive mechanism for selective and flexible control of attention and action.
The goals of the (new) experimental part of the project are to examine (i) the role of emotion priming in facilitating semantic interpretation, syntactic structuring and thematic role assignment in a listener; (ii) the time course with which emotions versus/and visual attention prime comprehension; (iii) the interaction of emotion-based priming of comprehension with speaker characteristics (e.g., perceived social competence of virtual agents vs. humans) and listener characteristics (e.g., age; visual and verbal working memory). The empirical research will yield information about which objects and/or events the human listener inspects in processing a speaker’s utterance and emotions: Relative inspection proportions of objects as the utterance unfolds provide insight into the listener’s evolving interpretation in response to the speaker’s words and emotions.
