TWO!EARS Deliverable D4.1 - Feedback-loop selection and listing (WP4: Active listening, feedback loops & integration of cross-modal information; FP7-ICT-2013-C TWO!EARS FET-Open Project 618075)

Jens Blauert, Thomas Walther
2019 Zenodo  
As stated in the project proposal, Two!Ears is going to challenge current thinking in auditory modeling by replacing common paradigms in this field with a systemic approach, whereby human listeners are regarded as multi-modal agents that develop their concept of the world by exploratory interaction. The goal of Two!Ears is to develop an intelligent, computational model of active auditory perception and experience in a multi-modal context. The resulting system framework will form a structural
more » ... k from binaural perception to judgment and action, realized by interleaved signal-driven (bottom-up) and hypothesis-driven (top-down) feedback processing within an innovative expert-system architecture. The main objective of WP4 in this context is to set up a framework that endows the Two!Ears system with active-listening capabilities. To capture bottom-up data processing as well as the top-down mechanisms as required for active listening tasks, the proposed framework must host suitable feedback loops. It is the general task of WP4 to design an appropriate system architecture for this requirement, investigate meaningful feedback paths, implement them, and finally evaluate them with respect to their functionalities. Input from modalities other than the auditory one will also be considered as a source of feedback information, particularly, position, direction of speed of head-&-torso movements (proprioceptive and sensorimotor input), further, identified optical objects (visual input). WP4 is split into five tasks. This deliverable mainly entails our advance on the key task for the current project period, namely, task 4.1. As this task consists of several sub-tasks, the remainder of our report is organized as follows. D4.1, part B A consolidated literature survey regarding "active listening" has been compiled, summarizing available information about the mechanics and functions of feedback in the auditory system. Also, multi-modal approaches have been reviewed and evaluated with regard to their value for Two!Ears. Her [...]
doi:10.5281/zenodo.2595244 fatcat:3oocvxholvgr3ecgubmq3uwxqa