Scaling up deep reinforcement learning for multi-domain dialogue systems

Heriberto Cuayahuitl, Seunghak Yu, Ashley Williamson, Jacob Carse
2017 2017 International Joint Conference on Neural Networks (IJCNN)  
Standard deep reinforcement learning methods such as Deep Q-Networks (DQN) for multiple tasks (domains) face scalability problems due to large search spaces. This paper proposes a three-stage method for multi-domain dialogue policy learning-termed NDQN, and applies it to an informationseeking spoken dialogue system in the domains of restaurants and hotels. In this method, the first stage does multi-policy learning via a network of DQN agents; the second makes use of compact state
more » ... by compressing raw inputs; and the third stage applies a pre-training phase for bootstraping the behaviour of agents in the network. Experimental results comparing DQN (baseline) versus NDQN (proposed) using simulations report that the proposed method exhibits better scalability and is promising for optimising the behaviour of multi-domain dialogue systems. An additional evaluation reports that the NDQN agents outperformed a K-Nearest Neighbour baseline in task success and dialogue length, yielding more efficient and successful dialogues.
doi:10.1109/ijcnn.2017.7966275 dblp:conf/ijcnn/CuayahuitlYWC17 fatcat:o4ijg3ryrfde5emfj2olnu62km