Hierarchical Deep Q-Network from Imperfect Demonstrations in Minecraft [article]

Alexey Skrynnik, Aleksey Staroverov, Ermek Aitygulov, Kirill Aksenov, Vasilii Davydov, Aleksandr I. Panov
2020 arXiv   pre-print
We present Hierarchical Deep Q-Network (HDQfD) that took first place in the MineRL competition. HDQfD works on imperfect demonstrations and utilizes the hierarchical structure of expert trajectories. We introduce the procedure of extracting an effective sequence of meta-actions and subgoals from demonstration data. We present a structured task-dependent replay buffer and adaptive prioritizing technique that allow the HDQfD agent to gradually erase poor-quality expert data from the buffer. In
more » ... s paper, we present the details of the HDQfD algorithm and give the experimental results in the Minecraft domain.
arXiv:1912.08664v4 fatcat:c7t67u2vxzgmjcqxmtwds4ve3i