Learning from delayed feedback: neural responses in temporal credit assignment

Matthew M. Walsh, John R. Anderson
2011 Cognitive, Affective, & Behavioral Neuroscience  
When feedback follows a sequence of decisions, relationships between actions and outcomes can be difficult to learn. We used event-related potentials (ERPs) to understand how people overcome this temporal credit assignment problem. Participants performed a sequential decision task that required two decisions on each trial. The first decision led to an intermediate state that was predictive of the trial outcome, and the second decision was followed by positive or negative trial feedback. The
more » ... back-related negativity (fERN), a component thought to reflect reward prediction error, followed negative feedback and negative intermediate states. This suggests that participants evaluated intermediate states in terms of expected future reward, and that these evaluations supported learning of earlier actions within sequences. We examine the predictions of several temporaldifference models to determine whether the behavioral and ERP results reflected a reinforcement-learning process.
doi:10.3758/s13415-011-0027-0 pmid:21416212 pmcid:PMC3208325 fatcat:j6j6dka3efd3nnnpwhlb2aahkq