RT Journal Article
SR Electronic
T1 A neural network framework for the orbitofrontal cortex and model-based reinforcement learning
JF bioRxiv
FD Cold Spring Harbor Laboratory
SP 116608
DO 10.1101/116608
A1 Zhewei Zhang
A1 Zhenbo Cheng
A1 Zhongqiao Lin
A1 Chechang Nie
A1 Tianming Yang
YR 2017
UL http://biorxiv.org/content/early/2017/03/20/116608.abstract
AB Model-based reinforcement learning (mbRL) has been widely used in explaining animal behavior. In mbRL, the model, or the structure of the task, is used to evaluate the associations between actions and outcomes. It has been proposed that the orbitofrontal cortex (OFC) encodes the model during mbRL. However, it is not well understood how the OFC acquires and stores model information. Here, we propose a neural network framework based on reservoir computing. Reservoir networks exhibit heterogeneous and dynamic activity patterns that are suitable to encode task states. The information can be extracted by a linear readout trained with reinforcement learning. We demonstrate how our framework acquires and stores the task state space. The framework exhibits mbRL behavior and its aspects resemble experimental findings of the OFC. Our study provides a theoretical explanation of how the OFC may contribute to mbRL and a new approach to understanding the neural mechanism underlying mbRL.