Model-based reinforcement learning (mbRL) has been widely used in explaining animal behavior. In mbRL, the model, or the structure of the task, is used to evaluate the associations between actions and outcomes. It has been proposed that the orbitofrontal cortex (OFC) encodes the model during mbRL. However, it is not well understood how the OFC acquires and stores model information. Here, we propose a neural network framework based on reservoir computing. Reservoir networks exhibit heterogeneous and dynamic activity patterns that are suitable to encode task states. The information can be extracted by a linear readout trained with reinforcement learning. We demonstrate how our framework acquires and stores the task state space. The framework exhibits mbRL behavior and its aspects resemble experimental findings of the OFC. Our study provides a theoretical explanation of how the OFC may contribute to mbRL and a new approach to understanding the neural mechanism underlying mbRL.