© 2019, The Author(s). Representations of our future environment are essential for planning and decision making. Previous research in humans has demonstrated that the hippocampus is a critical region for forming and retrieving associations, while the medial orbitofrontal cortex (OFC) is an important region for representing information about recent states. However, it is not clear how the brain acquires predictive representations during goal-directed learning. Here, we show using fMRI that while participants learned to find rewards in multiple different Y-maze environments, hippocampal activity was highest during initial exposure and then decayed across the remaining repetitions of each maze, consistent with a role in rapid encoding. Importantly, multivariate patterns in the OFC-VPFC came to represent predictive information about upcoming states approximately 30 s in the future. Our findings provide a mechanism by which the brain can build models of the world that span long-timescales to make predictions.