A Model of Adaptive Reinforcement Learning

17 Pages Posted: 3 Apr 2019

See all articles by Julian Romero

Julian Romero

University of Arizona - Eller College of Management

Yaroslav Rosokha

Purdue University - Krannert School of Management

Date Written: March 11, 2019

Abstract

We develop a model of learning that extends the classic models of reinforcement learning to a continuous, multidimensional strategy space. The model takes advantage of the recent approximation methods to tackle the curse of dimensionality inherent to a traditional discretization approach. Crucially, the model endogenously partitions strategies into sets of similar strategies, and allows agents to learn over these sets which speeds up the learning process. We provide an application of our model to predict which memory-1 mixed strategies will be played in the inde nitely repeated Prisoner's Dilemma game. We show that despite allowing the mixed strategies, strategies close to the pure strategies always defect, grim trigger, and tit-for-tat emerge -- a result that qualitatively matches recent strategy choice experiments with human subjects.

Keywords: Reinforcement Learning, Repeated-game Strategies, Repeated Prisoner's Dilemma, Mixed Strategies, Agent-based Models, Markov Strategies

Suggested Citation

Romero, Julian and Rosokha, Yaroslav, A Model of Adaptive Reinforcement Learning (March 11, 2019). Available at SSRN: https://ssrn.com/abstract=3350711 or http://dx.doi.org/10.2139/ssrn.3350711

Julian Romero

University of Arizona - Eller College of Management ( email )

McClelland Hall
P.O. Box 210108
Tucson, AZ 85721-0108
United States

Yaroslav Rosokha (Contact Author)

Purdue University - Krannert School of Management ( email )

1310 Krannert Building
West Lafayette, IN 47907-1310
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
182
Abstract Views
1,569
Rank
298,914
PlumX Metrics