Continuous-Time Mean-Variance Portfolio Selection: A Reinforcement Learning Framework

39 Pages Posted: 29 May 2019

See all articles by Haoran Wang

Haoran Wang

Columbia University - Department of Industrial Engineering and Operations Research (IEOR)

Xun Yu Zhou

Columbia University - Department of Industrial Engineering and Operations Research (IEOR)

Date Written: May 5, 2019

Abstract

We approach the continuous-time mean-variance (MV) portfolio selection with reinforcement learning (RL). The problem is to achieve the best tradeoff between exploration and exploitation, and is formulated as an entropy-regularized, relaxed stochastic control problem. We prove that the optimal feedback policy for this problem must be Gaussian, with time-decaying variance. We then establish connections between the entropy-regularized MV and the classical MV, including the solvability equivalence and the convergence as exploration weighting parameter decays to zero. Finally, we prove a policy improvement theorem, based on which we devise an implementable RL algorithm. We find that our algorithm outperforms both an adaptive control based method and a deep neural networks based algorithm by a large margin in our simulations.

Keywords: Reinforcement learning, mean-variance portfolio selection, entropy regularization, stochastic control, value function, Gaussian distribution, policy improvement theorem

JEL Classification: C02, C61, C63, G11,

Suggested Citation

Wang, Haoran and Zhou, Xunyu, Continuous-Time Mean-Variance Portfolio Selection: A Reinforcement Learning Framework (May 5, 2019). Available at SSRN: https://ssrn.com/abstract=3382932 or http://dx.doi.org/10.2139/ssrn.3382932

Haoran Wang (Contact Author)

Columbia University - Department of Industrial Engineering and Operations Research (IEOR) ( email )

331 S.W. Mudd Building
500 West 120th Street
New York, NY 10027
United States

Xunyu Zhou

Columbia University - Department of Industrial Engineering and Operations Research (IEOR) ( email )

331 S.W. Mudd Building
500 West 120th Street
New York, NY 10027
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
305
Abstract Views
1,401
Rank
181,711
PlumX Metrics