A Partitioning Algorithm for Markov Decision Processes with Applications to Market Microstructure

Forthcoming in Management Science

44 Pages Posted: 29 Nov 2013 Last revised: 16 May 2020

See all articles by Ningyuan Chen

Ningyuan Chen

University of Toronto - Rotman School of Management

Steven Kou

Boston University

Chun Wang

Tsinghua University, School of Economics and Management

Date Written: September 20, 2015

Abstract

We propose a partitioning algorithm to solve a class of linear-quadratic Markov decision processes with inequality constraints and non-convex stage-wise cost; within each region of the partitioned state space, the value function and the optimal policy have analytical quadratic and linear forms, respectively. Compared to grid-based numerical schemes, the partitioning algorithm gives the closed-form solution without discretization error, and in many cases does not suffer from the curse of dimensionality. The algorithm is applied to two applications. In the main application, we present a model for limit order books with stochastic market depth to study the optimal order execution problem; stochastic market depth is consistent with empirical studies and necessary to accommodate various order activities. The optimal execution policy obtained by the algorithm significantly outperforms that of a deterministic market depth model in numerical examples. In the second application, we use the algorithm to compute the exact optimal solution to the renewable electricity management problem, for which previously only an approximate solution is known. As a comparison, we show that the approximate solution can be quite inaccurate for some initial states and thus demonstrate an advantage of the exact solution.

Keywords: Markov chains, Large order execution, Electricity trading/production, Partitioning, Quadratic stochastic programming

JEL Classification: C61, D49, G10, G20

Suggested Citation

Chen, Ningyuan and Kou, Steven and Wang, Chun, A Partitioning Algorithm for Markov Decision Processes with Applications to Market Microstructure (September 20, 2015). Forthcoming in Management Science, Available at SSRN: https://ssrn.com/abstract=2360552 or http://dx.doi.org/10.2139/ssrn.2360552

Ningyuan Chen

University of Toronto - Rotman School of Management ( email )

Steven Kou (Contact Author)

Boston University ( email )

595 Commonwealth Avenue
Boston, MA 02215
United States
6173583318 (Phone)

Chun Wang

Tsinghua University, School of Economics and Management ( email )

Beijing, 100084
China

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
582
Abstract Views
3,358
Rank
86,056
PlumX Metrics