Deep Reinforcement Learning for Finance and the Efficient Market Hypothesis

Abstract

Is there an informational gain by training a Deep Reinforcement Learning agent for automated stock trading using other time series than the one to be traded? In this work, we implement a DRL algorithm in a solid framework within a model-free and actor-critic approach and learn it with 21 global Multi Assets to predict and trade on the S&P 500. The Efficient Market Hypothesis sets out that it is impossible to gather more information from the broader input. We demand to learn a DRL agent on this index with and without the additional information of these several Multi Assets to determine if the agent could capture invisible dependencies to end up with an informational gain and a better performance.
The aim of this work is not to tune the hyperparameters of a DRL agent; several papers already exist on this subject. Nevertheless, we use a proven setup as model architecture. We take a Multi Layer Perceptron (short: MLP) as the neural network architecture with two hidden layers and 64 neurons each layer. The activation function used is the hyperbolic tangent. Further, Proximal Policy Optimization (short: PPO) is used as the policy for simple implementation and enabling a continuous state space. To deal with uncertainties of neural nets, we learn 100 agents for each scenario and compared both results. Neither the Sharpe ratios nor the cumulative returns are better in the more complex approach with the additional information of the Multi Assets, and even the single approach performed marginally better. However, we demonstrate that the complexly learned agent delivers less scattering over the 100 simulations in terms of the risk-adjusted returns, so there is an informational gain due to Multi Assets. A DRL agent learned with additional information delivers more robust results compared to the taken risk. We deliver valuable results for the further development of Deep Reinforcement Learning and provide a unique and resourceful approach.

Keywords: Deep Reinforcement Learning, Automated Stock Trading, Finance, Efficient Market Hypothesis, AI

JEL Classification: C60, C61, G10, G11, G14, G15