Predicting Daily Probability Distributions of S&P500 Returns

31 Pages Posted: 23 Oct 2008

See all articles by Andreas Weigend

Andreas Weigend

Stern School of Business, New York University

Shanming Shi

J.P. Morgan Chase & Co. - Proprietary Positioning Business

Date Written: August 1998

Abstract

Most approaches in forecasting merely try to predict the next value of the time series.In contrast, this paper presents a framework to predict the full probability distribution. Itis expressed as a mixture model: the dynamics of the individual states is modeled with so-called"experts" (potentially nonlinear neural networks), and the dynamics between the states is modeledusing a hidden Markov approach. The full density predictions are obtained by a weighted superpositionof the individual densities of each expert. This model class is called "hidden Markov experts".Results are presented for daily S&P500 data. While the predictive accuracy of the mean doesnot improve over simpler models, evaluating the prediction of the full density shows a clear out-of-sampleimprovement both over a simple GARCH(1,l) model (which assumes Gaussian distributedreturns) and over a "gated experts" model (which expresses the weighting for each state non-recursivelyas a function of external inputs). Several interpretations are given: the blending ofsupervised and unsupervised learning, the discovery of hidden states, the combination of forecasts,the specialization of experts, the removal of outliers, and the persistence of volatility.

Keywords: Forecasting, Density Prediction, Conditional Distribution, Mixture Models, Time Series Analysis, Hidden Markov Models, Gated Experts, Hidden Markov Experts, Model Comparison, Density Evaluation, Computational Finance, Risk Management

Suggested Citation

Weigend, Andreas and Shi, Shanming, Predicting Daily Probability Distributions of S&P500 Returns (August 1998). NYU Working Paper No. IS-98-23, Available at SSRN: https://ssrn.com/abstract=1288468

Andreas Weigend (Contact Author)

Stern School of Business, New York University ( email )

MEC 9-74
New York, NY 10012
United States
212-998-0803 (Phone)
212-995-4228 (Fax)

HOME PAGE: http://www.weigend.com

Shanming Shi

J.P. Morgan Chase & Co. - Proprietary Positioning Business ( email )

United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
506
Abstract Views
2,173
Rank
102,466
PlumX Metrics