Ensembles of Crowds and Computers: Experiments in Forecasting

Abstract

This paper explores the power of news sentiment to predict financial returns, in particular the returns of a set of European stocks. Building on past decision support work going back to the Delphi method this paper describes a text analysis expert weighting algorithm that aggregates the responses of both humans and algorithms by dynamically selecting the best response according to previous performance. The proposed system is tested through an experiment in which ensembles of experts, crowds and machines analyzed Thomson Reuters news stories and predicted the returns of the relevant stocks mentioned right after the stories appeared. The expert weighting algorithm was better than or as good as the best algorithm or human in most cases. The capacity of the algorithm to dynamically select best answers from humans and machines results in an evolving collective intelligence: the final decision is an aggregation of the best automated individual answers, some of these come from machines, and some from humans. Additionally, this paper shows that the groups of humans, algorithms, and expert weighting algorithms have associated with them particular news topics that these groups are good at making predictions from.

Keywords: Human machine ensembles, forecasting, Delphi method, crowdsourcing, machine learning

JEL Classification: C53, C63, G12, G14, F30, C9