Predicting Consumer Choice from Raw Eye-Movement Data Using the RETINA Deep Learning Architecture

34 Pages Posted: 31 Jan 2023 Last revised: 25 Oct 2023

See all articles by Moshe Unger

Moshe Unger

Tel Aviv University - Coller School of Management

Michel Wedel

University of Maryland - Robert H. Smith School of Business

Alexander Tuzhilin

New York University (NYU) - Leonard N. Stern School of Business; New York University (NYU) - Department of Information, Operations, and Management Sciences

Date Written: January 1, 2023

Abstract

We propose the use of a deep learning architecture, called RETINA, to predict multi-alternative, multi-attribute consumer choice from eye movement data. RETINA directly uses the complete time series of raw eye-tracking data from both eyes as input to state-of-the art Transformer and Metric Learning Deep Learning methods. Using the raw data input eliminates the information loss that may result from first calculating fixations, deriving metrics from the fixations data and analysing those metrics, as has been often done in eye movement research, and allows us to apply Deep Learning to eye tracking data sets of the size commonly encountered in academic and applied research. Using a data set with 112 respondents who made choices among four laptops, we show that the proposed architecture outperforms other state-of-the-art machine learning methods (standard BERT, LSTM, AutoML, logistic regression) calibrated on raw data or fixation data. The analysis of partial time and partial data segments reveals the ability of RETINA to predict choice outcomes well before participants reach a decision. Specifically, we find that using a mere 5 seconds of data, the RETINA architecture achieves a predictive validation accuracy of over 0.7. We provide an assessment of which features of the eye movement data contribute to RETINA's prediction accuracy. We make recommendations on how the proposed deep learning architecture can be used as a basis for future academic research, in particular its application to eye movements collected from front-facing video cameras.

Keywords: Deep Learning, Eye Tracking, BERT, Metric Learning, AutoML

JEL Classification: C8, M31, C22

Suggested Citation

Unger, Moshe and Wedel, Michel and Tuzhilin, Alexander, Predicting Consumer Choice from Raw Eye-Movement Data Using the RETINA Deep Learning Architecture (January 1, 2023). Available at SSRN: https://ssrn.com/abstract=4341410 or http://dx.doi.org/10.2139/ssrn.4341410

Moshe Unger (Contact Author)

Tel Aviv University - Coller School of Management ( email )

Tel Aviv
Israel

Michel Wedel

University of Maryland - Robert H. Smith School of Business ( email )

College Park, MD 20742-1815
United States

HOME PAGE: http://www.rhsmith.umd.edu/directory/michel-wedel

Alexander Tuzhilin

New York University (NYU) - Leonard N. Stern School of Business ( email )

44 West 4th Street
Suite 9-160
New York, NY NY 10012
United States

New York University (NYU) - Department of Information, Operations, and Management Sciences

44 West Fourth Street
New York, NY 10012
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
188
Abstract Views
644
Rank
290,726
PlumX Metrics