Black Box Analytics and Ethical Decision Making

40 Pages Posted: 22 Jan 2019 Last revised: 20 Jul 2021

See all articles by Michael J Davern

Michael J Davern

The University of Melbourne - Department of Accounting; University of Melbourne

Pujawati Mariestha (Estha) Gondowijoyo

The University of Melbourne

Pamela R. Murphy

Queen's University - Smith School of Business; Queen's University

Date Written: January 19, 2020

Abstract

Using an experiment with participants having management experience, we examine sales forecast decisions when using an opaque versus transparent data analytics system. Participants have private information suggesting that the forecast significantly underestimates sales, making the forecast – and their bonus – easily achievable unless adjusted. We explore the extent to which participants act less ethically by not adjusting the sales forecast upwards. We employ a 2 x 2 between-subjects design, manipulating the description of the forecasting system as opaque or transparent, and measuring feelings of responsibility in the presence (absence) of a prompt before making the adjustment decision. We find that participants make less ethical decisions when the system is opaque than when it is transparent, but feelings of responsibility overcome this problem. We also find that the least ethical participants use both more rationalizations and more self-interested reasons than those whose decisions are not as unethical, supporting the use of both economic and psychology theory when studying ethical decision-making. Our results suggest that organizations should attempt to make data analytics systems more transparent to decision-making users. However, when they cannot, they should ensure that decision-makers feel responsible for their decisions; for example, with a prompt or decision aid.

Keywords: ethical decision making, IT, machine learning, moral disengagement theory, accountability

JEL Classification: M41

Suggested Citation

Davern, Michael J. and Gondowijoyo, Pujawati Mariestha (Estha) and Murphy, Pamela R., Black Box Analytics and Ethical Decision Making (January 19, 2020). Available at SSRN: https://ssrn.com/abstract=3318717 or http://dx.doi.org/10.2139/ssrn.3318717

Michael J. Davern

The University of Melbourne - Department of Accounting ( email )

Victoria
Melbourne, 3010
Australia

University of Melbourne ( email )

Victoria
Melbourne, 3010
Australia

Pujawati Mariestha (Estha) Gondowijoyo (Contact Author)

The University of Melbourne ( email )

Parkville, 3010
Australia

Pamela R. Murphy

Queen's University - Smith School of Business ( email )

Smith School of Business - Queen's University
143 Union Street
Kingston, Ontario K7L 3N6
Canada

Queen's University ( email )

Kingston, Ontario K7L 3N6
Canada
6135333260 (Phone)

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
205
Abstract Views
2,224
Rank
269,996
PlumX Metrics