Profit-Driven Experimental Design

41 Pages Posted: 2 Aug 2021 Last revised: 22 Nov 2022

See all articles by Yuhao Wang

Yuhao Wang

Tsinghua University - Institute for Interdisciplinary Information Sciences

Weiming Zhu

HKU Business School, The University of Hong Kong

Date Written: July 30, 2021

Abstract

From intense competition to the recent pandemic, companies face considerable volatility in the business environment. For companies that design experiments to identify parameters of interest and make subsequent policy decisions based on these parameters, the cost of such experimentation has become increasingly comparable to the economic gains obtained, as the insights offered by an experiment can be short-lived due to changing market conditions. In this paper, we develop a general framework to quantify the total expected profit from both the experimental and postexperimental stages given an experimental strategy. The proposed framework is constructed using the asymptotic properties of the underlying parameter estimates as a channel to connect the profits from the two stages. We demonstrate that the order of the optimal sample size and its regret are critically shaped by the curvature of the postexperimental profit function, which is defined as a quantitative measure of the local sensitivity of the postexperimental profit function to the parameter of interest. By exploiting this property, we are able to identify the order of the optimal sample size and its regret without prior knowledge. We illustrate through demand-learning newsvendor and pricing problems that the curvature function, the order of the optimal sample size and its regret can be derived in closed-form, despite that the postexperimental profit function may not be fully specified. We also develop an algorithm to numerically compute the order of the optimal sample size when the curvature function cannot be solved in closed-form. Finally, when prior information is available, we provide the lower and upper bounds of the optimal sample size that maximizes the total expected profit.

Keywords: experimental design, asymptotic analysis, cost-effectiveness, sample size

JEL Classification: C90, C44

Suggested Citation

Wang, Yuhao and Zhu, Weiming, Profit-Driven Experimental Design (July 30, 2021). Available at SSRN: https://ssrn.com/abstract=3896229 or http://dx.doi.org/10.2139/ssrn.3896229

Yuhao Wang (Contact Author)

Tsinghua University - Institute for Interdisciplinary Information Sciences ( email )

Beijing, 100084
China

Weiming Zhu

HKU Business School, The University of Hong Kong ( email )

Hong Kong
China

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
243
Abstract Views
1,006
Rank
228,791
PlumX Metrics