Testing the Effectiveness of Lottery Incentives in Online Experiments

Abstract

This article investigates the effectiveness of lottery incentive schemes for eliciting consumer valuations in large-scale online experiments. We implement a fully incentivized condition within a geographically dispersed sample of consumers in which bids for a criollo steak elicited by a BDM mechanism are realized with certainty and the products are priority-shipped in dry-ice coolers. The fully incentivized condition is compared to Between-Subject Random Incentivized Schemes (BRIS), in which only a fraction of subjects realize their choices. We tested two treatments with a 10% probability framed as a percentage or absolute number of subjects, one treatment with 1% probability, and a purely hypothetical reference condition. The results reveal that BRIS with 10% and 1% payment probabilities are effective in eliciting valuations that are statistically indistinguishable from the fully incentivized scheme. In addition to finding insignificant statistical difference between 10% and 1% and the fully incentivized scheme, all incentivized conditions mitigate hypothetical bias, resulting in lower product valuations than the purely hypothetical condition. We contribute a novel methodological framework for conducting large-scale experiments with geographically diverse and representative subjects, increasing the external validity and producing reliable valuations while significantly reducing financial and logistic constraints.