Decisions from experience: investigating individual-level decisions in bandit problems via experimentation and computational cognitive modelling (PhD)

Abstract

Decision-making is an act of either choosing options or judging values based upon one's knowledge about the environment. An important aspect of decision-making is a choice between options involving risk. Previous literature has used monetary gambles (called bandit problems) to investigate people's risky decisions. Specifically, bandit problems have been presented in two different formats: description and experience. In the description format, outcomes and the corresponding probabilities under different options are spelled out in a written description. In the experience format, outcomes under various options are experienced with certain frequencies. Several aggregate and hierarchical models have been proposed to account for people's decisions from experience. However, currently little is known on how these aggregate and hierarchical models account for choices made by individual participants. Furthermore, research shows that people exhibit a different pattern of choices between the descriptive and experiential formats - a phenomenon known as the Description-Experience (DE) gap. However, little is known on how the addition of intermediate options (options with intermediate expected values) and contextual framing influence the DE gap and people's search for outcomes. First, we tested the ability of several aggregate and hierarchical models to explain choices made by individual participants. Several models, Ensemble, Cumulative Prospect Theory (CPT), Best Estimation and Simulation Techniques (BEAST), Natural-Mean Heuristic (NMH), and Instance-Based Learning (IBL), had their parameters calibrated to individual choices using the log-likelihood function in a large dataset involving experiential decisions. Results revealed that the aggregate models (like CPT and IBL) accounted for individual choices better than hierarchical models (like Ensemble and BEAST) upon generalizations to different data sets. Furthermore, we evaluated the influence of intermediate options and contextual framing on the DE gap and people's search strategies via two experiments. In the first experiment, 120 participants made choice decisions across investment problems that differed in the absence or presence of an intermediate option. Results showed that adding intermediate options reduced the DE gap. In the second experiment, 160 participants made choice decisions in problems like those presented in experiment 1; however, now the problems lacked the investment framing. Results replicated findings from the first experiment and showed an absence of the DE gap on both the risky and intermediate options. Finally, we performed individual-differences analyses and tested the ability of computational models to explain the experimental results. Specifically, both the IBL and NMH models, where the driving mechanisms were recency and frequency of experienced outcomes, accounted for people's decisions in the presence or absence of intermediate options, different contextual framings, and different search strategies. We highlight the implications of our results from experiments and computational models for risky decisions in bandit problems.