Modeling decisions from experience: How models with a set of parameters for aggregate choices explain individual choices

  • Neha Sharma (Author)
    Indian Institute of Technology ,Mandi, India
  • Varun Dutt (Author)
    Indian Institute of Technology ,Mandi, India

Identifiers (Article)


One of the paradigms (called “sampling paradigm”) in judgment and decision-making involves decision-makers sample information before making a final consequential choice. In the sampling paradigm, certain computational models have been proposed where a set of single or distribution parameters is calibrated to the choice proportions of a group of participants (aggregate and hierarchical models). However, currently little is known on how aggregate and hierarchical models would account for choices made by individual participants in the sampling paradigm. In this paper, we test the ability of aggregate and hierarchical models to explain choices made by individual participants. Several models, Ensemble, Cumulative Prospect Theory (CPT), Best Estimation and Simulation Techniques (BEAST), Natural-Mean Heuristic (NMH), and Instance-Based Learning (IBL), had their parameters calibrated to individual choices in a large dataset involving the sampling paradigm. Later, these models were generalized to two large datasets in the sampling paradigm. Results revealed that the aggregate models (like CPT and IBL) accounted for individual choices better than hierarchical models (like Ensemble and BEAST) upon generalization to problems that were like those encountered during calibration. Furthermore, the CPT model, which relies on differential valuing of gains and losses, respectively, performed better than other models during calibration and generalization on datasets with similar set of problems. The IBL model, relying on recency and frequency of sampled information, and the NMH model, relying on frequency of sampled information, performed better than other models during generalization to a challenging dataset. Sequential analyses of results from different models showed how these models accounted for transitions from the last sample to final choice in human data. We highlight the implications of using aggregate and hierarchical models in explaining individual choices from experience.




Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19(6), 716–723. doi:10.1109/tac.1974.1100705

Anderson, J. R., & Lebiere, C. (1998). The atomic components of thought. Hillsdale, NJ: Erlbaum.

Barron, G., & Erev, I. (2003). Small feedback-based decisions and their limited correspondence to description-based decisions. Journal of Behavioral Decision Making, 16(3), 215–233. doi:10.1002/bdm.443

Bechara, A., Damasio, A. R., Damasio, H., & Anderson, S. W. (1994). Insensitivity to future consequences following damage to human prefrontal cortex. Cognition, 50(1–3), 7–15. doi:10.1016/0010-0277(94)90018-3

Birnbaum, M. H. (2008). New paradoxes of risky decision making. Psychological review, 115(2), 463. doi:10.1037/0033-295X.115.2.463

Bishop, C. M. (2006). Pattern Recognition and Machine Learning. New York, NY: Springer.

Busemeyer, J. R., & Diederich, A. (2010). Cognitive modeling. Sage.

Busemeyer J. R., Myung, I. J. (1992). An adaptive approach to human decision making: Learning theory, decision theory, and human performance. Journal of Experimental Psychology: General, 121(2), 177–194. doi:10.1037/0096-3445.121.2.177

Busemeyer, J. R., & Stout, J. C. (2002). A contribution of cognitive decision models to clinical assessment: decomposing performance on the Bechara gambling task. Psychological Assessment, 14(3), 253–262.doi:10.1037//1040-3590.14.3.253

Busemeyer, J. R., & Wang, Y. (2000). Model comparisons and model selections based on the generalization criterion methodology. Journal of Mathematical Psychology, 44(1), 171–189. doi:10.1006/jmps.1999.1282

Bush, R. R., & Mosteller, F. (1955). Stochastic models for learning. Oxford, England: Wiley & Sons.

Brandstätter, E., Gigerenzer, G., & Hertwig, R. (2006). The priority heuristic: making choices without trade-offs. Psychological Review, 113(2), 409–432. doi:10.1037/0033-295X.113.2.409

Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011). Model-based influences on humans’ choices and striatal prediction errors. Neuron, 69(6), 1204–1215. doi:10.1016/j.neuron.2011.02.027

Denrell, J. (2007). Adaptive learning and risk taking. Psychological Review, 114(1), 177–187. doi:10.1037/0033-295X.114.1.177

Dutt, V., & Gonzalez, C. (2012). The Role of Inertia in Modeling Decisions from Experience with Instance-Based Learning. Frontiers in Psychology, 3(177). doi:10.3389/fpsyg.2012.001777

Dutt, V. & Gonzalez, C. (2015). Accounting for Outcome and Process Measures and the Effects of Model Calibration. Journal of Dynamic Decision Making, 1(2),1–10. doi:10.11588/jddm.2015.1.17663

Erev, I., & Barron, G. (2005). On adaptation, maximization, and reinforcement learning among cognitive strategies. Psychological Review, 112(4), 912–31. doi:10.1037/0033-295X.112.4.912

Erev, I., Ert, E., Plonsky, O., Cohen, D., & Cohen, O. (2015). From anomalies to forecasts: A choice prediction competition for decisions under risk and ambiguity. Mimeo, 1–56.

Erev, I., Ert, E., Roth, A. E., Haruvy, E., Herzog, S. M., & Hau, R. (2010). A choice prediction competition: Choices from experience and from description. Journal of Behavioral Decision Making, 23(1), 15–47. doi:10.1002/bdm.683

Erev, I., Glozman, I., & Hertwig, R. (2008). What impacts the impact of rare events. Journal of Risk and Uncertainty, 36(2), 153–177. doi:10.1007/s11166-008-9035-z

Estes, W. K., & Todd Maddox, W. (2005). Risks of drawing inferences about cognitive processes from model fits to individual versus average performance. Psychonomic Bulletin & Review, 12(3), 403–408.doi:10.3758/bf03193784

Fox, C. R., & Tversky, A. (1998). A belief-based account of decision under uncertainty. Management Science, 44(7), 879–895.doi:10.1287/mnsc.44.7.879

Frey, R., Mata, R., & Hertwig, R. (2015). The role of cognitive abilities in decisions from experience: Age differences emerge as a function of choice set size. Cognition, 142, 60–80. doi:10.1016/j.cognition.2015.05.004

Gallistel, C. R., Fairhurst, S., & Balsam, P. (2004). The learning curve: implications of a quantitative analysis. Proceedings of the National Academy of Sciences of the United States of America, 101(36), 13124–13131. doi:10.1073/pnas.0404965101

Gigerenzer, G., & Goldstein, D. G. (1996). Reasoning the fast and frugal way: models of bounded rationality. Psychological review, 103(4), 650–669.doi:10.1037//0033-295x.103.4.650

Gilboa, I., & Schmeidler, D. (1989). Maxmin expected utility with non-unique prior. Journal of Mathematical Economics, 18(2), 141–153. doi:10.1016/0304-4068(89)90018-9

Gonzalez, C., & Dutt, V. (2011). Instance-Based Learning: Integrating Sampling and Repeated Decisions From Experience. Psychological Review, 118(4), 523–551. doi:10.1037/a0024558

Gonzalez, C., & Dutt, V. (2012).Refuting data aggregation arguments and how the instance-based learning model stands criticism: A reply to Hills and Hertwig. Psychological Review, 119(4), 893–898.doi:10.1037/a0029445

Hau, R., Pleskac, T. J., Kiefer, J., & Hertwig, R. (2008). The description-experience gap in risky choice: The role of sample size and experienced probabilities. Journal of Behavioral Decision Making, 21(5), 493–518. doi:10.3758/s13423-015-0924-2

Hertwig, R. (2012). The psychology and rationality of decisions from experience. Synthese, 187(1), 269–292. doi:10.1007/s11229-011-0024-4

Hertwig, R., Barron, G., Weber, E. U., & Erev, I. (2004). Decisions from experience and the effect of rare events in risky choice. Psychological Science, 15(8), 534–539. doi:10.1111/j.0956-7976.2004.00715.x

Hertwig, R., & Erev, I. (2009). The description-experience gap in risky choice. Trends in Cognitive Sciences, 13(12), 517–523. doi:10.1016/j.tics.2009.09.004

Hertwig, R., & Pleskac, T. J. (2010). Decisions from experience: Why small samples? Cognition, 115(2), 225– 237. doi:10.1016/j.cognition.2009.12.009

Horrace, R. H., William, C., and Jeffrey, M. P. (2009), Variety: Consumer choice and optimal diversity. Food Marketing Policy, Center Research Report, 115.

Houck, C. R., Joines, J., & Kay, M. G. (1995). A genetic algorithm for function optimization: a Matlab implementation. Ncsuietr, 95(9).

Jakobsen, T. (2010). Genetic algorithms. Retrieved from

Kahneman, D., & Tversky, A. (1979). Prospect theory: An analysis of decision under risk. Econometrica, 47(2), 263–291.doi:10.2307/1914185

Kudryavtsev, A., & Pavlodsky, J. (2012). Description-based and experience-based decisions: individual analysis. Judgment and Decision Making, 7(3), 316–331.

Lebiere, C. (1999). Blending: An ACT-R mechanism for Aggregate retrievals. Paper presented at the 6th Annual ACT-R Workshop at George Mason University. Fairfax County, VA.

Lee, M. D. (2008). Three case studies in the Bayesian analysis of cognitive models. Psychonomic Bulletin & Review, 15(1), 1–15. doi:10.3758/PBR.15.1.1

Lejarraga, T. & Dutt, V. & Gonzalez, C. (2012). Instance-Based Learning: A general model of repeated binary choice. Journal of Behavioral Decision Making, 25(2),143–153. doi:10.1002/bdm.722

Luce, R. D., & Raiffa, H. (1957). Games and Decisions: Introduction and Critical Surveys. New York, NY. Wiley.

March, J. G. (1996). Learning to be risk averse. Psychological Review, 103(2), 309–319. doi:10.1037/0033-295X.103.2.309

Marchiori, D., Di Guida, S., & Erev, I. (2015). Noisy retrieval models of over-and under sensitivity to rare events. Decision, 2(2), 82–106. doi:10.1037/dec0000023

Mathworks. (2012). MATLAB and Statistics Toolbox Release 2012b [Computer software]. Natick, Massachusetts, United States: The MathWorks, Inc.

Plonsky, O., Teodorescu, K., & Erev, I. (2015). Reliance on small samples, the wavy recency effect, and similarity-based learning. Psychological Review, 122(4), 621–647. doi:10.1037/a0039413

Rieskamp, J. (2008). The probabilistic nature of preferential choice. Journal of Experimental Psychology: Learning, Memory, and Cognition, 34(6), 1446–1465. doi:10.1037/a0013646

Rouder, J. N., & Lu, J. (2005). An introduction to Bayesian hierarchical models with an application in the theory of signal detection. Psychonomic bulletin & review, 12(4), 573–604. doi:10.3758/BF03196750

Shteingart, H., Neiman, T., & Loewenstein, Y. (2013). The role of first impression in operant learning. Journal of Experimental Psychology: General, 142(2), 476–488. doi:10.1037/a0029550

Stevens, L. (2016, June 8). Survey Shows Rapid Growth in Online Shopping. The Wall Street Journal. Retrieved from

Sutton, R. S., & Barto, A. G. (1998). Reinforcement Learning: An Introduction (Vol. 1, No. 1). Cambridge: MIT press.

Tversky, A., & Kahneman, D. (1986). Rational choice and the framing of decisions. Journal of business, 59(S4), S251–S278. doi:10.1086/296365

Tversky, A., & Kahneman, D. (1992). Advances in prospect theory: Cumulative representation of uncertainty. Journal of Risk and Uncertainty, 5(4), 297–323.doi:10.1007/bf00122574

Tversky, A., & Fox, C. R. (1995). Weighing risk and uncertainty. Psychological Review, 102(2), 269–283. doi:10.1037/0033-295X.102.2.269

Aggregate choice, individual choice, sampling paradigm, decisions from experience, computational models, likelihood