Lessons from a comprehensive validation of an agent based-model: The experience of the Pampas Model of Argentinean agricultural systems

Published in Ecological Modelling, v. 273:284-298
Authors

Bert, F.E., Rovere, S.L., Macal, C.M., North, M.J. and Podestá, G.P.
Publication year 2014
DOI https://doi.org/10.1016/j.ecolmodel.2013.11.024
Affiliations
  • Facultad de Agronomía, Universidad de Buenos Aires &ndash CONICET, Av. San Martín 4453, Buenos Aires, Argentina
  • Facultad de Ingeniería, Universidad de Buenos Aires, Av. Las Heras 2214, Buenos Aires, Argentina
  • Argonne National Laboratory, 9700 S Cass Avenue, Argonne, IL 60439, USA
  • Rosenstiel School of Marine and Atmospheric Science, University of Miami, 4600 Rickenbacker Causeway, Miami, FL 33149, USA
IAI Program

CRN3
IAI Project CRN3035
Keywords
Agent-based model
Validation
Agriculture
Land use
Land tenure

Highlights

•The Pampas Model (PM) is an agent-based model of agricultural systems in Argentina.

•We present the validation process of PM and lessons that emerged from this process.

•PM reproduced successfully the observed dynamics of structural and land use changes.

•Our experience highlights the need for complementing multiple validation strategies.

Abstract

There are few published examples of comprehensively validated large-scale land-use agent-based models (ABMs). We present guidelines for doing so, and provide an example in the context of the Pampas Model (PM), an ABM aimed to explore the dynamics of structural and land use changes in the agricultural systems of the Argentine Pampas. Many complementary strategies are proposed for validation of ABM's. We adopted a validation framework that relies on two main streams: (a) validation of model processes and components during model development, which involved a literature survey, design based on similar models, involvement of stakeholders, and focused test scenarios and (b) empirical validation, which involved comparisons of model outputs from multiple realistic simulations against real world data. The design process ensured a realistic model ontology and representative behavioral rules. As result, we obtained reasonable outcomes from a set of initial and simplified scenarios: the PM successfully reproduced the direction of the primary observed structural and land tenure patterns, even before calibration. The empirical validation process lead to tuning and further development of the PM. After this, the PM was able to reproduce not only the direction but also the magnitude of the observed changes. The main lesson from our validation process is the need for multiple validation strategies, including empirical validation. Approaches intended to validate model processes and components may lead to structurally realistic models. However, some kind of subsequent empirical validation is needed to assess the model's ability to reproduce observed results.