Most data employed in simulation modeling are estimates of the true parameters and, therefore, have an associated uncertainty. Error budgets can be used to assess the quality of the overall simulation system. Although progress has been made in the areas of uncertainty analysis and error budgets, there is a need to develop more advanced statistical and computational tools. These tools will enable model users to jointly assess and quantify the sources and magnitude of errors associated with large-scale Department of Defense (DoD) simulation models used for resource assessment and management.

With the growing importance of simulation modeling in natural and cultural resource assessment and management, the DoD recognizes the need for a comprehensive framework to analyze uncertainty in simulation results. Over four years, this project will develop the methodological framework to conduct this uncertainty assessment.

Example of Modeling System with Error

Research has begun by developing a Geographic Information System-based methodology to make spatial and temporal predictions, analyze uncertainty, and build error budgets for soil erosion status based on and applied to military training. Through this methodology, a grid-based database is generated containing a digitized elevation model and soil, rainfall, and vegetation maps (models). Spatial and temporal predictions of soil loss are made at different optimal operational scales. Using sensitivity techniques, spatial sensitivity of the input parameters on the prediction are analyzed. The error budget for the whole population, or a homogeneous sub-area, is determined by applying both analytical and Monte Carlo approaches. In addition, spatial error distributions and patterns are identified and quantified using geostatistical techniques, literature surveys, interviews, statistical methods, previous studies, and field studies, if necessary. Once these errors are estimated, component models will be developed to represent these errors and design specifications for the portable software package that will be created.

A detailed analysis of a case study in Fort Hood, Texas, was begun. Significant progress was made in both identifying the errors associated with the inputs and designing approaches to obtain that error information when it was unavailable. Researchers began the process of identifying errors in simulation systems. Currently, 100 different sources of error have been identified. Scientists also began to develop a formal framework for error analysis of large spatiallyexplicit simulation models, which also is being applied to the Fort Hood case.

This project will provide the rationale to account for the effect of different sources of error on the uncertainty of model predictions and the means for efficiently reducing this uncertainty.