Validating the accuracy of simulations is challenging. We use the results of validation to further refine our simulation models. Being able to discern the effects of different simulation elements and how they affect each other is an important part of this process. If you cannot isolate aspects of a simulation and their effects, you cannot determine what parts of the simulation require refinement.
Conflict simulation has been applied in many different fields, including economics, political science, history, diplomacy, military strategy, and even pure entertainment in the form of games. Unlike physical simulations, conflict simulations usually cannot be validated by conducting experiments in a laboratory environment. Therefore, it becomes very important to interpret the results of a simulation with qualifications rather than accept them as fact. As in weather prediction, where multiple models are applied in search of a convergence, accurate conflict simulation requires the application of multiple models, or at least the same model with different values applied to parameterized elements. For example, as with the weather, the outcome of a conflict can be very sensitive to initial conditions. Varying initial conditions, as well as other tunable parameters (e.g., strategy, logistics, weapon effects, intelligence accuracy, and so on) will provide a range of results that must be interpreted carefully to arrive at any conclusions.
The goals of a simulation can vary greatly. A commercial war game may stress playability and enjoyment over historical accuracy and realism, whereas a military simulation may stress the teaching of basic tactics. Whatever the goal of a conflict simulation may be, designing an effective simulation model requires an understanding of the interplay between simulation components. The current state of conflict simulation design is not very different from 30 years ago. Commercial war game development is a good example. Commercial war games rely primarily on playtesting to refine their simulation models. All games are undertested and all games are flawed. Sometimes the flaws are irrelevant, but in other cases they are severe. The problem with conflict simulation design today is that it is limited by the human scale of time and space. Only so many tests can be performed on a game that takes 10 hours for two human players to complete.
It is far too difficult for a simulation designer to determine the effects of a design decision. For example, if you have three different possible combat resolution models and want to evaluate each for historical accuracy, you would have to conduct hundreds, if not thousands, of simulation runs utilizing each model under different conditions. This type of testing is not currently performed. I'd like to develop a framework for assisting the design, development, validation, and execution of conflict simulations—CASD (computer-assisted simulation design). The idea is to build tools that help designers create better simulations, whether their goal is to increase playbility or achieve more realistic results.
