Model Validation Examples
The evaluate the "quality" of model results, as a basis for public information or as the
basis for regulatory policy such as permiting or emission control measures and policies,
the quality of results against observational data is analyzed.
Model validation measures the agreement between model generated and observed
concentration values (e.g., AIAA 1998; Macal, 2005; Sargent, 2003).
The new consolidated European air quality framework Directive 2008/50/EC (replacing 1999/30/EC)
defines acceptable model performance as within 50% plus or minus hourly observation data.
However, the comparison of model results and monitoring data is difficult due to the every different nature
if not incommensurability of these data: continuous point measurements of a highly dynamic
process involving turbulent mixing and high spatial variability, and the hourly average
over a large volume of air. The direct comparison of model results and monitoring data therefore
requires careful consideration of sampling statistics. Alternative monitoring data for
model validation can be derived from large scale synoptic observations, such as aerosol
optical density from satellite imagery. While covering large areas in space,
they are comparably infrequent, and describe the entire vertical atmosphere,
while the monitoring data only sample the bottom layer.
Model validation is often controversial but the display and analysis of model
generated results together with matching observational data is at the very least,
good practive, and most often, instructional. While it may be rather naive
to expect perfect correspondence at high spatial and temporal resolution,
aggregate results and trends should certainly "correspond".
Model are tools; they will answer "well posed" (adequate) questions, considering uncertainty and errors
in all inputs, the model itself, the observations; sensitivity analysis, error propagation, and asking the
"meaningful questions" is therefor essential for meaningful answers.
It is important to remember that a model is a "complex hypothesis" that includes the
assumption of "known inputs" which are primarily the emission data
hardly ever known completely or precisely, especially
with regard to the temporal variations - and boundar conditions (initial concentrations,
imports) which are even explicitly ignored in the case of Gaussian steady state models !
So the basis concept should read:
IF all the assumptions made (including inital and boundary conditions) are correct,
THEN (and only then) these are the expected concentrations, to be compared with measurements
IF (and only if) the measurements are also correct and commensurable.
A compilation of typical "validation" examples of 3D dynamic model results (MM5, WRF, CAMx)
against observation data from selected application: Lower Austria, Cyprus, Malta, Brazil, Korea, Iran
shows a set of thumbnails, where each of the small images constitutes a link to the full sized screen dump.
| print page