Editorial Policy Statement on Numerical and Experimental Accuracy

The purpose of this statement is to improve the quality of computational and experimental investigations published in the AIAA journals. In January 1994 the AIAA journals published their first statement regarding the accuracy of numerical solutions and experimental results. This statement has been in effect since that time and will remain so in the future:

The AIAA journals will not accept for publication any manuscript reporting (1) numerical solutions of an engineering problem that fails to adequately address the accuracy of the computed results or (2) experimental results unless the accuracy ofthe data is adequately presented.

To aid prospective authors submitting manuscripts to the AIAA journals and to aid Editors-in-Chief, Associate Editors, and reviewers of the AIAA journals, additional guidance concerning numerical and experimental accuracy is now provided. The implementation of this policy statement will be at the discretion of the Editor-in-Chief of each journal and the Associate Editor evaluating the submitted manuscript.

 Numerical Accuracy

Many technical issues impact the accuracy of a mathematical model of a physical process or system of interest. This portion of the policy statement deals with only one issue: the numerical solution accuracy of the assumed mathematical model that is solved using numerical algorithms that are implemented in software and solved on a computer. Other issues, such as the assumptions of the mathematical model, appropriateness of the initial and boundary conditions, and the accuracy of the simulation results with respect to experimental data, are considered in the decision to accept or reject a manuscript for publication. However, these issues are not the topic of this policy statement. The intent of this policy statement is to improve the credibility and reproducibility of the numerical aspects of the simulation results.

Authors should address the following criteria, as applicable, in summary form in the manuscript and can provide additional detailed information in referenced documents:

1. Statement of Numerical Methods: Authors should be clear and precise in the description of all important numerical methods used in the investigation. For example,in the numerical solution of partial differential equations (PDEs), the author should state the formal accuracy of the numerical method for interior points as well as the formal accuracy of the numerical boundary conditions. If the specific numerical method used is one of a class of methods, then all pertinent information, such as specific values of adjustable parameters in the method, should be given or referenced.


2. Minimum Formal Accuracy of Numerical Methods: Numerical methods for solving PDEs should be at least formally second-order accurate in space for spatially smooth solutions. For non-smooth solutions, such as solutions with shock waves and singularities, spatially first-order methods are appropriate. Temporal difference methods for initial value problems are recommended to be at least formally second-order accurate.


3. Statement of Code Verification Activities: For computer codes that numerically solve PDEs, some level of verification testing of the code should have been demonstrated. For example, comparisons should have been made to analytical solutions or highly accurate numerical solutions. However, because of manuscript length requirements, it is recommended that references be given to documentation of the verification testing of the code. If commercially available computer codes are an important element in the simulations presented, references for publicly available codetesting should be given.


4. Spatial Convergence Accuracy: Spatial discretization errors in the solution of PDEs should be quantified. Procedures typically involve the use of multiple refined spatial grids or procedures that use different order-of-accuracy methods on a single grid. It is recommended that a posteriori error estimation procedures be used, as opposed to a priori error estimation methods. An alternative, but less precise, method for indicating spatial discretization errors is to show sensitivity of important dependent variables or solution functionals on multiple refined spatial grids.


5. Temporal Convergence Accuracy: Temporal discretization errors should be quantified by either direct control of estimated local time-step error or by multiple solutions of an initial value problem, each with a significantly different time-step. In direct control of the local time-step, as is commonly done in the numerical solution of ordinary differential equations, the author should state how the error was controlled and the magnitude of the estimated local time-step error. When using the technique of comparing solutions obtained with different time-steps, the sensitivity of important dependent variables or solution functionals to the time-step should be addressed.


6. Iterative Convergence Accuracy: The accuracy of convergence of important iterative schemes in the numerical solution should be addressed. Examples of iterative schemes are iterations needed to advance to a new time-step and iterations needed in the solution of a nonlinear boundary-value problem. Iterative convergence criteria should be based on a relative error estimate of the quantity of interest, not an absolute error estimate. For the case of iterative solution of a nonlinear matrix equation, iterative convergence should be based on an appropriate matrix norm of the error.

For more detailed discussion of the above topics, authors should consult modern texts and published articles dealing with code verification testing, numerical algorithms, and numerical solution error estimation.

 Experimental Accuracy

The accuracy of experimental results is concerned with how well the specified measurands in the manuscript have been determined, and the statistical confidence with which they can be assessed. The appropriateness of the measurands for describing the physical phenomena and for comparing the results with numerical investigations is not part of this policy statement. The intent of the policy statement is to encourage authors to provide sufficient information for readers to independently assess the statistical confidence of results presented in AIAA journal articles. Authors should address the following criteria, as applicable, in summary form in the manuscript and can also provide additional detailed information in referenced documents:

1. Statement of Measurement Techniques and Apparatus: Authors should be clear and precise in the description of the test articles and facilities, test methods, and instrumentation, including the data reduction techniques.


2. Statement of Experimental Design: To assist others in assessing the uncertainty in the experimental results, for such purposes as reproducing them experimentally or simulating them analytically and for identifying corresponding statistical techniques to analyze the results, the experimental conditions and order of acquisition should be presented or concisely described.


3. Estimation of Uncertainty: Unexplained variance in results should be addressed as follows:


1. Estimation of Bias Uncertainty: The bias uncertainty of the results (systematic component of the unexplained variance) should be estimated. In addition, the method of estimation should be described.


2. Estimation of Precision Uncertainty: The repeatability of the results (random component of the unexplained variance) should be quantified. Also, the method of determining the repeatability should be specified. Unusual results, whether explained or unexplained, should be identified.


3. Statement of Total Uncertainty: The total uncertainty is the combination of the bias and precision uncertainties and should be presented with the experimental results (in text, tables, or graphs as appropriate). Authors should identify any known potential sources of bias and precision uncertainty that have not been explicitly included in the estimates of total uncertainty.
4. Statistical Analysis: Statistical analysis may be warranted in a number of cases, such as when comparing the effects of independent variables, when correlating experimental conditions and measures, and when performing hypothesis tests.


1. Description of Methodology: The choice of statistical analysis method should be clearly described and justified relative to a priori knowledge, the experimental design, and the characteristics of the measurements.


2. Treatment of Impact: The findings of the statistical tests should be carefully substantiated and discussed relative to their confidence level, parametric assumptions, and experiment design.
5. Coverage Factor: The coverage factor used for all expanded uncertainty estimates should be specified. That is, the author(s) should associate a specific level of confidence with each uncertainty value, by estimating the probability that reported error limits will encompass the true result.

For more detailed discussion of the above topics, prospective authors should consult modern texts in measurement methods, experimental design, uncertainty analysis, hypothesis testing, and statistical data analysis.