|
One of the quality control steps most frequently performed on databases is the database audit, an exercise in which source documents (typically CRFs and query resolution forms) are compared to a study database, usually as part of database lock. A tremendous amount of time, energy and resources are devoted to this typically critical-path activity, and the results are treated with much reverence. It is unclear, however, what those results actually signify in terms of data quality. This project will address the following questions:
1. What question are we asking when we perform a database audit?
2. Is this question of value and relevance? If not, is there a related question that is of value and relevance?
Assuming that a question can be formulated that relates to the quality (a term that needs to be clearly defined) of a final database in its entirety:
3. What (quantitative) level of quality of the final database is considered acceptable, and why?
4. Assuming that it is desirable (for reasons of time, cost and resource) to investigate only a sample of the database:
i) What sampling method(s) would be considered “ideal” for addressing the question at hand?
ii) What level of confidence should we use when generalizing from the sample to the population, and why?
iii) How should the sample size be calculated?
iv) How should the sample be drawn?
5. Is the “ideal” methodology and sample size achievable in practice? If not:
i) Are there “less-than-ideal” sampling plans described in the statistical literature that are more practical?
ii) If so, what is the “cost” (bias, increased sample size) of using them?
iii) If not, is it possible to estimate the effect of using practical sampling plans on the generalizability of results from the sample to the population?
Assuming that the above leads to a practical, relevant definition of a database audit estimating some form of error rate, the numerator (number of errors) and the denominator (number of elements that could have been in error) must be defined in line with the process being examined.
|
