Content Evidence
Content relevance and representativeness, narrowly defined, refers to the quality of the sample of content from a specific content domain. It is based on professional judgments about the test content and the content domain. For this aspect of validity, the Survey of the Natural Sciences on the DAT reflects how well the test items cover areas of general biology, and general and organic chemistry typically contained in the undergraduate curriculum. For the Dental Admission Test battery, this aspect of validity is assessed primarily by the evaluation and judgment of its test construction committee members, who are subject matter experts. The committee members judge the appropriateness, the relevance, and the representativeness of the test content based on what is being taught in predental courses. The validity of the Reading Comprehension Test is based on the judgment of dental faculty, who write reading passages and items representative of reading material encountered in the first year of dental school.
External Correlational Evidence
External correlational evidence is also investigated to determine the extent to which an individual's future level on various criteria is predicted from prior test performance. It is desirable that the test be related to future performance in dental school and later on the National Board Dental Examinations (American Dental Association, 2009). The performances on these external variables can provide some evidence of the relationship between the test and different methods for measuring the same and distinct constructs or traits (Messick, 1989, pp. 16-46).
Evidence indicating the validity of the DAT is available from two sets of criteria: dental school performance and performance on the National Board Dental Examinations. Each year the relationships between DAT scores and first year dental school grades are analyzed by means of Pearson product moment correlations. Table 11 indicates the percentage of dental schools whose first year grades (2007-2008) have significant positive correlations with quantified admission criteria. Table 12 indicates the median correlation coefficients obtained between 2007-2008 first year dental school grades and admission criteria. Both Tables 11 and 12 indicate that, for the most part, Dental Admission Test scores have a significant positive relationship to performance in the first year of dental school.
In most cases, the DAT Academic Average and Total Science score have a stronger relationship with first year performance than predental grade point averages. As indicated in Table 12, multiple regression using the individual DAT test scores (Quantitative Reasoning, Reading Comprehension, Biology, General Chemistry, Organic Chemistry, and Perceptual Ability) result in better prediction of first year GPA than the Academic Average. Multiple regression using individual DAT scores, predental GPA, and predental science GPA result in the best combination of predictors for first year GPA. This multiple R, on the average, almost doubles the amount of explained variance of the best single predictor of dental school first year GPA. The Perceptual Ability Test scores have the strongest relationship with technique performance as compared with the other variables. See Kramer (1986) for a further discussion of the validity of the DAT.
In addition to the validity of the DAT in relation to dental first- and second-year performance, the DAT has provided correlational information periodically between DAT and National Board Dental Examinations scores. There is consistent evidence, as seen in Table 13, that DAT scores have a significant positive relationship with the scores on the National Board Dental Examinations. This relationship provides evidence that the DAT is valid for predicting performance beyond the first year.
The construct validity of the DAT battery is in part evaluated by the Perceptual Ability Test (PAT). The PAT was constructed in the belief that perceptual abilities are important requirements for successful completion of the technique courses in dental school, and that perceptual abilities have a direct relationship to a person's measure of eye-to-hand fine motor coordination, also required in the curriculum. The reader is referred to the research conducted by Kramer, Kubiak, and Smith (1989) and Kramer and Kubiak (1990) for discussions of the validity of the PAT in the DAT battery.
Reliability Evidence
Reliability is a primary type of validity evidence. It is often defined as the precision or consistency of the test scores. The assumption that the DAT is a sufficiently precise instrument to permit meaningful descriptions of the abilities measured is based on the reliability of the scores. The reliability, or internal consistency, of the scores traditionally is evaluated by the Kuder-Richardson Formula 20 (KR20). The reliability coefficients of the scores derived from the four tests are as follows: Quantitative Reasoning Test (40 items), 0.79; Reading Comprehension Test (50 items), 0.81; Survey of the Natural Sciences Test (100 items), 0.93; and Perceptual Ability Test (90 items), 0.90. These reliability coefficients are well within the acceptable range and are typical of standardized tests and of the DAT battery used in this program since the early 1970s. Also, it has been customary to report some of the descriptive statistics from the test results. Table 10 shows the means and standard deviations for the 2009 DAT.
Hopefully I won't have to care about it in 39 days.
