DAT Harder/Easier-The Myth Revisited

This forum made possible through the generous support of SDN members, donors, and sponsors. Thank you.

doc toothache

Full Member
15+ Year Member
Joined
Jan 17, 2006
Messages
8,514
Reaction score
2,792
A few SDNers have been insisting that the 07 DAT in particular has gotten harder. The ADA has recently released the DAT scores for July 06 to June 07. Purists will have to wait another six months for the 07 verdict. The raw data used in this analysis was obtained from www.ada.org/prof/ed/testing/dat/dat_score_frequency.pdf and from www.ada.org/prof/ed/testing/dat/dat_users_manual.pdf . Manipulation or such a large amount of data invariably leads to error. If noted, prompt corrections will be made.

Only DAT scores of 17 and above were used. The format was changed to provide the percentage of applicants that scored above a given number. For convenience the scores representing 1 and 2 standard deviations are included. It should be remembered that only 15.9% are expected to have a score above 1 std. deviation and 2.3% above 2 std. dev. (http://en.wikipedia.org/wiki/Image:Normal_distribution_and_scales.gif)

Trend
The preliminary 07 data does not support the assertion that the DAT has gotten harder. The mean scores for all sections have remained virtually unchanged.

Are my DAT scores competitive?
The minimum score, in general, needs to be at least at the mean for the schools in question. To be more competitive the score should be at least 1 point above the mean for the school that applicant is considering.
The usual advice on this forum is to shoot for 21 and above for an AA score. There is nothing wrong with shooting for the stars, provided we are prepared for what will likely be a devastating crash back to earth. We should recognize that a score of 21 is a reasonable expectation for only 10% of the applicants.

Should I retake the DAT?
If the score is 1 point below the mean, consideration should be given to retaking, provided there will be a drastic change in the approach in preparing for the test. Statistically, retakes are usually a lose-lose situation. Repeaters usually score from 0.25 to 1 point lower the 2nd…3rd….nth time around (see dat users manual 2006).

What materials should I use?
We may be relying too heavily on limited resources such as Kaplan, topscore, achiever, destroyer. If the scores are low, serious consideration should be given to using more comprehensive review sources.

The Blame Game
Declaring that the DAT has gotten harder may serve to assuage our fragile egos and this may be sufficient reason for blaming the ADA for our misfortune. It could not be that we were ill prepared for the exam or that we are woefully unprepared in the basic sciences. An even more difficult reality to accept is that our inability to attain a higher score on the DAT is beyond our control. We are subject to the laws of probability and we cannot escape the distribution we find under the Bell curve. At best, a collective increase in preparation will only lead to a shift of the mean to the right without any changes in the percent distribution. One OP suggested that the harder DAT is the result of the test makers doing their job well. Nothing could be further from the mandate of the ADA test construction committee. The DAT is designed to be valid and reliable. In ADA lingo a test is valid if it accurately tests the basic knowledge of an applicant and it is deemed reliable when they provide consistently accurate measurements enabling adcoms to compare applicants year after year. If the test were indeed harder one year than another, there would utter chaos. Adcoms would no longer able to look at the DAT scores across the board. They would have to devise formulas to standardize the scores based upon the year and month the applicant took the test. This is just not happening.

www.ada.org/prof/ed/testing/construction/dat_selection.pdf
www.ada.org/prof/ed/testing/construction/dat_manual.pdf


Code:
	 QR Frequency Tables				

	1995	2000	2005	2006	2006/07

17	42.4	50.3	42.7	49.5	46.6
18	30.4	36.2	31.4	37.2	34.2
19	21.8	26.0	22.0	27.0	24.1
20	14.0	18.0	15.3	18.2	15.0
21	9.0	11.4	10.5	13.0	10.6
22	6.3	8.3	7.1	8.8	6.7
23	4.1	5.5	4.7	5.9	4.2
24	2.5	3.2	3.2	4.0	2.8
25	1.5	1.8	2.6	3.4	2.4
26	1.2	1.2	2.0	2.6	1.8
27	0.6	0.6	0.9	1.3	0.9
28	0.6	0.4	0.9	1.3	0.9
29	0.0	0.2	0.7	0.3	0.6
30	0.0	0.0	0.0	0.0	0.0
					
Mean	16.15	16.73	16.34	16.83	16.58
Std dev	3.30	3.27	3.37	3.48	3.26
1 s.d.	19.45	20.00	19.71	20.31	19.84
2 s.d.	22.75	23.27	23.08	23.79	23.10

	10259	7231	12528	8936	9261


	RC				
	Frequency Tables				

	1995	2000	2005	2006	2006b

17	72.5	84.5	81.1	83.7	81.6
18	55.2	72.7	70.9	74.6	70.6
19	40.7	59.8	57.8	62.1	56.1
20	28.1	46.6	42.5	46.7	40.4
21	16.0	32.5	29.7	33.6	27.7
22	11.3	22.1	17.5	19.5	15.6
23	6.2	14.1	11.1	12.5	9.7
24	3.3	7.7	6.4	7.2	5.5
25	2.9	4.6	3.3	3.8	2.8
26	1.4	2.4	1.6	2.0	1.5
27	1.2	1.8	1.1	1.3	0.9
28	0.3	0.4	0.3	0.3	0.2
29	0.3	0.4	0.3	0.3	0.2
30	0.0	0.0	0.0	0.0	0.0

Mean	18.13	19.35	19.03	19.31	18.94
Std dev	2.78	2.94	2.89	2.87	2.78
1 s.d.	20.91	22.29	21.92	22.18	21.72
2 s.d.	23.69	25.23	24.81	25.05	24.5

	10259	7231	12528	8936	9261


	Bio				
	Frequency Tables				

	1995	2000	2005	2006	2006b

17	43.6	61.0	61.9	67.0	64.2
18	29.9	45.5	46.2	52.8	19.9
19	18.5	31.6	32.0	39.3	35.4
20	11.8	21.0	19.0	25.6	22.9
21	5.9	11.0	10.6	15.1	13.1
22	3.4	6.7	6.7	10.1	8.3
23	1.8	3.5	3.3	5.2	4.3
24	0.6	1.9	1.6	2.6	2.1
25	0.1	0.8	1.0	1.8	1.3
26	0.1	0.8	0.5	0.8	0.7
27	0.0	0.1	0.3	0.5	0.4
28	0.0	0.1	0.2	0.4	0.3
29	0.0	0.0	0.0	0.1	0.1
30	0.0	0.0	0.0	0.0	0.0

Mean	16.27	17.32	17.35	17.77	17.59
Std dev	2.66	2.74	2.67	2.84	2.76
1 s.d.	18.93	20.06	20.02	20.61	20.35
2 s.d.	21.59	22.8	22.69	23.45	23.11

	10259	7231	12528	8936	9261

	GC				
	Frequency Tables				

	1995	2000	2005	2006	2006b

17	46.6	55.0	56.8	63.3	61.8
18	33.4	44.7	46.2	53.3	50.8
19	21.6	33.6	34.7	41.5	38.2
20	15.1	23.2	23.2	29.9	26.9
21	11.0	17.5	15.6	21.0	18.6
22	7.0	10.8	10.1	14.1	12.5
23	4.1	7.3	8.3	11.7	9.8
24	1.9	5.1	4.3	5.8	4.9
25	1.9	4.4	4.3	5.8	4.7
26	0.6	1.6	1.9	2.9	2.2
27	0.6	1.3	1.3	2.0	1.5
28	0.6	0.5	0.8	1.1	0.9
29	0.0	0.0	0.8	1.1	0.9
30	0.0	0.0	0.0	0.0	0.0

Mean	16.4	17.27	17.35	17.90	17.76
Std Dev	3.20	3.50	3.49	3.64	3.46
1 s.d.	19.24	20.77	20.84	20.73	21.22
2 s.d.	22.42	24.27	24.33	24.37	24.68

	10269	7231	12528	8936	9261


	Org				
	Frequency Tables				

	1995	2000	2005	2006	2006/07

17	45.0	57.2	54.8	60.7	60.8
18	34.8	45.3	43.3	49.8	49.0
19	24.1	35.4	31.8	38.9	37.5
20	16.3	25.0	22.4	28.9	27.0
21	10.0	18.6	16.1	21.2	18.8
22	6.3	12.4	10.2	14.1	12.5
23	5.4	8.8	7.4	10.4	8.6
24	3.1	5.9	5.1	7.3	5.8
25	2.5	5.0	4.2	5.8	4.8
26	1.3	2.0	1.5	2.5	2.1
27	0.8	1.7	1.5	2.5	1.9
28	0.2	0.0	1.1	1.9	1.6
29	0.0	0.0	0.7	1.1	1.0
30	0.0	0.0	0.0	0.0	0.0

Mean	16.29	17.34	17.01	17.58	17.50
Std Dev	3.44	3.66	3.73	3.95	3.74
1 s.d.	19.73	21.00	20.74	21.53	21.14
2 s.d	23.17	23.66	24.17	25.48	24.88

	10264	7231	12528	8936	9261


	TS				
	Frequency Tables				

	1995	2000	2005	2006	2006/07

17	45.4	59.9	59.8	65.0	63.5
18	30.4	44.2	44.8	52.0	49.4
19	19.4	31.2	30.8	39.2	36.0
20	11.7	20.2	19.1	27.0	23.2
21	5.6	11.9	11.1	16.4	13.6
22	3.0	6.5	5.8	9.1	7.4
23	1.3	3.3	2.9	4.9	3.8
24	0.6	1.3	1.3	2.4	1.7
25	0.3	0.6	0.5	0.9	0.6
26	0.1	0.2	0.2	0.5	0.3
27	0.0	0.1	0.1	0.1	0.0
28	0.0	0.0	0.1	0.1	0.0
29	0.0	0.0	0.1	0.1	0.0
30	0.0	0.0	0.0	0.0	0.0

Mean	16.36	17.25	17.21	17.69	17.54
Std dev	2.54	2.72	2.72	2.89	2.76
1 s.d.	18.90	20.07	19.93	20.58	20.30
2 s.d.	21.44	22.79	22.65	23.47	23.06
					
	10269	7231	12528	8936	9261

	AA				
	Frequency Tables				

	1995	2000	2005	2006	2006b

17	49.8	65.3	63.2	69.7	67.9
18	34.3	50.0	47.2	54.8	51.7
19	20.7	34.6	31.3	40.0	35.8
20	12.2	22.4	19.8	27.0	22.8
21	6.3	13.6	11.6	16.4	13.0
22	3.0	7.1	6.5	9.3	7.1
23	1.2	14.1	3.0	4.8	3.7
24	0.4	1.3	1.3	2.1	1.7
25	0.1	0.3	0.4	0.8	0.7
26	0.0	0.1	0.1	0.2	0.2
27	0.0	0.0	0.0	0.1	0.1
28	0.0	0.0	0.0	0.0	0.0
29	0.0	0.0	0.0	0.0	0.0
30	0.0	0.0	0.0	0.0	0.0

Mean	16.64	17.60	17.42	17.88	17.68
Std dev	2.43	2.56	2.57	2.70	2.55
1 s.d.	19.07	20.16	20.09	20.58	20.23
2 s.d.	21.5	22.72	22.66	23.28	22.78

	10269	7231	12528	8936	9261

		PA			
		Frequency Tables			

	1995	2000	2005	2006	2006b

17	40.6	60.2	60.2	64.2	65.9
18	29.1	46.8	46.2	50.2	52.5
19	17.6	31.8	32.3	36.6	38.8
20	10.6	20.8	21.6	24.7	26.3
21	5.6	11.7	10.7	13.0	14.6
22	3.9	6.9	5.3	7.0	7.8
23	1.4	3.0	2.0	2.9	3.8
24	0.6	1.3	0.8	1.3	1.9
25	0.1	0.7	0.4	0.6	0.9
26	0.1	0.4	0.1	0.2	0.3
27	0.1	0.2	0.1	0.2	0.2
28	0	0.1	0.0	0.0	0.0
29	0	0.0	0.0	0.0	0.0
30	0	0.0	0.0	0.0	0.0

Mean	15.96	17.19	17.19	17.44	17.61
Std dev	2.83	2.85	2.79	2.86	2.89
1 std dev	18.79	20.04	19.98	20.30	20.50
2 std dev	21.62	22.91	22.77	23.16	23.49

	10268	7231	12528	8936	9261

Members don't see this ad.
 
Last edited:
What I would like to see is when they release all of 2007 stats alone. But what would have been more useful here is to see a month-by-month breakdown. Even so, though, it does seem that the AA has gone down by about 0.3 or so, not insignificant in my opinion and certainly statistically valid considering the sample size.
 
What I would like to see is when they release all of 2007 stats alone. But what would have been more useful here is to see a month-by-month breakdown. Even so, though, it does seem that the AA has gone down by about 0.3 or so, not insignificant in my opinion and certainly statistically valid considering the sample size.



It is unclear why having the month-to-month breakdown would be of particular importance. Let us look at a possible scenario. Let us say we have the scores for two consecutive months. In the first month the applicant pool is composed strictly of the lower scoring group with a mean in the teens. In the second month we find them in the higher end with scores in the 20’s. Since in the first group we will have low mean scores the conclusion would be that the test was hard. In the second group, with high mean scores we have to conclude that the test was easier. Yet, we are dealing with the same exam.

It may well be that a 0.2 point difference in the mean AA score is statistically significant. Even if that were the case, the difference would does not necessarily support the hypothesis that the test was harder/easier. It is conceivably that the small fluctuations in the mean scores are more a function of the difference in how well prepared the applicants (quality) are and less a function of whether the exam was harder or easier.
 
Members don't see this ad :)
It is unclear why having the month-to-month breakdown would be of particular importance. Let us look at a possible scenario. Let us say we have the scores for two consecutive months. In the first month the applicant pool is composed strictly of the lower scoring group with a mean in the teens. In the second month we find them in the higher end with scores in the 20’s. Since in the first group we will have low mean scores the conclusion would be that the test was hard. In the second group, with high mean scores we have to conclude that the test was easier. Yet, we are dealing with the same exam.

It may well be that a 0.2 point difference in the mean AA score is statistically significant. Even if that were the case, the difference would does not necessarily support the hypothesis that the test was harder/easier. It is conceivably that the small fluctuations in the mean scores are more a function of the difference in how well prepared the applicants (quality) are and less a function of whether the exam was harder or easier.

Well, if the month to month breakddown yields a result, how do you necessarily know that we are dealing with the same exam? Unless someone here knows better, it is wholly possible that ADA released its versions with the apparently harder QR and RC at a specific set date, and having a time course breakdown could possibly lend support to the notion that the 2007 version is harder, especially if the monthly breakdown starts showing a decline around January or so. But I doubt that anyone would have access to this kind of material, so I guess we will never know for sure.

As for the AA difference, I would tend to argue that the quality of the applicants is likely to be a more constant factor than the test, or, if anything, actually get better since each year nowadays more applicants join the fray. While you certainly could be right in that the observed 0.3 AA difference could be due to statistic drift alone, at least it's not a positive trend. This is why I think having a monthly breakdown would be much more indicative than a 50-50 split, because I really want to see the Jan-Jun 2007 vs. Jun-Dec. 2006.
 
Celebrating the 0.2 point (not 0.3) negative "trend" in the AA score may be a bit premature. The change is 1.1 % and even without the requisite statistical test to determine the statistical significance, it appears more likely that the change is due to randomization. If the changes were about 3-4% we could be having a different dialog. Looking at the mean scores from 1995 to 06-07 we notice that all the means (with the exception of PAT) have a sine wave appearance, giving credence to the changes resulting from randomization rather than a change in the difficulty of the test or of a change in the quality of applicants. On the other hand, the quality of applicants can have an influence on the mean DAT scores if there is a shift towards dentistry at the expense of other fields involves the group at the right side of the spectrum and we may wish to look at the possible influence the influx of applicants may have had on the 06 DAT scores. One of the best explanations to date on the renewed interest in dentistry was given by an ADEA staff member, who suggested that the more "entrepreneurial" applicants looked at dentistry as a profession, life style and remuneration and decided this was the place for them. This may explain why number of applicants to dental schools has seen a ~13% increase while medical school applicants has remained at ~5%. And if this new breed of applicants comes with dynamite academic backgrounds, it may also explain the increase in the DAT scores from 0.27 to 0.55 point for 06. The proviso is that we agree that a 0.57 change in org and 0.46 change in AA is not due to randomization. With an expected 13+K applicants this year it will be interesting to see if the mean DAT scores stay at the level of 2006. Be that as it may, the modest changes in the DAT scores for 06 is testimonial to the ability of the DAT to act as a buffer in spite of a significant increase in the number of applicants and perhaps in the quality of applicants.
FYI. Last year the ADA posted Jan 06-June 06 DAT scores. The low number of applicants ( <3K) may explain why this year they chose to report it differently. For 06, the mean DAT for the limited and the protracted period remained relatively unchanged.
 
To sum it all up.. the DAT is not harder its the DAT candidates that are harder.. whish I had taken my DAT in 1995 :)
 
Celebrating the 0.2 point (not 0.3) negative “trend” in the AA score may be a bit premature. The change is 1.1 % and even without the requisite statistical test to determine the statistical significance, it appears more likely that the change is due to randomization. If the changes were about 3-4% we could be having a different dialog. Looking at the mean scores from 1995 to 06-07 we notice that all the means (with the exception of PAT) have a sine wave appearance, giving credence to the changes resulting from randomization rather than a change in the difficulty of the test or of a change in the quality of applicants. On the other hand, the quality of applicants can have an influence on the mean DAT scores if there is a shift towards dentistry at the expense of other fields involves the group at the right side of the spectrum and we may wish to look at the possible influence the influx of applicants may have had on the 06 DAT scores. One of the best explanations to date on the renewed interest in dentistry was given by an ADEA staff member, who suggested that the more “entrepreneurial” applicants looked at dentistry as a profession, life style and remuneration and decided this was the place for them. This may explain why number of applicants to dental schools has seen a ~13% increase while medical school applicants has remained at ~5%. And if this new breed of applicants comes with dynamite academic backgrounds, it may also explain the increase in the DAT scores from 0.27 to 0.55 point for 06. The proviso is that we agree that a 0.57 change in org and 0.46 change in AA is not due to randomization. With an expected 13+K applicants this year it will be interesting to see if the mean DAT scores stay at the level of 2006. Be that as it may, the modest changes in the DAT scores for 06 is testimonial to the ability of the DAT to act as a buffer in spite of a significant increase in the number of applicants and perhaps in the quality of applicants.
FYI. Last year the ADA posted Jan 06-June 06 DAT scores. The low number of applicants ( <3K) may explain why this year they chose to report it differently. For 06, the mean DAT for the limited and the protracted period remained relatively unchanged.

I like bread! :D
 
It is unclear why having the month-to-month breakdown would be of particular importance. Let us look at a possible scenario. Let us say we have the scores for two consecutive months. In the first month the applicant pool is composed strictly of the lower scoring group with a mean in the teens. In the second month we find them in the higher end with scores in the 20’s. Since in the first group we will have low mean scores the conclusion would be that the test was hard. In the second group, with high mean scores we have to conclude that the test was easier. Yet, we are dealing with the same exam.

It may well be that a 0.2 point difference in the mean AA score is statistically significant. Even if that were the case, the difference would does not necessarily support the hypothesis that the test was harder/easier. It is conceivably that the small fluctuations in the mean scores are more a function of the difference in how well prepared the applicants (quality) are and less a function of whether the exam was harder or easier.

Lets approach this question logically, without math or statistics, so everyone can understand. We all agree dental school became much more competitive since 1995, or even since last year in 06. Correct? Which means the quality of the applicant has significantly increased while the means have remained the same. This makes it obvious that the test has gotten harder. Here's an analogy, a runner is running on a flat treadmill at 7mph. Suddenly the treadmill begins to incline, and in order to maintain the same speed of 7mph, the runner expends more energy. Treadmill inclines more and even more energy is required to maintain the same rate. The speed represents the mean AA score and the incline represents the increasing competitiveness. Just because the means are the same, it does not take into account the incline of the competitiveness. Which means applicants are working harder, expending more energy, and are receiving the same scores. Therefor the test has significantly increased in difficulty. Make sense??

And even more proof, schools wont accepts DAT scores more than 3, sometimes 2 years old. If they were the same difficulty a score from 1999 should be just as valid as one from 2007. And its not bec. they're afraid the applicant forgot the material, since biology credits are valid for 4 years before one is required either to retake the course or take a higher level.
 
I can say without a doubt that certain sections on the DAT have gotten more difficult/changed. I took the DAT originally back in 2005 then i took it again in Aug of 2007. The math definitely got harder/changed. Originally i scored a 21 on the math section, which consisted of maby 3 word problems. My recent math score was a 16 and about 2/3 of the questions were word problems. The RC section was also much different in that it consisted of several questions concerning "tone" and what the author i trying to imply. Those are the only two sections i can confidently say have changed.

Simply by going back through old posts of peoples scores you can see and compare the difference between 2005 QR/RC scores and 2007 QR/RC scores.

Just my 2 cents.
 
Simply by going back through old posts of peoples scores you can see and compare the difference between 2005 QR/RC scores and 2007 QR/RC scores.

Just my 2 cents.

SDN is not a random sample of the population.

bucci said:
Lets approach this question logically, without math or statistics, so everyone can understand. We all agree dental school became much more competitive since 1995, or even since last year in 06. Correct? Which means the quality of the applicant has significantly increased while the means have remained the same. This makes it obvious that the test has gotten harder.

Not necessarily. The more likely scenario is that the quantity of above-average and below-average applicants has increased. This would explain why the applicant pool AA/GPA has been relatively stable over the past 10 years while the enrollee pool AA/GPA has increased.
 
Lets approach this question logically, without math or statistics, so everyone can understand. We all agree dental school became much more competitive since 1995, or even since last year in 06. Correct? Which means the quality of the applicant has significantly increased while the means have remained the same. This makes it obvious that the test has gotten harder. Here's an analogy, a runner is running on a flat treadmill at 7mph. Suddenly the treadmill begins to incline, and in order to maintain the same speed of 7mph, the runner expends more energy. Treadmill inclines more and even more energy is required to maintain the same rate. The speed represents the mean AA score and the incline represents the increasing competitiveness. Just because the means are the same, it does not take into account the incline of the competitiveness. Which means applicants are working harder, expending more energy, and are receiving the same scores. Therefor the test has significantly increased in difficulty. Make sense??

And even more proof, schools wont accepts DAT scores more than 3, sometimes 2 years old. If they were the same difficulty a score from 1999 should be just as valid as one from 2007. And its not bec. they're afraid the applicant forgot the material, since biology credits are valid for 4 years before one is required either to retake the course or take a higher level.


Nice try. Unfortunately, the kinesiology/physiology/physics analogy leaves the argument with a hole big enough you can drive a Mack truck through it and still have enough place for a Mini Cooper on each side. To have a mean you need to have a range of scores. Why you chose to equate the constant speed (7mph) on the treadmill as the mean AA score is unclear. As it has previously been pointed out, over the years,the applicant pool scores have remained virtually unchanged as evidenced by the small variations in mean DAT scores. What has changed is the number of applicants. As the number increases there is an increase in the number of applicants in a given range. With ~13K+ applicants for the ~4500 slots adcoms do not need to dip below the mean DAT scores. As a result, the mean DAT for the enrollees is shifted significantly to the right. As for the proof that schools will not accept DAT scores that are more than 2 or 3 years old, it may come as a surprise, but there only 4 schools that require DAT scores to be no older than 2 years and 8 that require scores be no older than 3 years.
 
I can say without a doubt that certain sections on the DAT have gotten more difficult/changed. I took the DAT originally back in 2005 then i took it again in Aug of 2007. The math definitely got harder/changed. Originally i scored a 21 on the math section, which consisted of maby 3 word problems. My recent math score was a 16 and about 2/3 of the questions were word problems. The RC section was also much different in that it consisted of several questions concerning "tone" and what the author i trying to imply. Those are the only two sections i can confidently say have changed.

Simply by going back through old posts of peoples scores you can see and compare the difference between 2005 QR/RC scores and 2007 QR/RC scores.

Just my 2 cents.

It was certainly much harder/changed for you. Apparently, it was not true across the board since the usual number of applicants that received high scores was there. And by your own admission you were not well rounded in math in spite of your original 21 since you apparently excelled in some math sections but didn't fare too well on others (word problems).
 
Top