Why Diagnostic Tests are Worthless

SDN Members don't see this ad. (About Ads)

As you may have read in my other posts, I think diagnostic tests are completely worthless. Since I've seen numerous threads pop up on the subject (ex. “My diagnostic is horrible, can I do well on the real thing?”), I thought I'd give my reasons as to why I feel this way. To start, I will define what I consider a diagnostic test. When I talk about diagnostic tests, I am referring to a test that is taken prior to any major studying. Now that the term is out of the way, time to go into the reasons.

For the purpose of this analysis I will not mention any specific companies. However, I will say that people should not put much credence into them. These tests are typically designed for the taker to score low so the company can accomplish two things. First, an abnormally hard diagnostic helps a company fulfill their point guarantee. I know that even after I finished studying, I would still not score well if I took the diagnostic. Secondly, it demeans a student into believing they must take this course and/or study hard. While it’s true that the scare factor can help motivate people to study, I don’t think it’s necessary. If your drive to become a doctor isn’t enough to get you to buckle down, there’s a problem. Worse still, the diagnostic given may be in a completely different format from the actual MCAT.

Even if the diagnostic is an actual AAMC full length, it does not give the taker an accurate assessment of their weaknesses. This statement sounds surprising because this is supposed to be the goal of a diagnostic, but bear with me on this point. To describe my reason for this I'll give an example. Let's say you're really good at physics, but you haven't taken a course in awhile. At the same time you're bad at biology, but you just finished a course. On the practice test your biology score would probably be higher because it's fresh in your mind. That said, once you start studying, your physics skills will have the rust taken off of them and you'll start seeing huge gains. Unfortunately, since your practice test said your physics was weak, you've been focusing on that instead of biology, your real weakness. Thus, rather than detailing a student’s weaknesses, a diagnostic tells you what courses you’ve just taken.

Now you may say, “I know my weaknesses already. That example won’t happen to me because I would already know physics is my strength and biology my weakness” If this is truly the case, why take the diagnostic?

All that a diagnostic tells you is that you are not ready to take the test. Well, is that honestly a surprise? It's not like you have been studying for months before you took this test. The diagnostic tells you what you already know, you have to actually study for the MCAT. Take a practice test after you've gotten through at least half of the content, then you'll be able to gain a better understanding of your weaknesses. In the meantime, pound away at those TIMED practice problems and use them to help gauge your strengths and weakness. Just stay away from most of your full lengths until you've finished your content review.

Agreed. It's not worth your time unless maybe you really want to find out which topics you NEED to nail down before you start studying, but you can find that out after you take your first real practice test rather than a diagnostic.

Complete agree! I think it is important to take a verbal section, just to know if you completely suck or are amazing at it...but, like roadrunner said, you could see that in your first practice test.

Agreed

Except I would go as far as to say you shouldn't do FL's until completely done with studying.

my diagonistic: 22
actual: 34

the diagnostic was a complete waste of my time. what the hell do I care how well I could have done before studying?

Well said.
The real use in a diagnostic is just for the people that have never been in an MCAT situation before so they can just see what they are up against. In fact, a diagnostic where you weren't shown your score afterward would have the exact same usefulness than one where you get your score.

After you've taken one exam before studying, you get a feel for what the exam is like, and that's really the entire point. Your score on that exam is completely meaningless for how well you will do on test day.

I still say people should take one exam before they start studying, but I really think that they should not even bother scoring it once they are done. Do a month or two of content review with practice problems and passages here and there, and then take your first "real" practice exam.

I actually think the practice tests are an important (secondary) part of your preparation. It's a longish test for some people, and it's good to work up your testing endurance and get a feel for the test prior to taking the actual one imo.

Practice tests are a very different beast from what is being talked about here - diagnostic tests. Diagnostics are tests you take before you do any studying to supposedly show you how much you need to study. I've never taken one, because they sound dumb. I know how much I need to study - it's simply the difference between what I need to know and what I already do know.

Yeah, we're just talking about the pre-study diagnostic. I doubt many (if any) people here would argue that taking timed, full-length practice tests are not useful. I did 20+ full length exams myself!

Yeah thats why I waited until half way through my content review before taking the AAMCs. Taking #3 before studying at all would have been a complete waste of an exam.


I remember back in HS when I took a TPR review course and the diagnostic was impossible. Made everyone feel good when their grades started getting higher and higher.

PSAT before the course: 1240
SAT after the course: 1260


I ended up taking it again two months later with almost no studying between the exams and got 1350. None of it means ****.

Oh and my final TPR exams were something like high 1400s. They go from harder to easier to make everyone feel good. Its all bull****. Its one of the main reasons that I didn't even think about taking a mcat review course.

oh, ok then. so much for my reading skillz.

Originally Posted by SN2ed

As you may have read in my other posts, I think diagnostic tests are completely worthless.

For the purpose of this analysis I will not mention any specific companies. However, I will say that people should not put much credence into them. These tests are typically designed for the taker to score low so the company can accomplish two things. First, an abnormally hard diagnostic helps a company fulfill their point guarantee. I know that even after I finished studying, I would still not score well if I took the diagnostic. Secondly, it demeans a student into believing they must take this course and/or study hard. While it’s true that the scare factor can help motivate people to study, I don’t think it’s necessary. If your drive to become a doctor isn’t enough to get you to buckle down, there’s a problem. Worse still, the diagnostic given may be in a completely different format from the actual MCAT.

Even if the diagnostic is an actual AAMC full length, it does not give the taker an accurate assessment of their weaknesses. This statement sounds surprising because this is supposed to be the goal of a diagnostic, but bear with me on this point. To describe my reason for this I'll give an example. Let's say you're really good at physics, but you haven't taken a course in awhile. At the same time you're bad at biology, but you just finished a course. On the practice test your biology score would probably be higher because it's fresh in your mind. That said, once you start studying, your physics skills will have the rust taken off of them and you'll start seeing huge gains. Unfortunately, since your practice test said your physics was weak, you've been focusing on that instead of biology, your real weakness. Thus, rather than detailing a student’s weaknesses, a diagnostic tells you what courses you’ve just taken.

Now you may say, “I know my weaknesses already. That example won’t happen to me because I would already know physics is my strength and biology my weakness” If this is truly the case, why take the diagnostic?

All that a diagnostic tells you is that you are not ready to take the test. Well, is that honestly a surprise? It's not like you have been studying for months before you took this test. The diagnostic tells you what you already know, you have to actually study for the MCAT. Take a practice test after you've gotten through at least half of the content, then you'll be able to gain a better understanding of your weaknesses. In the meantime, pound away at those TIMED practice problems and use them to help gauge your strengths and weakness. Just stay away from most of your full lengths until you've finished your content review.

THANK YOU so much for this post. So many stressed out premeds fall for this emotional and financial trap. A test at the start of a course (or before a course) tells you what you already know... you are not ready for the MCAT. To throw an abnormally difficult test for the sole purpose of setting a low starting point to make people feel like they have improved is not only unethical, but it undermines the purpose of a course. If your confidence is damaged, you will not perform as well as you could.

Doing well on the MCAT is about knowing the material, knowing how to take the exam, and most of all walking in confident so you don't second guess yourself. This is a brilliant OP!!!

Thanks for all the responses.

Originally Posted by SN2ed

Thanks for all the responses.

Well...it's a good topic and a good post!

My Kaplan diagnostic if I recall was not even a full length test. It was a half test. Agreed, take it with a grain of salt.... or is it like a grain of salt? haha

The kaplan diagnostic test is harder than the real test, so that they can make sure that ur score "improves on the real one" so that u dont come back to them and ask for a refund.

ah ok, i wasn't clear about the distinction between practice tests vs diagnostics right off the bat. yes i agree with you OP.

Diagnostic: 16R
Real MCAT: 32Q

No Joke.

Originally Posted by SN2ed

As you may have read in my other posts, I think diagnostic tests are completely worthless. Since I've seen numerous threads pop up on the subject (ex. “My diagnostic is horrible, can I do well on the real thing?”), I thought I'd give my reasons as to why I feel this way. To start, I will define what I consider a diagnostic test. When I talk about diagnostic tests, I am referring to a test that is taken prior to any major studying. Now that the term is out of the way, time to go into the reasons.

For the purpose of this analysis I will not mention any specific companies. However, I will say that people should not put much credence into them. These tests are typically designed for the taker to score low so the company can accomplish two things. First, an abnormally hard diagnostic helps a company fulfill their point guarantee. I know that even after I finished studying, I would still not score well if I took the diagnostic. Secondly, it demeans a student into believing they must take this course and/or study hard. While it’s true that the scare factor can help motivate people to study, I don’t think it’s necessary. If your drive to become a doctor isn’t enough to get you to buckle down, there’s a problem. Worse still, the diagnostic given may be in a completely different format from the actual MCAT.

Even if the diagnostic is an actual AAMC full length, it does not give the taker an accurate assessment of their weaknesses. This statement sounds surprising because this is supposed to be the goal of a diagnostic, but bear with me on this point. To describe my reason for this I'll give an example. Let's say you're really good at physics, but you haven't taken a course in awhile. At the same time you're bad at biology, but you just finished a course. On the practice test your biology score would probably be higher because it's fresh in your mind. That said, once you start studying, your physics skills will have the rust taken off of them and you'll start seeing huge gains. Unfortunately, since your practice test said your physics was weak, you've been focusing on that instead of biology, your real weakness. Thus, rather than detailing a student’s weaknesses, a diagnostic tells you what courses you’ve just taken.

Now you may say, “I know my weaknesses already. That example won’t happen to me because I would already know physics is my strength and biology my weakness” If this is truly the case, why take the diagnostic?

All that a diagnostic tells you is that you are not ready to take the test. Well, is that honestly a surprise? It's not like you have been studying for months before you took this test. The diagnostic tells you what you already know, you have to actually study for the MCAT. Take a practice test after you've gotten through at least half of the content, then you'll be able to gain a better understanding of your weaknesses. In the meantime, pound away at those TIMED practice problems and use them to help gauge your strengths and weakness. Just stay away from most of your full lengths until you've finished your content review.

bumping a great thread

whew, i'll be sure to show that to my mom when she disagrees with me on not really taking a diagnostic to see how i fare

and also, as a sidenote though, at least for verbal, before you start timing yourself, would you say it's good to really focus on the passages & the questions (more on the questions though) so that you learn how to actually do the section? and then start timing yourself? or should it first be timed and then keep practicing with the timing and somehow get better. i can't figure out which way to study T_T

I definitely agree with you, here is why

When I took my diagnostics test, I got a 10, however that score was bull****, I got 60% of the verbal reasoning answers right but I got a 4 according to the report. I got 30% of the Biological Sciences questions right but I got a 2, I got 27% of the Physics questions right but I got a 4.....

Originally Posted by Xbocker

I definitely agree with you, here is why

When I took my diagnostics test, I got a 10, however that score was bull****, I got 60% of the verbal reasoning answers right but I got a 4 according to the report. I got 30% of the Biological Sciences questions right but I got a 2, I got 27% of the Physics questions right but I got a 4.....

Not to burst your bubble or anything, but on a real exam, that's pretty much what the scale would have awarded you. As stated earlier though, you should not get angry about your diag because it is not indicative of your potential to learn MCAT material, to really get down to the nitty gritty of understanding the concepts and the exam itself, and finally, to get an amazing score. Shurg that diagnostic off, and get excited about learning and studying for the exam.

I'm going to shamelessly bump my old thread since it looks like people are starting to ask about diags again.

I think the purpose behind diagnostic's are to scare kids into taking kaplan/PR classes. i remember my kaplan diagnostic being incredibly low (i think an 19 or something and coincidentally my verbal was the best)....in real life my verbal was the worst....so i don't think its very important. just take practice tests...thsoe are important

Originally Posted by babyxpink

whew, i'll be sure to show that to my mom when she disagrees with me on not really taking a diagnostic to see how i fare

and also, as a sidenote though, at least for verbal, before you start timing yourself, would you say it's good to really focus on the passages & the questions (more on the questions though) so that you learn how to actually do the section? and then start timing yourself? or should it first be timed and then keep practicing with the timing and somehow get better. i can't figure out which way to study T_T

Oooh, a fellow Korean American pre-med! 안녕하세요!

I'm a former tpr teacher who still independently teaches mcat prep and I think diagnostics are useful.

Big test prep companies do some fishy things with their diagnostics. It's wrong to make the first diag more difficult than the real thing and the potential motivational advantages don't justify the breach of trust.

However, real, valid diagnostics inform both the student and the teacher about the student's starting point. The starting point doesn't define potential but it is significant. It's rare to see a student improve more than 20 points from an initial diagnostic.

It's hard to set legitimate scoring goals and decide on the best path towards improvement without the data that comes from a diag.

-aj

got a 21 on kaplan diagnostic. worst score in verbal now i gotta start studying for may mcat i guess examkrackers verbal will help

Yeah, diagnostics were pretty useless to me. I got a 30 on my diagnostic, 8Physical , 9 verbal and 13 bio. After studying I got consistently 33-35 range on kaplan practice exams, and only ended up getting a 28Q on the real thing. Needless to say, I'm rewriting in the summer.

As a Kaplan teacher, I don't make predictions on scores based on the Diag. It is needed for the terms of enrollment/Higher Score Guarantee, but is generally only useful in convincing people they need to study.

Of course, I've had a few students Diag @ 33... So whatever.

Also, practice tests are predictive on average, just not perfect. Maybe an r = .8? so 64% of variance... Is a decent range that I just pulled out of nowhere...

The Diag's are sneaky, but I'd say the free practice tests are pretty on board. I dropped 6 points from my PT to my Diag... I took them a month apart.

17 diag (ha, yeah...I know) to 25 then 31 actual.

Really I think they just design them to make you freak out and sign up for a class. Those tricky people!

i totally agree with the OP, and having known this prior to taking the diagnostic really helped me not freak out. I don't know if this is accurate, but I got 8 out of 32 wrong on the verbal and I got a 5! Does that make any sense?

Diagnostic tests aren't completely worthless. If you've never taken a CBT before, it's a good experience. It doesn't hurt to familiarize yourself with the format.

But yeah, people who freak out about their diag scores need to exercise some common sense.

Originally Posted by mmmcdowe

The Diag's are sneaky, but I'd say the free practice tests are pretty on board. I dropped 6 points from my PT to my Diag... I took them a month apart.

I just took a practice exam from kaplan and got a 16..with a 4 on verbal..I just finished my fall semester as a freshman and have to take the MCAT summer after my sophomore year because I'm in a B.S/D.O program and the major reason y people don't make the prog is due to low mcat scores because you have a full schedule on top of it.

But I was looking through all the posts and they'r for diag tests with huge improvements..I was wondering if anyone just took a practice test and saw any improvements? Is it possible to increase my score by 11 points by the end of next year? I didn't think that the PT was extremely difficult compared to the other passages I've started to due in the Princeton review/ ExamKrackers online stuff. I've started to do the EK verbal passages since I scored a 4 on my verbal section on the PT. 6 for each Bio and PS. which was weird because I though that I would do worse in PS since I never took physics or org chem before.

Any Advice? Thanks!

Bumping great thread.

One thing though. Diagnostics are not worthless if you know what you are getting into. That is, I am taking it, not expecting to do well. However, I feel like it will give me a basic overview of what I need to study more, and looking at my score, perhaps motivate me to study more. I know I am not going to purchase the class if I feel like I do poorly.

Those diagnostic tests truly are worthless.. the test prep companies just want you to do horrible in order to discourage you and convince you to take their class
Thats why the director always comes in when you get the scores and gives you some ridiculous $200 off coupon

Thanks SN2ed. Great name. Back side attacked! Yes, they are talking about Diags here though...Im talkin its 6 weeks before hand. score went up from where i began...but not breakin 30 yet...
Im gettin off this website and disconnecting my internet while I study.
cheers

It really depends on which diagnostic test you're using. There are many different providers and each one varies considerably: Kaplan, Princeton Review, etc.

being in mcat study mode, i will take this opportunity to be a total douchebag due to the fact that my brain is now only primed to see the world in terms of passages:

1. with which of the following statements would the author of the passage agree?

I. diagnostics are a way to make money for private prep companies

II. the fact that people get low scores points to the fact that they need to study, which they already know

III. people have an intuitive sense of their areas of weakness, rendering diagnostics futile.

a. I only
b. II only
c. II and III only
d. I, II, and III

lol

Originally Posted by pandoraaj009

being in mcat study mode, i will take this opportunity to be a total douchebag due to the fact that my brain is now only primed to see the world in terms of passages:

1. with which of the following statements would the author of the passage agree?

I. diagnostics are a way to make money for private prep companies

II. the fact that people get low scores points to the fact that they need to study, which they already know

III. people have an intuitive sense of their areas of weakness, rendering diagnostics futile.

a. I only
b. II only
c. II and III only
d. I, II, and III

lol

Options I and II contradict each other but according to what is written in the "passage" I guess they would both be correct.

If the diagnostics are written in such a way so as to make the test taker feel unprepared for the exam then it would be quite possible for him/her to score low and still be fully prepared for the real exam.

... it is way too late

Originally Posted by azrk

Options I and II contradict each other but according to what is written in the "passage" I guess they would both be correct.

If the diagnostics are written in such a way so as to make the test taker feel unprepared for the exam then it would be quite possible for him/her to score low and still be fully prepared for the real exam.

... it is way too late

touchè.

Some people use them as a barometer after they have studied.... This is probably best for the LSAT, since a Kaplan diag (and would assume for other companies) are actual released LSAT tests. If AAMC were to release more questions, the diagnostic could be a much more useful tool.

Especially the disparity in the types of questions Kaplan writes vs. the AAMC.

Originally Posted by Chuck's Right Foot

Some people use them as a barometer after they have studied.... This is probably best for the LSAT, since a Kaplan diag (and would assume for other companies) are actual released LSAT tests. If AAMC were to release more questions, the diagnostic could be a much more useful tool.

Especially the disparity in the types of questions Kaplan writes vs. the AAMC.

"To start, I will define what I consider a diagnostic test. When I talk about diagnostic tests, I am referring to a test that is taken prior to any major studying."

Originally Posted by SN2ed

"To start, I will define what I consider a diagnostic test. When I talk about diagnostic tests, I am referring to a test that is taken prior to any major studying."

And the majority of my post still stands, the more closely it resembles an actual test, the more worth it has.

I made a 31 on the Kaplan diagnostic... That was pre-second semester physics/organic and studying. Pretty sure I would have made a sub-20 on the real MCAT had I trusted that diagnostic

I followed the EK 10 week and burned the AAMC 5 test...I felt stupid after reading this thread.

Originally Posted by aevea

I'm a former tpr teacher who still independently teaches mcat prep and I think diagnostics are useful.

Big test prep companies do some fishy things with their diagnostics. It's wrong to make the first diag more difficult than the real thing and the potential motivational advantages don't justify the breach of trust.

However, real, valid diagnostics inform both the student and the teacher about the student's starting point. The starting point doesn't define potential but it is significant. It's rare to see a student improve more than 20 points from an initial diagnostic.

It's hard to set legitimate scoring goals and decide on the best path towards improvement without the data that comes from a diag.

-aj

i just took my PR diags and saw your post. if 20+ is rare, what would you say is a reasonable range for improvement?

Originally Posted by wicked398

I just took a practice exam from kaplan and got a 16..with a 4 on verbal..I just finished my fall semester as a freshman and have to take the MCAT summer after my sophomore year because I'm in a B.S/D.O program and the major reason y people don't make the prog is due to low mcat scores because you have a full schedule on top of it.

But I was looking through all the posts and they'r for diag tests with huge improvements..I was wondering if anyone just took a practice test and saw any improvements? Is it possible to increase my score by 11 points by the end of next year? I didn't think that the PT was extremely difficult compared to the other passages I've started to due in the Princeton review/ ExamKrackers online stuff. I've started to do the EK verbal passages since I scored a 4 on my verbal section on the PT. 6 for each Bio and PS. which was weird because I though that I would do worse in PS since I never took physics or org chem before.

Any Advice? Thanks!

If you take a course or study hard for a couple months your score will go up significantly. Diagnostic exams are stupid. My score went up 13 points after the course.

Originally Posted by Protactinium

I followed the EK 10 week and burned the AAMC 5 test...I felt stupid after reading this thread.

What do you mean by "burned?" Because I'm doing the EK 10 week as well and I just did the practice test that came with the 10 week books and got a 23. (Off the top of my head I think I got a 22 or 24 (dont remember) on the AAMC #3 test.)

Lol, so yea, these practice/diagnostic tests do seem to be frustrating, I hope I follow the trend and get a better score on the mcat! (Takin it June18!!)

Shameless bump because we're starting to get more diag threads.