Composite Inflation

This forum made possible through the generous support of SDN members, donors, and sponsors. Thank you.

RyanMaverick

Senior Member
10+ Year Member
5+ Year Member
15+ Year Member
Joined
Feb 14, 2006
Messages
150
Reaction score
0
I've been looking over the scores that people are posting and the general trend seems to be that the composite average is approx 10% higher then the overall average of each subsection. Is there an explanation for this? Are different sections weighted differently (as I was under the impression they were all equal)? And finally the essay score out of 5 does not count towards the composite correct?
 
RyanMaverick said:
I've been looking over the scores that people are posting and the general trend seems to be that the composite average is approx 10% higher then the overall average of each subsection. Is there an explanation for this? Are different sections weighted differently (as I was under the impression they were all equal)? And finally the essay score out of 5 does not count towards the composite correct?
I haven't noticed this
my composite is exactly the average of my subsections
you are looking at the scaled score and not the percentile rank, right?
the essay has no effect on the composite
 
Ender17 said:
I haven't noticed this
my composite is exactly the average of my subsections
you are looking at the scaled score and not the percentile rank, right?
the essay has no effect on the composite

My composite is much higher than my breakdown. I think the explanation lies in the fact that not everyone has an even distribution of high scores. Technically someone could have a 99 in one subject but poor distributions in others, so not everyone performs on an equal level. For me I had relatively close scores in the low 90s/80s and still managed a 95. Perplexing, is it not?
 
evilolive said:
My composite is much higher than my breakdown. I think the explanation lies in the fact that not everyone has an even distribution of high scores. Technically someone could have a 99 in one subject but poor distributions in others, so not everyone performs on an equal level. For me I had relatively close scores in the low 90s/80s and still managed a 95. Perplexing, is it not? Does this suggest that there is NOT as many students in the 99th composite for every 1% composite?
You don't average the percentile rankings, you average the scaled scores.
And you made a 95? 😉
 
Ender17 said:
You don't average the percentile rankings, you average the scaled scores.
And you made a 95? 😉

I think that's what I was describing without actually taking a step back and looking at it 🙂
 
But how do you average the scaled scores? I mean your composite is given to you in terms of a percentile, whereas each subsection has you provided with a scaled and a percentile. I was averaging up the percentile averages for each subsection trying to come up with some formula for the composite. It still looks pretty bush-league, perhaps I'm missing something.
 
Your scaled scores are added up and average to get your composite scaled score (which is also given). Then they use whatever distribution they use (probably normal) to figure out what percentile that score is in (using average and standard deviation). So each section counts the same.

If you tried adding percentiles, it wouldnt make sense because the avg and std dev on each section is different and there is more than just one scaled score for each percentile (and the same scaled score on one section doesnt give the same percentile as on another section). Also, people don't generally do well in all categories. So if you get 4 really good scores and one not so good (or all 5 scores in low 90s/80s)...your composite will probably be really good (I got 95, 97, 99, 99, and 64--dont know what happened--my composite was 99).
 
RyanMaverick said:
But how do you average the scaled scores? I mean your composite is given to you in terms of a percentile, whereas each subsection has you provided with a scaled and a percentile. I was averaging up the percentile averages for each subsection trying to come up with some formula for the composite. It still looks pretty bush-league, perhaps I'm missing something.


It's a pretty complicated system. Scaled scores can range from 200 to 600, with 400 as the stated median score. This seems to suggest that someone scoring 50% correct (missed 20 out of 40, for example) would receive a 400 for their scaled score, while someone who correctly answered 75% of the questions would receive a 500, and a perfect test would result in a 600. However, I have never seen an individual scaled score reach the 500's, and it is hard for me to believe that NO ONE answered even 75% right. I would suggest that the correlation between actual questions missed and scaled score is NOT linear, although I don't know what the correlation is. Let's just assume it's complicated, but that increasing the number of questions you answer correctly also increases your scaled score.

When it comes to percentile scores, Harcourt simply compares your scaled score for the section (however that score was derived) with the scaled scores of test takers from the norm group—all first-time examinees that took the test between October 1998 and March 2003. This comparison results in a percentage. If you got a 92% on the Biology section that means that your scaled score was higher than 92% of the norm group's Biology scaled scores. However, there is usually a range of scaled scores that compose each percentile score - maybe you had a scaled score of 436, your friend had a 434, and your mom had 430 - these could all conceivably correspond to the 92% window.

Composite percentile score simply compares your composite scaled score with that of the norm group's. Your composite scaled score is just the average of the individual sections' scaled scores. So, if you got 479, 430,446,431,and 436 on the individual sections, your composite scaled score would be 444. It is this number that dictates your overall percentile ranking.

So you see, it doesn't really matter what the individual percentile rankings are. As discussed above, you, your friend, and your mom all scored in the 95% for Biology, but your scaled score on that section was a full 6 points higher than your mom's and those points DO matter. Even if you and your mom had the same percentile scores in every sub-section, if you consistently outscored her by a few points in the scaled score, your composite scaled score could be quite a bit higher than her's. When that composite scaled score is compared to the norm group's, you may end up in the 98% range while your mom is in the 94%.

There's actually quite a bit of variation between individual scaled scores and percentile scores, especially if you got a 99% on any individual section. Some people in the 99% literally got every question right, resulting in a very high scaled score. Other people may have missed 7, but still did better than 99% of the norm group. They will have a lower composite scaled score than their perfect counterparts, thus they will probably also have a lower composite percentile score. In other words, your overall percentile score may seem higher than you expected because you totally rocked one or two of the sections.

Sorry for the long post. Hope that helps.
 
Ok thanks, that clears things up a lot more. Thanks for the informative post.
 
Well, since I am stat major, I can clear this with a short explanation.

This pattern is correlated but not linear. Hence, there is some type of transformation (could be quadratic, most likely) of everyone's scores for each section. which Harcourt is using.

So let us say x1-x5 represent each persons scaled score and if we had a linear relationship we could write the model as:

Yi = Bo + B1x1 + .... + B5x5 + Ei

However, let us assume that it is harder to score a very high scaled number. That is a 95% could be a 500 and a 100% could be a 600. This is obviously not linear. On the other hand a 0% could be a 200 while a 60% could only be a 250. Hence, there is a lot of distance between percentage and scaled score on the bottom with very little room on the top. Hence this is a skewed "non-normal distribution"

So if I worked for Harcourt as a statistician to analyze the pattern. I would then try another model where Xi = sqrt (Yi) or even another type of transformation could even be cubic, quartic, etc. and plot a histogram to see if the scores now exhibit normality, and test whether the model does not fit by using SAS software. If so, then I would tell Harcourt to keep the current scoring system. If not, then they would have to come up with something else.
And then as a statistician you would keep on testing the model until you came up with the best fit line which would cause the least variation between testing results as compared to the norm group. Hence these techniques are used to ensure that if someone took the test at one sitting and without studying took the same test at another sitting, the scores would not vary significantly.

Quite interesting!!!
 
RyanMaverick said:
But how do you average the scaled scores? I mean your composite is given to you in terms of a percentile, whereas each subsection has you provided with a scaled and a percentile. I was averaging up the percentile averages for each subsection trying to come up with some formula for the composite. It still looks pretty bush-league, perhaps I'm missing something.


Like previopus posters said it is not an average of your percentile for each section. This is what most people believe but not true. Your scaled scores for each sections are averaged and that average is given a percentile. For example every1 from 500-600 might get a 99% every1 from 490-500 might get 98% like that. Basically rememeber that ppl are designated into 99 different sections with 1% of people in each section.

This is kinda unfair bc there is a wide gap in score range as you go up. For example a person with 450 might have a 95% and a person with 490 might still have 95% if 1% of people fall in between 450 and 490 score. Over years I have found that scoring range is very short at bottom and wide at the top like:

370-400 = 0 - 50 percentile Assuming the lowest any1 would get is 370 which is realistic.
400-410 = 50 to 70 percentile
410-430 = 70 to 90 percentill
Above 430 = 90 to 99 % percentile.

This is how scores ususally fall. Post yr scores and scaled score so you can cofnirm this,'






If you have
 
RyanMaverick said:
But how do you average the scaled scores? I mean your composite is given to you in terms of a percentile, whereas each subsection has you provided with a scaled and a percentile. I was averaging up the percentile averages for each subsection trying to come up with some formula for the composite. It still looks pretty bush-league, perhaps I'm missing something.

haha, thats funny ! :laugh:
 
Top