If you are achieving a 4.0 at a state school, it is likely perceived as "this student has reached the maximum performance one can at their institution" - what more can you really ask from a student? The 4.0 at the state school will beat out the 3.4 at Harvard, Yale, Stanford, these types of schools. That being said, however, this is an extreme gap that is being used as an example. My suspicion is that a student with a 3.6 in a difficult major, say, Chemical Engineering at school like M.I.T., will probably out-do the state-school-student majoring in Applied Physics at a 3.75 GPA.
The truth of the matter, however, is that this process boils down to much, MUCH more than GPA, the college you attended, MCAT, etc. I graduated from an Ivy, and my GPA was a hair shy of a 3.3 - does that GPA impress anyone at all? NO. I earned a 30 MCAT, which is on the lower end of "acceptable," but that is more often the case for a student with a counter-balancing high GPA.
Somehow, my 3.3 at a top university didn't keep me out of medical school, as I will be attending this fall. I am an extreme case, and perhaps a statistical outlier in this process as I was a non-traditional applicant. However, it should be noted that when the margins in GPA are much less (3.8 vs 3.6, an 0.2 difference) the school you attend can give you an edge. Conversely, I did a traditional M.S. program at a god awful university, with a very poorly ranked medical school. I earned a 3.85, and so this is an example of the high-GPA/low-rank college scenario.
What is my point? If you are a top student at a lesser known/reputable institution, you don't have much to worry about. Your counterparts at top institutions have some leeway in the GPA they typically need to achieve to be equally as competitive as you. Bottom Line.