To me, it is intuitive. There are several reasons to expect this trend and not the reverse.
Wikipedia article explaining why electromagnetic waves slow down in dense material (e.g. glass):
http://en.wikipedia.org/wiki/Refractive_index#Microscopic_explanation
Here is a mnemonic: Blue is on the inside of a rainbow, and red is on the outside. The blue light bent the most. The rainbow encircles the sun.
Here is another mnemonic: On the Pink Floyd album cover with a prism, blue light is on the bottom. The prism is a triangle with the tip at the top.
Here is the real explanation: EM waves slow down in material because the EM wave causes the electrons in the material to physically move up and down (or side to side, depending on polarization). Moving electrons create new EM waves. The EM wave is seemingly slowed down because there is a delay between the incident EM wave and the emitted EM wave. If this delay is 0 degrees, then no change in phase velocity occurs. If this delay is 90 degrees, then phase velocity (apparent wave velocity) will decrease considerably. If the delay is 45 degrees, then phase velocity will decrease half as much (directly in between 0 degrees and 90 degrees).
If the delay is a constant amount of time (a tiny fraction of a second), then the delay in phase (in degrees) will be larger for higher frequency light. Consider 400 nm light (blue) and 800 nm light (IR). If the delay between incident wave and electron movement is fixed then let's say the phase delay is 90 degrees for the 400 nm (blue) light. The blue light will be slowed considerably. That same exact time delay corresponds to a phase delay of only 45 degrees in the 800 nm (IR) light, so the IR light will not be slowed down as much.
This is not the only reason that chromatic dispersion takes place. In fact, it also matters how much of the incident wave energy is transmitted to the electrons, and that is wavelength-dependent as well. However, this factor is hard to predict, because it depends on the resonance modes of the electrons. If the electrons resonate strongly at a certain frequency, that will often give you a colored material.
If you think of the electrons as being delayed by a constant amount of time, then you will remember at least the direction of the trend (although not the shape of the curve).