nono, it's the opposite. silica gel is very polar. so more polar material moves more slowly than nonpolar material, which feels less attraction from the silica gel. it's used in TLC and column chromatography (not paper chromatography).
obviously you've got the reasoning down right (that if the stationary phase is nonpolar, nonpolar compounds will migrate more slowly, and if the stationary phase is polar, polar compounds will migrate more slowly) so the only thing you need to do is remember that silica is polar. how can you do this? well, I have two suggestions:
first, you can just remember that silica is silicon dioxide. silicon-oxygen bonds are polar for the same reason carbon-oxygen bonds are polar.
the alternative is that you can remember that there's something called reverse-phase chromatography where long chain hydrocarbons are attached to the silica to make it nonpolar. this is called C18 silica (because it's made of 18-carbon long hydrocarbons). so you could just remember that, because this one has the hydrocarbons, it's nonpolar, while regular silica will be the opposite, so it's polar.
hope that's clear
EDIT: heh, quantum beat me to it! but that's the formula, silicon dioxide (SiO2)