Now I could certainly be wrong, and I love learning new things, but federal loans do not come directly from the government. In particular I am thinking of Subsidised Stanford Loans which are issued by banks and the government pays the interest for you while you are in school. That is taxpayer money benefiting the student. The student will never pay back the interest from when he was in school (except in the form of taxes). The government does not benefit from the interest after you graduate, the bank does.
Mind you I am not against student loans, quite the contrary. But if someone is against government aid, redistributing wealth, etc, etc, they should be against student loans as well. Unless of course said person is a hypocrite who is only against aid they do not qualify for (this is not aimed at anyone in particular, just a general comment).