The question is this: Do you want to stand out because of what you wear? or because of your CV?
You'd probably be fine with most people, but the risk would be that someone at the programs at which you interviewed would think it unprofessional. It would really only take one person to get a stick up their butt to make it more difficult for you.
It's a risk, and as I said, you'd probably be fine... but I wouldn't do it. Keep the olive one for professional functions after you're in.
just my 2 cents.