There's a compelling Constitutional argument for some form of carry, either open or concealed, or both. And indeed SCOTUS has commented on this in the past.
I'm not a fan of open carry (of handguns) but I wouldn't go so far as to make it illegal. Broadly speaking, in the context of self defense, I think the practice is unwise and counterproductive. My personal opinion is that when it comes to handguns, concealed is far more practical. Open carry obviously advertises the presence of a gun, which might be a deterrent, but can draw unwanted and unkind attention as well. This is the polar opposite of the purpose of carrying a weapon for self defense. Concealed carry is simple and practical in every place where it's legal to possess a firearm. It fulfills the basic need for self defense and ought to be the default self-defense right, IMO. I favor legal concealed carry everywhere[1], and probably federally-mandated national reciprocity for state-issued permits (same as a driver's license).
Persons exercising their right to open carry are often doing it as a part of exercising their right to assembly and free speech. They're carrying for the express purpose of being seen carrying. The guys who carry AR15s into Starbucks aren't doing so because they're actually worried about being attacked, they're doing it to get on the news and draw attention to their 2A advocacy. I think they're dumb, but being dumb in public shouldn't be a crime unless it creates a hazard to other people and their property.
I don't think banning open carry of long guns should pass Constitutional review, since the core of the right is to keep and bear arms. The 2A doesn't actually say anything about self-defense (or deer hunting, or Olympic competition, or recreation) although SCOTUS has ruled that those rights are implied and protected. If there's any meaning or teeth to the Amendment that preserves the right of armed resistance to tyranny, the people who keep the arms have got to be able to load them, carry them outside their homes, and freely assemble with other people doing the same.
I understand there are people who don't think any of that should be legal, but their remedy is also right there in the Constitution: amend or repeal the 2nd Amendment.
[1] with some limited and notable exceptions, e.g. sterile areas of airports, courthouses etc - locations where security is present and actually has a duty to protect the unarmed people inside