I have some colleagues at 2 different sites that do open houses. Their ranks are essentially set before the open house based on application. The only way ranks change for them is if someone does something wildly inappropriate during the open house. But, from what they tell me, it's pretty rare. Just an n of 2, so take it for what it's worth.
When I applied, I declined the one open house invite I had. Would have been too expensive and would have made my travel schedule very hectic due to distance factors and what not. If you've got plenty of interviews and it's down your list, no harm no foul. If you don't have many, or it's in your top 3 or 4, I would go.