I believe you should go, even if you don't expect to get any good information out of the new adviser. At many schools, the pre-med advisers advise the pre-health committees who give your school's committee letters (some places they also sit on the committee), so building a positive relationship with these people can only help.