You can probably either lighten your friday schedule or take the day off if you want. However, keep in mind a couple of things. Unlike say primary care, it is impossible for a neurologist to see 4-6 patients per hour, so to a neurologist, a day off it lost money. But hey, if you want to do it, there is nothing to stop you from doing so.
But also keep in mind that even if you lighten up your Friday clinic schedule, somebody in your group has to be doing inpatient consults or taking call.