There are a few things to consider.
Re: fellowship goals, those can change. Might end up loving ped surg, onc, or plastics along the way. If you do, then research might help.
Re: uncertainty about research. If you aren't into research at this point I think it is unlikely that you will come to like it later. If you somehow find yourself in a lab you might enjoy it, but even then I still think it's unlikely that you will run a lab in the future, get NIH funding, etc. So if you look at the lab time as an investment for academics, I would argue that is probably not a good investment. I never met anyone who was only sorta kinda into research actually go into research oriented academics... seems they were all pretty gungho about this stuff, but that's just anecdotal.
Re: time off. I took some time off in college. It is annoying watching my friends become attendings and my co-interns going on job interviews, but I was able to pay off some of my student loans and basically just waste a ton of time, which I love to do. Time enjoyed is never time wasted in my book. I'm in no hurry to be a real adult; being an adult sucks.
In my personal case, I like research, but I don't really love it. What I like the most about lab time is the chance to show up to work when I want, leave when I want, turn off the pager for 2 years, and basically have control over my life again. The writing, pipeting, presenting, conferencing, and "getting my name out there" stuff is way less important to me, which probably does not bode well for an academic future. Previously I was pretty gungho research and academics, was looking at 7 year programs, and all that. Man, but that stuff doesn't get me up in the morning, which is probably why I show up to work at 10, about 4 times a week.