I would say that there are downsides to either path you take, but you definitely shouldn't just join the military for money. Personally, I think that the early leadership opportunities in the military, along with the work atmosphere and the dedicated people you work with and serve are unparalleled. I also love to travel and love change, but I will note that as a doctor in the military you really have more say over what you do compared to other people. I know doctors who have stayed in the same major hospital for over 10 years now. I also know a doctor who ran a department only three years out of residency in Germany. You'll have chances you won't have elsewhere, but you'll hate it if you really think it's not for you. If it's not too much of a financial burden, I'd go. You really learn so much about their mission and resources while you are there and my interview with a faculty member was very truthful. They'll tell you what they do and don't like, and really give you an honest answer to help you decide for yourself.