Harder how? Maturity and having seen what it is like out in the real world gives perspective to those of us who didn't move directly from college to med school, making the choice more certain and keeping the ultimate goal in our mind. Being out of school for a little while does not, according to any statistics I have seen, reduce your chances for admission, provided there is some relevant course work in the recent past (defined differently by different schools). If anything, a break from school may be just what someone needs to realize the additional 7+ years of school and training are worth the opportunity to be a physician.
OP, if you aren't sure, don't jump in. Get a job, really investigate the work required of doctors, and see if you really want to commit another decade or so to becoming an attending physician. Some time off will not hurt you if you aren't sure you want this right now. If you jump in now, unsure, you could burn out only to realize later, when in the real world with a failed attempt at medical school as a blemish on your record, that you really do want this. After all that, it will be much harder to gain admission to a medical school and fulfill your dreams than if you take a break of a year or three now.