I presume you want to do EM as a career, so I would strongly recommend doing core rotations (IM, surgery, ob, FP, peds) first to set yourself up for success. As said, EM requires a solid foundation in those and many others (anesthesia, ENT, ortho, ophtho, radiology, psych, etc).
While having been a paramedic undoubtedly gave you great experience and familiarity, being in the ED requires a different knowledge base and mindset.