While having anesthesia letters might help, I think if you apply from another specialty, the most important aspect is having letter(s) from your program chair/PD that reflect you work ethic, attitude, etc. as a resident. Having recently gone thru the application, very few programs actually have a requirement of 1 anesthesia letter (and the ones I did said minimum of 1). This is in contrast to the competitive surgical specialties, where ALL of your letters should be ortho/urology/ENT, and so on.
I think that many anesthesia PDs/chairs get applicants from other fields. I remember one of them asking, "What made you see the light?" (I was doing a surg prelim).
Keep in mind, though, as WholeLottaGame mentioned, you'll still have to come up with a reason explaining your new-found commitment to anesthesia.
Good luck.