I had a friend in a similar situation. He was in a bad resideny with a bad program director who used his power unscrupulously to keep residents from leaving. He's found a new program, far better than his one, and he's happy he left.
Most PDs will dislike you leaving, but won't really try to hinder you. Don't expect any help from them, though. Very few PDs are enlightened enough to help you leave their program.
The worst programs tend to want to hold onto their candidates most strongly. Residency programs regularly abuse residents, but they have no power to keep the resident who wants to leave.
Some even use scare tactics and other below-the-belt intimidation in order to hold onto residents. First they say you're disloyal, then they say no other program will touch you, then they try to get you fired. Many tell you that changing residency programs will make it difficult to find a job later. It's just not true: people change residencies all the time now, for good reasons.
Program directors have a lot of power but they're not omnipotent. If the PD or any other attending threatens the resident, the resident can always go to the school's GME office or directly to the ACGME.
Most programs that accept transfers want a letter from the former program director. The PD is REQUIRED by ACGME rules to write a letter of recommendation whenever the resident wants.
Other hints:
1) There will be at least one attending who is willing to help you. Get help from him/her. Ask about what happened to previous residents who tried to leave.
2) Any conversation you have behind closed doors can always be denied by an unscrupulous person. Any conversation that you want to be recorded, you should write them a letter and keep a copy, or email them and CC someone important (The GME office, ACGME, etc).
3) Don't mention ANYTHING to ANYONE in your program you can't absolutely trust until you've found someone else willing to hire you. News travels fast, especially juicy news about someone leaving, and the PD will probably hear about it. When you've found a new spot, ask the old PD for a letter or have the new PD call the old one.
The old PD might badmouth you to the new one, but everyone knows that PDs get mad when residents leave, so a certain amount of that is expected. Just get good letters from other faculty members, and most of it will be ignored.