This is a weird concept and does seem counterintuitive. Since Phase 2 of the myocyte AP is balanced by Ca2+ influx and K+ efflux one would think that inhibiting Ca2+ would "unrestrain" K+ efflux.
However, you have to remember that the forces driving K+ out of the cell are both electrical and chemical.
The chemical forces (e.g. concentration gradient) is essentially always in favor of K+ efflux as [K+] is much higher intracellularly than extracellularly. What restrains K+ efflux is the electrical forces inside the cell. That is what causes an RMP.
The idea is basically the same during phase 2 of the myocyte AP. For the discussion below, assume that both K+ and Ca2+ conductance is high (e.g. the channels for the respective ion is open).
I like to think of it this way:
At the plateau(phase 2), RMP is roughly +5-10 mV. At this point, K+ efflux is almost completely determine by the concentration gradient and fairly unrestrained by the positive intracellular potential (in fact, the interior is positive and the electrical forces almost push K+ out of the cell).
As K+ leaves the cell, the membrane potential will rapidly repolarize (become more negative). This will increase the "restraint" on K+ leaving and lower the chemical (concentration) gradient too.
This is where Ca2+ comes into the equation. During the plateau phase, Ca2+ and K+ are basically swapping places. Because calcium is 2+ and potassium is 1+, when the two ions "swap" and "balance" each other, the positive interior charge is maintained at +5-10 mV and it keeps pushing K+ out of the cell. Basically, for every one Ca2+ going in, two K+ have to leave to maintain the plateau.
If you block some of the Ca2+ channels, that "swapping" is slowed down, resulting in an increased Phase 2 and ERP. (I think of it as K+ is leaving more slowly because of the alteration in electrical and chemical forces explained above.)
It is basically the mirror image of what happens when a class Ia or III agent is used that blocks K+ channels. The plateau is maintained because the "swapping" is prolonged. Just because the K+ channel is blocked doesn't mean that Ca2+ rushes in and the RMP jumps to the Ca2+ equilibration potential of +120 mV...
Hope that helps.