Thinking more. It's the electrons that are moving. They can drag the bar up or down along with them. So the assumption is that all forces exerted on the bar by the B field is suppressed as the bar is forced to move linearly in one direction and therefore there is work done.