You really can’t put your nose to the grindstone for 18 more months? One person shouldn’t have THAT much of an impact on your training - if it’s a cultural thing, not much can be done. But at the end you’ll be trained and you can leave.
Or you can quit and work as a generalist in your field. In most subspecialties, though, if you leave a fellowship midway you essentially close that door forever. Your move!