Very quickly - 6 months maybe.
Most fellows graduating are still on the steep learning part of the curve when they graduate. I think I learned how to be a pain physician the first two years AFTER fellowship. That is an important time to be practicing pain.
Personally, I feel you should practice as pain physician for a while - solidify your skills and decision making. Also, a lot of our fellows towards the end of the fellowship think they don't want to do pain - and I get it - Tis a very tough, mentally draining year and by the end, you are DONE. But after they graduate, work as staff in pain for a while, they all realize they really enjoy it once the crap of being a fellow is gone.
You did anesthesia for 3 years. That will take longer to loose and forget.