It depends.
As long as you are board certified, it probably won't matter for most private practices. In the community, one's academic pedigree isn't as important as being skilled, dependable, and having a good work ethic.
In academia, especially in upper echelon teaching hospitals, you might have a problem getting hired, though not always. Some ACGME programs may not want an AOA-boarded physician on their clinical faculty, and vice versa.