It depends on who you ask. The federal government has specific criteria for what determines "underserved," and this is typically done on the population level based on the number of healthcare providers available to the population in a certain area.
In colloquial use, though, "underserved" can mean a variety of things.