Even if the profession is going downhill, it shouldn't stop you from becoming what you aspire to be. If it means you will have to find a job outside of your local area, then do it. There are several pharmacy staff and manager positions hiring at retail pharmacies when I did a job search.
I say, ignore what pharmacists are saying. I had a pharmacist tell me that it is harder to get a job now than it was 10 years ago.
I don't know if you want to ignore them outright, there are valid points being made (such as the one that the pharmacist told you).