I just think about it the way that Tris-HCl buffer is prepared. The chemical itself is neutral, but it becomes protonated and thus charged at the lower pH. In solution, it's charged. When the chemical comes out of solution, it brings nearest anion with it. Chloride is a good anion because it's harmless. Fluoride is actually beneficial, but I wouldn't want to be messing around with HF any more than I had to if I were running a drug company.