This would be the best way I can explain it.
Look at the Halogen groups, and the monoprotic acids you can form with those. Ignore H-F momentarily.
If you look at the pKa of those monoprotic acids, you'll see
H-Br
H-I
H-Cl
H-Br is stronger than H-I which is stronger than H-Cl
Now the reason for this is because, the reaction I will layout.
H-A + H2O ---> <---- (my best attempt to show reversibility) A- + H3O+
So anyhow, we can see here that the Anion forms. All of these acids are strong. As we can expect, their conjugate bases will be weak, and the reason they are unreactive, is because they are stable.
out of all 3 atoms, I can BEST stabilize that negative charge when it is an anion, and the reason is because it is the largest atom, and so can "spread out" the negative charge better. Unlike Br-/Cl- which are smaller, and thus would be slightly more basic (but are still considered weak bases).
I hope that helps. If that does NOT help, I can try to clarify more. Let me know.