Think of it this way.
If you had two nitrogen atoms with lone pairs fighting over to create an h-bond with a proton (covalently connect to some R group) it would look like this:
N:--H--:N
Both nitrogens would "give up" the same amount of electron density and there would be no clear winner.
But now, imagine nitrogen fighting with oxygen over the same H:
N:--H---:O
Here nitrogen is "willing" to give up more of its electron density to hydrogen than oxygen will. Oxygen likes to hoard its electron density. Nitrogen can't. (Because it has fewer protons, it can't hold the electrons as well.) So hydrogen gets a bigger share of the electron density by sharing with nitrogen instead of oxygen. i.e. Hydrogen is on more equal footing with nitrogen.
The same is true for fluoride but even more so because it is less willing to share electron density than oxygen.