The amide's lone pair won't H-bond MUCH (maybe to some extent) because it has a resonance structure where it's involved in forming a C=N double bond, which gives the N a partial positive charge - not something a partially positive H would like to interact with.
The relative electron deficiency of N (compared to N's in other compounds like amines) will pull electrons away from the proton bound to the amide nitrogen, giving it a partial plus charge. This H (from primary and secondary amides) will be able to H-bond with the partially negatively charged oxygen atom.