If you went through and drew out all the electron flows, you'd see why they both direct ortho and para. Halogens are just one atom pulling electron density towards them, while carbonyls and other pi systems can withdraw electrons through resonance. It really comes down to the fact that it is awkward to draw a double bond to a halogen.
For the most part, there were 3 things I tried to keep straight in my head:
-If it's a halogen or NO, it deactivates the ring (through inductive effects) and directs ortho and para.
-If it was not a halogen and had any lone electrons (ie, O-R, O-H), those electrons got donated to the ring, activated it, and directed ortho and para.
-If it was a group with double bonds (ie carboxylic acid, NO2, etc) and no free electrons immediately next to the ring, it withdrew electrons from the ring, deactivated, and directed meta.