The signs get kind of funky, but here's a simple example that might help.
Say we have a positive source-charge +q. The E-field from this source-charge points away from it, in the direction that a positive test charge would go.
If we drop a positive test charge into the near vicinity, it will move away from the source-charge (like charges repel). U = k * q(source) * q(test) / r. The "r" term gets bigger as the test charge moves away, and the test charge decreases potential energy as it moves.
If we drop a negative test charge into the near vicinity, it will move towards the source-charge (opposite charges attract). This time U = k * q(source) * -q(test) / r. The negative sign makes it so that as r gets smaller, U gets more negative. The test charge decreases in potential energy as it moves closer to the source-charge.
Charges always move from higher to lower electrical potential energy, but the direction may differ depending on the sign of the charge.