Think of potential energy in the form of heat, where heat = energy...If you are making bonds and releasing heat you are lowering the potential energy because you used that energy to make the bonds...
Now if you break a bond which requires energy in the form of heat again, heat is taken in and potential energy increases...just think as potential energy in the form of heat...
Endo.= heat absorbed = energy absorbed
Exo.= Heat released = energy lossed/released