The conventional wisdom is that yes, it will hurt you.
While USCE helps you learn about how a US hospital and medical education works, the major benefit is to obtain letters of recommendation about your clinical skills from US faculty. Many US faculty will not recognize letters from foreign faculty when evaluating your residency application.
If you have already graduated from medical school, then you are not eligible for US electives. About all that is available are research positions or observerships in which you have no clinical contact with patients.
However, many applicants have no USCE and still manage to match. As noted above, if you have already graduated from medical school, then you have little option but to do well on the USMLEs and try to get some US experience, even if it isn't clinical.