Don't feel discouraged! If you're finding the EK 1001 book difficult I'd say definitely try to go back to your physics book to solidify your knowledge of concepts. Because from my own practice I find that the questions in the EK 1001 book just really aim to make sure you understand the concepts more than they try to ask blatantly difficult questions. If physics isn't really your forte (like me) I would try supplementing EK with another resource, say Khan Academy videos? I'm not sure if you own Kaplan or TPR but they do a much more thorough job at their physics sections than EK in my opinion.
It's not too late, don't worry! But again, I think if you're struggling with EK 1001 you need to go back and brush up on making sure you understand physics on a more conceptual basis (the why's and the how's) rather than memorizing which equation to plug based on info provided in the text. If you do that, then it will be easier to recognize problems and pinpoint what approach you need to take to solve it. I've found this to be much more effective than just mindless plug and chug.