When I interviewed at Loyola a couple of years back, I thought the program was mostly IMG's. Of course things may have changed since then. Regardless of that, I would say go wherever you feel that you can thrive.
You do want to go to a residency program that gives you a good volume of diverse cases, but there is also a difference between that and scutting you out to do frequent 24-30 hr neuro call and round for 10-12 hours everyday for all three years of your residency. As a soon to be neurologist in training, you want to go to a program where you actually leaves you some time to read (VERY IMPORTANT!) and one that values resident education.
Most Neurology programs are front loaded, meaning that you do most of the work during PGY2 year and then it gets much easiar. That's not the impression that I have of Loyola. Getting routinely hit with 20 consults during your calls eventually provides diminishing returns in the learning department.
