As you mentioned above a lot of times you make your presumed diagnosis on a CT and lose the patient on followup. Thus, you will never find out that your call on CT was a correct call or not. You won't have the pathology results for correlation.
Many referrals get overread of their imaging studies or follow up in tertiary care centers. Let's talk about neuroendocrine tumor of the pancreas. You may see one of these on every 500 CTs in community. Hence, you may see one or two of these cases on your whole yen month body rotation. Most likely, you won't eventually even find out that the hyper-vascular tumor that you called on CT was a NET of pancreas. You read somehow in vacuum. No pathology results, no other modalities, nothing. This is the typical scenario in community (like my current practice). Now compare it to a tertiary care center. There is an oncologist who exclusively sees pancreas NET. You go to the body room and read 20 pancreas NET CTs in one day (the day of his clinic). All cases have CT, may be MRI, Nucs study, EUS, Biopsy results and post treatment scans. Also there is a weekly NET tumor board with pathology, surgical oncology, med oncology, radiology, .... paticipating. Your attending has seen 1000 of these case so far. He has also published a radiographics article about pancreas NETs. He gives lecture in AFIP or RSNA about pancreas NET tumors. The same for Carcinoid tumor, ... For something like HCC you read US, CT, MRI, you participate in TACE or Y-90 or ablation in your IR rotation , you follow the post treatment scans, .... Your IR attending happens to be one of the Gurus in TACE. Don't you think there is a difference?
The best radiologists have become the best because they constantly attended multi-disciplinary conferences and they receive constant feedback by following their cases. Reading in vacuum is like doing something wrong and not getting any feedback. Do you know how many liver tumors I call on MRI or CT and never ever find out about the final pathology results? I have been on both sides. As a resident and fellow in a tertiary care center and as a pp radiologist. Believe me. I feel that I am losing or at least not improving a lot of my imaging skills because I am not in a tertiary care center.
Subspecialty training is like a gem. Never take it for granted. The chest attending who has seen all sorts of ILD with pathology correlates or the Neck imager who pretty much lives in the soft tissue spaces of the neck are way different than your average community radiologist (like me) who reads a CT neck just before his kyphoplasty case and an HRCT lung after he sings the ankle MRI.
All of this doesn't mean that you can not get a good education at a community hospital. But if you can match in a tertiary care center, go for it. It is worth it. You will appreciate it once you finish training and start your job.