General Surgical fellowships, other than Plastics, are not all that difficult to match into if you do the right things -- letters, research and such. From what I understand, there are more spots than applicants for CT fellowship each year. Given the general downswing in people going into Gen Surg due to lifestyle, you gotta figure there are fewer people going into the tougher lifestyle fellowships (CT, Surg Onc) than ever before.
If you're thinking about going the route of your Dad and brother, then one other thing you need to think about is the fact that interventional Cardiac procedures are advancing at a rapid clip to the point that in many cases they are supplanting or replacing the need for people to go under the knife at all. My dad is an internist, and 2 of his CT surg buddies specifically counseled me away from CT surg, if what I wanted to do was in the heart. Specifically, they told me that if I liked hearts, the future would be in Cardiology, not CT Surg. Don't get me wrong, there will always be a need for talented, capable CT surgeons who can do the job well. It's just that the evolution of the care of patients with heart disease, whether it be coronary or valvular, more and more is evolving toward the least invasive way to deal with things. Some interventionalists at Harvard already do percutaneous bypass experimentally, and there will one day be valves deployable by catheter. One of dad's buddies lamented the fact that as interventionalists get more and more aggressive, there were fewer cases for him to take to the OR.
One other thing -- getting "invasive" Cards is different than interventional. Invasive guys do diagnostic cath, but not intervention. Interventionalists do PTCA, stent, rotablator, laser myomectomy, peripheral intervention, etc. etc. If you still need a knife, you could always do EP and implant AICDs and pacemakers. Indications for those procedures are exploding faster than there are EP docs to do them.
Getting interventional Cards is probably the toughest thing to get as an IM subspecialist -- there are so few spots it's crazy and doesn't even meet the demand. My philosophy -- go to the best IM program you can get into, then the best Cards spot, and so on and so forth. To that end, it's definitely more competitive than Gen Surgery, or even CT to the extent that there are always open CT surg spots, but no interventional Cards spots.
I don't know what year of med school you're in, but focus more on figuring out what you like and dislike about each rotation during third year, and realize that there's a lot of overlap (e.g. procedural IM subspcialties or more "cognitive" surgical subs like Trauma/CC). Find what fits you best.