This is a very broad question to answer, especially without knowing the Caribbean school or island you are contemplating or the specifics of your mental health condition (which you may reasonably prefer not to share here). Every school and island will be different. Also, I wouldn't want to rely on any forum for this answer. I'd want to speak directly with the school you are considering to find out what supports and services are available, both through the school itself and on the island.
Some schools may have robust supports available through the school, including psychiatrists, psychologists, and social workers. Others may have nothing. I imagine some islands may have very few if any psychiatrists who practice there, depending on the size of the island. Pharmacies on the island may have difficulty obtaining some of the newer psych meds, so you may have to bring what you need.
Also I would want to be very sure the stress of med school won't destabilize your condition, and I'd look for a school with a clearly written accommodation policy for disabilities in case you require accommodation.
In short, there is a lot to consider and some potentially serious consequences if you destabilize psychiatrically on the island. I'd want to be very careful about choosing this path.
