OK, I want to say something a little more encouraging.
To OP, hey, don't treat it like it is an all-or-none decision. Aside from your potential problems in studying in general, please remember, it is not like "you have to do it now or you will never do it in your life time" type of things. Try to treat it like a journey. Some people may have a shorter one, some people may have a longer one, to medicine. It took me almost 20 years.
In my high school, I hated everything in relation to math and sciences. In college, I took 2 required stats classes, and 2 required math classes, and I refused to take any additional math or science classes beyond those 4. In fact, I got two Cs for the second stats and math classes. Then I majored in applied psychology, and worked in human service professions for more than 10 years. During this long process, I changed. I never expected that one day I would receive a graduate degree in math-related disciplines, and then use it for my main source of income. I never expected that, one day I will decide to go to med school in the US, and took all the 9 required science classes in one year, with straight As. Back in the 90s, I hated all those subjects, as I just can not get my head around. My journey led me to some other places, and then led my back to where I began. And this time, things look differently, they all make sense, perfect sense.
What I want to say is, after you switch to other fields, you can see whether you are struggling or not. If you do very well, then perhaps it is not a good time for you to do science. You can go on another journey to see where it leads you to, and then revisit science again a few years later. Things may begin to look different. However, if you find yourself still struggling, then you know you have other issues to work on - it is not science, it is your study approach.
Good luck!