Hmm, it depends! Where are you making these detail-oriented mistakes? Is it in the first four passages (the fast ones) or the ones that you take more time on?
In general, any time that you have left at the end of the verbal section isn't very useful - it's tempting to spend it going back to the passages and checking your answers, but you risk forgetting integral passage ideas and changing correct answers to wrong ones (which happens overwhelmingly often). I generally do advise slowing down the reading but not the questions. This way, you pick up as many details as possible and get a better idea of the overall passage, without giving yourself enough time to "talk yourself into a wrong answer" in the actual questions.
But if you don't end up slowing down, you can at least make the most of any extra time at the end! I advise choosing the hardest of the passages and spending all of your leftover time reviewing that one. That way, you're being as efficient as possible while minimizing the time you spend skipping around!