A lesion of the vestibular nuclei or nerve produces a vestibular nystagmus with a slow deviation of the eyes toward the lesion and a fast correction back in the opposite direction. The way I understand it is that a lesion disrupts the balance between the two sides of the vestibular system so it is basically as if the vestibular nuclei on the opposite side is active (because there is no counteraction from the lesioned nerve or nuclei). This makes your eyes drift toward the lesion. Then when your eyes figure out that the head didn't really turn, there is a fast nystagmus in the direction opposite the lesion. As for the caloric test, I just remember that cold water simulates a lesion on that side and warm water simulates vestibular activation on that side. The direction of the fast component of the nystagmus can be remembered by COWS: cold opposite; warm same. This is from the Kaplan neuro section.