depends, depends, depends. For all of these it really does come down to the department. The information isn't always easy to find on websites; you might consider e-mailing current grad students or asking during interviews.
This is for us, but probably isn't indicative of anywhere else.
We're paid every two weeks, we're not taxed and I think we should be, and I don't think we get paid during the summer. Summers for us usually entail us teaching a course for additional money. The bonus there is the more courses you teach, the more money you get for teaching subsequent courses.
I have a teaching assistanceship (TA), get paid monthly, and it is taxed (amount depending on number of exemptions). I generally don't get paid during the summer unless I get one of the limited summer TAs. Although I also have a couple department related clinical externships that pay me hourly.