depends, depends, depends. For all of these it really does come down to the department. The information isn't always easy to find on websites; you might consider e-mailing current grad students or asking during interviews.
This is for us, but probably isn't indicative of anywhere else. We're paid every two weeks, we're not taxed and I think we should be, and I don't think we get paid during the summer. Summers for us usually entail us teaching a course for additional money. The bonus there is the more courses you teach, the more money you get for teaching subsequent courses.
FYI, your stipend is most likely still taxable income, even if your university never takes taxes out. You're still technically obligated to report it and pay taxes on it.
I have a teaching assistanceship (TA), get paid monthly, and it is taxed (amount depending on number of exemptions). I generally don't get paid during the summer unless I get one of the limited summer TAs. Although I also have a couple department related clinical externships that pay me hourly.
I have a grad assistantship (GA) with a stipend that's paid bi-weekly, with tax withholding. It is considered regular, taxable income.
