Legally, you are supposed to pay tax on your fellowship*. At most schools (perhaps all?), they will not deduct it for you, and the onus is on you to pay quarterly taxes. That said, some people do cheat and not pay. I even know a few who claim their accountant staked his reputation on the legality of that position. But the IRS literature is very clear, and those students are tax evaders. Bottom line: pay quarterly taxes.
A gray area IMO is whether you have to report the cost of your health insurance premium and pay the tax on that. In my reading the IRS literature is vague on this, and it's not entirely clear to me whether the premium is paid by your employer/university (tax exempt) or whether that contribution is from your fellowship (taxable). You could also, perhaps erroneously, make the argument that the health insurance is required for your coursework (see asterisk). In any case, it's a relatively small slice of the fellowship.
*you may deduct from your gross income the cost of required expenses for coursework (e.g. textbooks, computer, etc.)