I think that a lot of PIs come across as cheap when it comes to this stuff, but mostly it is being old school. Many PIs hate kits because "back when they were students they had to...." yadayadayada. They think that kits are mindnumbing since you don't have to actually think about how anything works. Also, if something goes wrong, you have no way to figure out if buffer C went off, etc. Keep in mind that I love a good kit that saves time...
The glassware, stationary, etc., could also be an old-school thing. However, it could be that they are choosing to have a certain tech in your lab, or to pay for an expensive facility (i.e. confocal or something) rather than spend money on "useless" things like paying someone to clean glassware or buying stationery.
Above all, this stuff should NOT be a factor when picking a supervisor. Would you pick a spouse based on if they loaded the dishwasher?
You should look for someone who has a good track record with students in your position (i.e. does well by summer students vs. using them for grunt work, or his/her grad students never publish before the 3rd year of work, etc.). Then, pick someone who has research that facinates you.
The worst mistake I have seen my fellow students make is to go to a lab because it was the lesser of the evils during their rotation, or because they liked the other students. You have to have a passion for your work, otherwise it will be hard to justify missing Friends to go and transfer your western blot overnight, rather than leaving it until tomorrow. 🙂
(Sorry this completely deviated from your post-good luck with whatever decision you make 🙂 )
Treg