Well, as usual with workplace issues, it's not quite as straightforward as it is when you're dealing with students. I can count on one hand the number of people at work who are not what your average American would define as white. In my job category, I'm literally the only minority, and these conversations happen in the back, not the front, because the staff up front have to watch their mouths in front of the clients. If management addresses it, it'll be 100% obvious who spoke up, and I'm not so sure I want to deal with the potential repercussions.
The other issue is that although I trust management to listen if I do say something, I don't trust their ability to really change much. We've brought up more serious issues directly related to work problems (like someone pretending to clock out, staying clocked in for an extra few hours, and then coming back later to clock out after having been somewhere off site) and nothing ever seems to change or happen. So this all leaves me with the feeling that I may be sticking my neck out for absolutely nothing. I may say something anyway out of principle and because I'm so sick of dealing with these little subtle comments that have followed me in school, work, and in my personal life for years, but I do hesitate because this will be prime fodder for the toxic gossip machine. If it continues or escalates, I think I'll get over that hesitation pretty quick, though.