In general, the reason antibiotics act only on bacterial ribosomes is due to the differing structure between the RNA in bacterial ribosomes and the ribosomal RNA in eukaryotes and archaea.
Specifically, streptomycin binds to the S12 protein in the 30S subunit (smaller of the two sections) in the bacterial ribosome. This protein either does not exist in eukaryotic cells, or is structurally different and streptomycin cannot bind to inhibit protein synthesis (I am not sure of which. Would need to do a bit of searching through an academic database).
The specifics for chloramphenicol is that it binds to the A2451 and A2452 residues on the 23S rRNA in the 50S subunit (the larger of the two sections) and prevents peptide bond formation by interfering with substrate binding. Again, this drug binds specifically to this section of the bacterial ribosomal RNA only. There is no equivalent binding site in non-bacterial ribosomes.
