They are all just different, in my opinion. Most people probably consider physical chemistry the most difficult, but coming from a strong math background, I found biochemistry and organic to be more difficult.
It depends most on each individual class and professor. Organic could be the easiest or the hardest, completely dependent on the curriculum.