First, nothing in NYC is cheap. The cheapest hotel I've found in all of Manhattan that is remotely legit is the Ramada on Lexington and 30th. It's not that close to Columbia, but everything is pretty doable with the subway. From there, you would just take the downtown NRQW line to Times Square and hope on the 1 to get to Columbia.
Just stay at a youth hostel. I'm staying in one for my interview next week. The night before the interview I'm paying about $60 for a private room. The night after the interview, I'm switching to a shared room and paying $30. The rooms might not be that fancy and you probably have to share a bathroom, but you can't beat the price.
I used priceline.com to get a hotel in manhattan. It was still fairly expensive, but about 75$ less than the listed price. The area around Columbia doesn't have many hotels, so I recommend staying in midtown. You can take the A express train right uptown and get off across the street from the med school. It's super easy. Good luck!