What was the book?
You should talk about what you like to read. It doesn't matter if it's fiction or non-fiction, just don't lie to your interviewer because you think it's what they want to hear. If they ask you for more info on a book you didn't read or don't like that much, it'll probably be pretty easy to tell that you're BS-ing.
The only exception I can think of is if you told them you love something like Twilight or 50 Shades of Gray, which I believe were both on the NYT bestseller list.