I agree. I love to ask people about their books and I'm happy to be asked about mine. Most of mine are in the lounge-diner and I've had the following comments from friends on their first visit:
"Wow - you can tell this is the house of a well-read family!"
"Now I know why you know such a lot about so many things!"
(That particular friend borrowed a number of my [adult] books and I enjoyed discussing them with her when she'd finished them.)
Some friends of my son and daughter commented:
"What a lovely, cosy room. It's just like a library!"
"Where did you get all these books and things? It's like being in a charity shop!" (The boy who said that meant it as a compliment.)
"Oooh - so many books! Could I borrow one, please?" (The girl who said that went on to borrow several of my children's books, though she never chose an Enid Blyton.)
I've also had interesting conversations with a washing machine repair man and a plumber about the books they loved as children.