I used to be a bit of a hoarder and it was kind of ruining my life (my closet shelf caved in due to books, no one could come into my bedroom due to books, I was spending insane amounts of money on books). I thought I was doing the books a favor. It's important to remember that, if you really have to, you could always buy them again on Amazon or download them (legally, of course, coughcough).
Here is my decision process:
1) Is it one of my favorite books of all time? Do I constantly quote it, suggest it, reread it? Do I consider buying the 50th anniversary hardcover edition of the book just because it was so damn good? If yes, keep the book.
2) Is it in my shelf because it makes me look smart? If yes, toss the book.
3) Have I owned it for over 2 years and never read it? If yes, toss the book.
4) Is it badly damaged/smells bad? If yes, toss the book (even if it was amazing).
5) Did I get it because it was free/cheap? If yes, toss the book.
6) Did more than one person urge me to read it and I am genuinely interested? If yes, keep the book, but don't let it just sit around forever.
Then, go through the books you decided to keep and repeat the process. There will be some in there that you shouldn't be keeping!
And remember, by "toss the book" I don't mean throw it in the trash! D: Oh no! I mean donate it to a local library/care home/hospital/day care/school/boy scouts/church/goodwill/whatever! Some churches and schools have programs set up for donations like this. There's a church here that sends books to Belize every year. Books are much more useful in the hands of someone who will actually enjoy them than in a box in your closet.
Getting a Kindle did a strange thing to me: I suddenly didn't want to waste the hard drive space on my computer on books I might not read (I download the books to my computer first because of wifi issues). Then I went, "Wait, why am I more concerned with hard drive than my real life space!?" Yeah...
My major problem was that I wanted to be impressive. But remember, any old fool can buy some fake decorative books, you know? Let your intelligence shine through in your words and actions. Getting rid of some clutter will make you feel so much better about everything. It's amazing.