That is the problem of the whole piracy debate. There is a difference between legality and morality, and there are illegal acts which are clearly morally identical to legal acts (and in reverse obviously). It's not that easy to explain why downloading a book morally wrong but borrowing a book is not. Because to the reader, it's the same act, he's reading a book for free. (And to the author it's also the same, he's not getting payed)
If I buy a book, read it, and give it to a friend, there's still only one copy of the book involved. If another friend says, "Oh, I heard that book was good, did you like it?" and I tell them yes, I no longer have a copy to give them. If I want to read the book again later myself, I have to either borrow it back (in which case my friend now no longer has the book) or I have to go buy another copy.
Sure, you can wait until your friend is done with the book and then borrow their copy. You can check a book out of the library. (Keep in mind that paperback books tend to fall apart after about 20-30 checkouts, so the library /will/ eventually have to buy another copy.) But in all, the supply of books does not change unless the publisher prints more. If I give away my copy of the book, then I no longer have a copy of the book. If I want a copy again, I either need to buy one (money to the publisher), or else retrieve my old copy or find a used one (in which case someone else no longer has their copy). Rinse, repeat.
With pirated eBooks, that's not the case. I have my book, and give a copy to a friend /and I still have my copy/. Now the other friend comes up and says, "Oh, I heard that book was good, did you like it?" and I tell them, yes, it was good, and hey, here's a copy! Now there's /three/ copies of the book off of that one initial one. The reader -- not the publisher or author -- is controlling the supply.