To me it's more of a question of why does it hit writers so hard... Sort of depressing that all we have is words, we can't let people into our imagination in an interactive way like musicians and filmmakers. So there is no real experience to sell, only words.
And words, well...Information is free. That goes for education (you can copyright textbooks but not physics), no logical reason why it shouldn't go for fiction, really. It's not a tangible product. Neither is music nor films in the digital age and as far as I know, those industries are still thriving despite the fact they are pirated tons more than books are - reading is a minority thing as it is.
You can't perform a book live or stream it in a cinema to have people pay for that. Authors don't have fans in the same sense musicians and actresses do. So there won't be this sense of support, of buying a book because you want to see this particular author succeed. No hype over the person rather than the books, of why this author is the best, most talented etc. So the author might as well be anyone in most people's eyes.
And anyone can write a story. A good one, maybe not, but everyone has an imagination all the same. And most people's tastes in art is not refined at all. So all it takes to succeed as far as popularity to the masses is imagination.
So how do you make imagination valuable, as if yours is better than others somehow? Tough. I think this battle is already lost. Screenwriting might be the new way for novelists in the future, with films becoming more dynamic and complex. And someday there will maybe be a way to create your own virtual world and have people experience it for a fee, and be inside the story like in a videogame. So being able to create good characters and tell a good, realistic story will be a big thing. And so being a good writer will be rewarded because not anyone can do that.
Until then, I think authors will be underdogs, unless they're the 0.000001% whose story really blows up. Or have connections. Or crazy luck.
