I sure as hell hope not. I have numerous ideas and quite a few outlines just waiting to start turning into a novel. Out of all of them, I'd say only two came from my mind without any help. When I listen to music, it's for many different reasons. One of them is to clear my head of all the BS and from that, usually a good story idea comes in my head. In fact, the title to each of my books is named either using the song title or lyrics from the song that was playing when the book idea came into my head.
I do agree that as long as it's not fan-fic you're trying to sell, all should be good. Not dissing fan-fic, to me writing is writing, no matter what it is, and having fun with fan-fic might help jump start your creative mind and get you writing more seriously. Never know.