As far as the quote, it's obvious it could be interpreted many ways, although during my writing today I would say it really hit home as I started a story years ago RIGHT after an event, but only got half way done typing it. Today, as I was trying to finish the story, I realized it wasn't so much of a story anymore because I couldn't remember half of what happened or what the amazingly funny incident was that I wanted to share originally! I've also gone back and re-read a story that was typed years ago, however while I was reading it I was like "holy cow, I totally forgot about that! I'm so glad I wrote that right away or I would have never remembered all that happened!" (right now I'm thinking about the birth story of my son as an example... when I retell it to him in person 8 years later, it is much less interesting and WAY shorter!)
As far as reading your own memoirs though, it might be that when you originally typed it, you told the story, fact for fact, and maybe didn't put in things like motivations or reasoning or feelings with it. Often, the tale is crazy or odd or unique enough that it's really interesting and astounding, but if not told with feeling then it can be less exciting to read.
For example: Today as I was leaving a meeting at work, I saw that it was pouring rain outside. My coworker stopped and hesitated at the door, at which point I noticed she wasn't wearing a coat so I offered her a ride to her building. It was a short drive so to finish our conversation, we sat in the parking lot for a while to finish our conversation before parting ways.
Rewritten: I was leaving the meeting feeling energized and excited about everything we had talked about. The people I had just spent the last hour with were new to me, but it wasn't our first time meeting. My initial nervousness talking to them had worn off and I was really enjoying this new friendship I was developing with a new group of people I hadn't worked with before. Two of us were walking out of the building talking to each other, and as we got to the door, I noticed that she stopped and said "Oh no." I saw the rain outside, but it took me a second to register why she seemed so put out by it. She didn't have a coat on. "Do you have to walk back to your office or are you parked here?" I asked her. When I found out she was going to have to walk to her office, I offered her a ride. She didn't want to inconvenience me, but to be honest, being able to give her a ride to her office made me feel like I was investing in a friendship, and friends were in short supply for me at that moment.....
See what I mean? Is that possibly what you are seeing? Are you describing facts in your book, or are you describing emotions? Orrrrr.... could you be assuming that people don't like your book while really they are just lazy and haven't had time to give you a good review of them?