You don't say what the memoir is about or what sort of relationship you have with the doctor and staff. If you had and still have a good relationship, this should be simple. My hunch is that your relationship may not be so good at this point, and what seems simple could become complicated with lots of layers of possibility.
Since you question intrigues me, I did what you may have done and Googled "who owns rights to medical records". Each of the dozen results I scanned gives patients the right of access to records, but none discuss what patients can do with those records. Perhaps the more interesting aspect of the results is that it is becoming increasingly unclear who actually "owns" them, and who has what rights. It's complicated as physicians are employed by practices, which may be large corporations. Insurance companies have claims. There are lots of fingers in this pie.
Here is a summary of the options as I see them:
1) the record is about you. You don't have to ask permission to share it for medical purposes. If there are no names, it seems reasonable to assume you can share it. All the privacy regulations are to protect you from having THEM share it! This is the reverse.
2) you could ask the doctor and explain what you are doing. Chances are, there won't be a problem. But ... you must be prepared for a "no."
3) you could just do it and deal with consequences later. Which could get costly.
4) you could ask an attorney. Which could get costly, since it appears there isn't clarity at any point about this matter.
5) you could just tell what it said without specifically quoting.
Perry Foster may have dealt with this matter in his memoir, HANDS ACROSS MY HEART. I can't recall if he's a member here or not, and I can't access the member list. You can find an e-mail address for him on PolkaDot banner.
Considering the fact that there isn't much clarity at the moment, especially if you are dealing with a large practice, I'd be inclined to "just do it" -- unless the entries are damning. If you are documenting what could be considered malpractice, that may be touchy, and it may be better to tell, not show.