I don't think it's a disadvantage, on its face. I mean, I don't think that most women would be reluctant to buy erotica written by a male author. In some cases, it could be a marketing advantage (m/m, for example).
And I would absolutely advise against writing under a female pen name. If you want to go neutral (using an initial or something), that might work, but there has been kerfuffle in the past where readers felt deceived by someone switching gender, and it wasn't pretty.
But I do think it's important that you've recognized that the market is mostly female. There are definitely exceptions, and at least on publisher (Ellora's Cave) has got a line they're trying to aim at the male audience, but in general, you're probably writing for women. So if there's any disadvantage for you, it may be that you're writing for a group that you may not fully understand. I don't mean that no man understands any woman, or that no man understands women as a mass audience. But in my experience on this very board, there are a significant number of men who seem to consider women as some sort of alien species, unknowable and unexplainable. If you're one of those men, yeah, I think you're at a disadvantage, unless you can find your way into the niche erotica-for-men market.
So to some extent, your question is unanswerable. Can you write erotica that a significant number of women will want to read? If you can, I think you'll be fine.