Yes, exactly. Even then, it's not necessarily a huge issue if you don't address the ingrained culture. I think it's the responsible thing to do, but the story should come first and not every story is or should be a great vehicle for doing that.
The difference comes in what you-as-author are aware of and how you show that in the text itself, as well as, crucially, the subtext. There's some self-analysis happens here, both of yourself and your narrative. Part of the writer toolbox.
We all indulge certain tropes that, together with others, can create boxes around PoC or female characters, or any other type of character. Most of these are, when you pick them apart, narrative wobbles. It's not the wobbles that are offensive on their own, it's the pattern and the way that pattern shows how you've constructed the world. Even then, a lot of it is subconscious and we all do it. It's just you have to look at your meta-narrative and think "how am I doing this? What patterns and tropes am I relying on more than others?" It's a very hard thing to do, but a very important thing to do. Nobody's ever going to be perfect at it, which is why I get titchy when critics pile on, because I think it just creates a chilling effect.
At the same time, we're responsible for what we write--it's a critic's job to dig into the text and find out what the text is saying. If you have a text full of straight, white dudes and all of them are very obviously straight and their dudeness is constantly put in the role of a heroic trait you've created, possibly subconsciously, a pattern that amplifies that ingrained sexism. It's not that you have the dude, it's how you use the dude.
ETA: I was mostly responding to Viridian's first post, but absolutely yes you're going to get criticized because that's what happens and the best (possibly only) policy is to basically shut up and take it/ignore it. Maybe you did create an unfortunate pattern, but every story has these. There has yet to be a perfect feminist story, or perfect racially-cast story. There won't be. It's OK for there to be cultural discussion around that. Your author persona has patterns, too. If everything you make is a sausage-fest (lookin' at you, Sandler), then that says something different than if you have one story that's a bit sausagey but a bunch of others that mostly aren't (to stay visual, look at Whedon: the guy generally writes women well, I'd say. Every so often, he blops. Sometimes there are extenuating circumstances, sometimes it's just 'wow, this isn't Whedon's best work.' Different pattern).