How do you incorporate backstory into a short story, in your opinion?

I've been working on a short story for a while, and though I'm not finished yet, I've been working on revising the beginning. Currently, I have the backstory at the beginning. The backstory is incredibly important for the rest of the story, however, people say never to start exposition with backstory. Although, they're usually referencing novels, I'd assume it would be the same for short fiction.

The backstory is explained a few paragraphs into the story, right after I established the setting and atmosphere. It's a few lengthy paragraphs long, with some present action in-between to break it up. It finishes at less than a thousand words, and overall, I estimate that the entire story itself will fall between 4.5k to 7k words. However, without the backstory at the beginning, I feel that the rest of the story would be irrelevant or uninteresting.

What are your thoughts?