I’m not an expert, not even close, but last year the swallow nest under my veranda fell after we got some torrential sideways rain. The babies were OK, fortunately, and I rescued them before my dog found them. I wasn’t able to put the nest back as it was, so I cut the side out of a plastic milk carton, put the nest and the babies inside, and hung it as close to the place where it had been built.
The parents were very agitated throughout this process, but when I was done they came back to check on the little ones. Weirdly, they accepted the new nest and raised the young ones with no further misadventure.
I realise that some species might not be so accommodating of such interference, but if I hadn’t done anything they’d have died.
To answer your question, if they don’t go back to the nest once they’ve left it, you lose nothing by hanging it up. If they do go back, same outcome…