Right of first refusal

I published my first YA book (non-fiction) without an agent, submitting directly to a small but reputable publishing house. There's a clause in my contract giving them the right of first refusal on any subsequent work. My first book has done pretty well commercially, considering it has a fairly niche topic, and my editor and the rest of the team were lovely to work with. But I'm not sure they're the right publisher for my current project. I've written a YA novel that I know would be of interest to them, but I think I'd prefer to find an agent and try for a bigger publisher, as mine specialises in non-fiction and their fiction list appears to be just a small sideline. I'm worried they won't be able to market a novel as well as they marketed my last book. How binding is the clause on right of first refusal, as a general rule? Has anyone asked to be released from this without damaging their working relationship?

Today I just bit the bullet and asked upfront. A reply came within hours. They were very pleasant about it and agreed with me that they wouldn't be the best people to market it. So I'm fine to submit to an agent and I have it in writing that it's OK to do so. I'm relieved and also excited. YA fiction is a new adventure for me.