I know it's hard to have a contract offer in hand only to find the house doesn't look very promising, but it's the better choice for the long game. This house/agent/whatever is unlikely to sell your book. Taking the risk will cost you at the least the rights to your book and possibly actual money in the future--small houses that don't sell books can't keep the business afloat and have to find income somewhere. Sadly, too many turn to their own authors. Or they close, taking the rights with them, forever locked in creditors' hands.
It's hard, but keep trying with agents and well-known, established small houses that specialize in your genre. If your first book doesn't find a home, trunk it and write another one. Never hurts to have other books on hand anyway.