Its not crazy to walk away. If the characters are from the same "series" so to speak... the contract offered would like contain a "first refusal" clause in it, which would prevent you from publishing any other books with the same characters or world, with another publisher without first giving the opportunity to Carina... so it they arent right for the other two books, then they are not right for this 1st one.
You could try to negotiate out that clause, but I imagine it would be difficult and you'd likely want an agent or attorney to make sure it was re-worded properly.
Well, you could alter the 1st refusal clause, that if they refuse any subsequent books in the series, that digital rights to the series revert to you. That would let you take the series (with its established fan base) to another publisher.
I'd also want a clause where if the book didn't sell very many copies in a year, I'd want the rights back, as well. Maybe a different market outlet would let you sell more copies somewhere else.
I can see no reason why they wouldn't agree. After all, if your book isn't selling in their market, they aren't making money on it anyway.
Also, since they aren't publishing in print, I'd make sure to retain print rights. If they want, they can have first refusal on putting the book into print, with one of their other lines, but if they refuse, and someone else wants to print it, then they have to allow it. They should only get to hold what they are paying for and selling, which is digital publishing rights.