If I were Publisher A and heard this, I'd likely call Publisher B to verify it. That, of course, is going to somewhat insult Publisher B (thinking you're possibly more interested in working with A.) It's a VERY small community out there, and people in the same line of work often talk. Now, there's nothing WRONG with wanting to make sure that everyone has had a chance to look at (and make an offer on) it. Naturally, your goal is to ensure the best deal for yourself.
If I were in your shoes, I'd stall for a little bit with B, by asking specific details about the offer, and then email A to ask the status. If it's going to be months before they decide, then you'll have to take the offer from B at face value and decide whether it's the publisher for you. Or, you could try an agent--although a lot of agents don't want to step on toes by walking in at the last minute. But if you've only just received an offer, asking an agent to handle the negotiations might not be a bad thing.
What's your genre? If you like, I could post up some questions you can ask of B that you should know the answer to anyway.
