Two agents, or choose?

Ideally, you want an agent who reps both (all) genres you write. And if you get picked up by one, you definitely need to mention the other book to him/her. Letting someone else handle it wouldn't be fair to your agent.

Is this placing the cart before the horse? Why worry about it before you get any offers?

The normal procedure is when one agent offers to notify the other one so that he/she can make a counter-offer. Agents call it a beauty contest when several of them are strutting their stuff in the hopes of landing a promising client.

And thinking ahead to be prepared for all eventualities is not putting the cart before the horse, it's just smart. Especially because this particular eventuality could theoretically happen at any time.

If both are "the same" (fiction, non fiction) then you only use one agent. Agents rep you, not a book. If they're separate (both fiction and non fiction), you can have two agents, though many have one agent who does both. If one agent picks you up for either book, just ask them what to do about the second project and they can help you decide what's best for your career.

Assuming the agent handles the type of book in question, you'll probably have to decide on one agent. Not many agents are willing to share a client, nor should they be. Every agent contract I've seen stipulates that the agent handles everything the writer writes.

The only reason an agent would not want to handle everything you write would be if they didn't handle a specific genre you write in. I know someone who has one agent for adult fiction and non-fiction, and another agent for children's picture books--but that arrangement is with the blessing of both agents, who don't work in each other's field at all.