Query away. The goal is to find an agent and sell your work. Put your work out there when it's ready.
Space out the queries for the same agents though, because otherwise it can seem like you threw work together and are slapping it out there, and that can look bad. Leave at least three months between them, unless they asked to see the next project, then you can mention that. You don't want to send multiple projects at the same time to the same agent.
If you happen to get two offers for two different books, treat it just like you got two offers on the same book. Talk to all agents involved, decide who you like best and sign. Then you say "thanks so much but I went elsewhere," to the others and show your new agent the other manuscript. They'll likely be thrilled you have a second book ready to go. They'll probably be trying for a two-book deal minimum anyway. I believe that's pretty normal.
Just because you have two books ready to sell doesn't mean you'd release them both at the same time. They'd most likely space them out, or in rare cases, publishers have done a series a few months apart to build on momentum. Naomi Woulk's Dragon series did that.