Only can offer some advice from 15+ years ago, but the publishing world changes and doesn't change, as you have or will discover.
I had a children's book already published, and had written the second one, and the illustrator was all done. At the same time I sent out query letters to publishers in regard to the children's book, and to agents in regard to a novel I had written. I got lucky in both areas, and landed a publisher for the children's book and an agent for the novel.
When the publisher started negotiations, I knew I was way out of my depth, so I wrote my agent (then it was only snail mail) asking her if she would take over representing the book for me. She was fine with it, only made sure I understood that she would get her 15% to which I was totally cool with. Indeed, since I lived in Israel and both publisher and agent were in NYC it was well worth it.
She actually saved me from giving me away a few rights on the book and in the end it was the only book I actually sold through her services.
So my advice would also be, go for both. See who jumps first. If it is a decent publisher, most valid and bona-fide agents would be glad to represent you. After all you have done 90% of the work.
Good luck.
Teddy