I'm not published, so maybe I don't understand how the business works. When a publisher picks up a writer's manuscript, do they hire the writer? Does the writer now work for that publisher, and does the writer get a W2 tax form from the publisher (or equivalent outside the US)? Does the publisher pay the writer regularly for work done to an agreed upon schedule? Same questions for agents -- when you sign the contract, do you become the agent's employee, or the other way around?
If writers do work for their publishers and agents, then yes, publishers and agents have some limited right to check up on how their writers are working, or goofing off, or farming out their tasks, etc.
But if not, then frankly, it's none of their business what the writer does on his/her own time, so long as the contracted work arrives on time, in publishable condition, and in accordance with the binding terms of the contract.
Granted, I can see checking up on someone before committing to an ongoing money-based relationship with them. One wants to make sure they are who they say they are, at the very least. It's only smart to want to know who one is doing business with, and if after checking them out, one decides not to do business with them, that's perfectly okay, too.
But frankly, I would object pretty strongly if I submitted, for instance, my real-world-setting horror WIP that I wrote all by myself, after fully crafting it to the best of my ability, with the input of betas and proofers, only to have it rejected because the editor or agent in question looked up my fantasy adventure collaboration project and decided they didn't like it because it was too raw and lacked a single consistent voice. My response would be, "Did I submit the fantasy collaboration for you to consider publishing? No? Then why did you read it instead of the book I sent you?" If I ask you to judge one project, do not judge another, entirely different project of your own choosing instead.
I have the same objection to any judgment that is not related to what I would be asking these professionals to do for me, such as judging my productivity and speed if I am not offering them a series, or judging my lifestyle or politics, etc., if they do not affect my career as a writer, and so forth. A publisher might want to make sure the children's book writer they are about to sign up is not a registered sex offender, for instance, but for the most part, unless something about the author would make it impossible to market the book, then it's not relevant, in my opinion, and if it's not relevant, then it's not their business.
That's just my take on all professional relationships. Maybe I'm wrong, but in other areas of life it hasn't hurt me any.