I note that many writers who decide to ditch their agent haven't actually talked to their agent about the concerns which have led them to this point.
In other words, try letting your agent know of your worries. Give her a chance to explain before you end the relationship.