I'm not sure I get the reasoning. If you had a good reason before for wanting to self-publish, then go for it, but if it's because you got two rejections, particularly when one isn't even a real rejection but a statement that you're submitting to the wrong genre, I don't get it at all. If you want to be commercially published, it's worth sending out to many, many more than two people. You also don't say whether or not you were submitting to agents or publishers.
Something else makes me wonder. One, if you're calling your book the wrong genre, at least submit it to some people under the proper genre before you decide to give up. I mean, what you've done is essentially taken a pair of shoes to a dress store and then been told they don't want your shoes because they only sell dresses. It doesn't say anything at all about your product.
Second, it makes me wonder if you're marketing this project wrong, which is something to be concerned about whether your self-publishing or not. If you're calling something fantasy when it's really sci-fi, or calling something a thriller when it's really a fantasy, that sort of thing, then you're not going to find your audience. People looking at your book will be disappointed to find that it isn't what they expect.
Third, this could also be an indication that you need to do more research and learn more about the other books in your field before you move on. Is this the first book you've written? Are you writing something in a genre that you don't normally read? I've written a book that kind of fell in between the cracks and those are really difficult to classify, but if you can find some books similar to yours that you can compare to, then you can see what those are called and try to appeal to their audience.
Hope this helps some.
As for several at once, I've heard many people say that your numbers increase the more books you have out. I don't know enough about that to say whether or not you would be better off writing more in the series or writing different books. The difficulty is that if the first book doesn't sell well, people aren't likely to buy the others. On the other hand, if the first is a really great book, having several books for sale might give you some added credibility and make readers more likely to take a chance on you.