You have to start with the understanding that genres are completely artificial constructs that are motivated by everything from classical Greek scholarship to library/bookstore shelving practices to P&Ls on individual novels or the desire of one or more publishers to market to a specific market segment at a given time. They aren't real things, they're tools of convenience.
Like all classifications, they have problems arising from their subjectivity. Like calling something "comfort food," for example. If I grew up in Pakistan, my definition of that term is different than if I grew up in Alabama.
And not everyone agrees with what the genre classifications are (strictly speaking, most people do not believe that YA and NA are, technically, genres, but they are still functionally treated that way). Their only real significance is that some agents and publishers categorize that way, some bookstores shelve that way and, in certain cases, specific "rules" (tropes, word counts, even advances and publishing terms for writers) have grown up around some of the more commercially produced genre books.
An incomplete list would include "Well-Defined Topic-Based Genres" [Mystery, Erotica, Sci Fi, Romance] and "Age Based Genres" [Children's, MG, YA, NA].
Then you have things that don't fit into those categories, which is where the more subjective genre calls come in. There is no magic number of bullets that separates a thriller from commercial or literary fiction. This is where the really fuzzy genres start coming into play. "Legal thriller?" Really? WTF is that? But you can use "thriller" and other designations to carve out fast-paced high-tension books into one or more genres. Humor/satire takes care of a number of the remaining books.
So now we're left with novels for grownups that aren't thrillers or humor books, but not all of them should be called literature (that's not smack-talk, this is the space I write in). Depending on their narrative arc (the 'hook,' the story that drives everything) and literary depth ('this is less about the story and more about the language or perspective or character), they are roughly categorized as Commercial Fiction, Up-Market, or Literary Fiction. And, yes, up-market is a category that exists solely because they wanted to find a way to assign a genre to the gray area between commercial and literary fiction.
More than anything, though, it's all a matter of subjective and arbitrary designation.
[To avoid inadvertently starting a flame war, I'm listing things in the order that makes them easiest to discuss in relation to one another, and not proclaiming anything superior based on where it shows up on this list].
